I believe it's hard for the catcher to concentrate on receiving and blocking and everything he has to do and still see the hitters swing. When I'm calling pitches I record everything I see in his swing. From swinging around the ball to leaning over the plate to watch a low and away FB out of the zone. The catcher can't diagnose the swing and concentrate on the game.
As far as spray charts go, it's hard to get a good read on where to pitch a hitter if you don't know the location of the pitch on the spray chart. When calling pitches I am looking for holes in the swing more than anything. But like you said, if you can't hit spots it's all for naught.