I've always heard this also, and wondered why it's conventional wisdom. I certainly could never see a true grip no matter what glove the pitcher wore, and only went on the normal advice of seeing a pitcher "fiddle" with the grip of meaning some sort of offspeed was likely...even then I sometimes leveled myself thinking "what if he's just trying to find the seams" or "maybe he's doing that to make me think it's offspeed."
In addition, I always tried to lengthen my focus out into center field during the early stages of the windup, and pick up a release point to see the ball quickly, rather than focusing on the pitcher as a whole early.