I think the reason you see coaches acting that way is because it is generally an advantage in baseball to be bigger and stronger. The average adult male in the U.S. is 5'9", the average MLB player is listed at 6'2".
That's not to say smaller kids can't play. In fact, one of the Vandy commits in the 2019 class is a pitcher listed at 5'10". But he is a lefty.
I look at it like speed. In general, it's an advantage in baseball to be able to run faster. I'd be willing to wager that the average MLB 60-yard dash (6.9 - 7.0) is way, way faster than the average among the population at large. But, still, there are some slow guys who can play.