Soccer seems to be the most agnostic to size/height. I imagine rugby, lacrosse, ice hockey, and field hockey might be similar. Running too esp distance running.
In response to the original question: is baseball different in a way that makes more young people want to play it in college? Probably not.
But…there is a potential pot o’ gold at the end of the baseball rainbow. Not so for swimming and most other sports.
A better comparison might be football or basketball.