Truthfully, JB, I don't think it matters all that much. But you can't go wrong with the khakis,nice polo, casual (but not athletic) shoes look. Not too dressy for a ball game, but you don't have to worry about being shabby.
If you REALLY want to make an impression, stop into the gift store and sport a team hat the whole game!
(Just kidding, not necessary!)