I'm going to say that's a good sign. Lots of these guys know when they don't like a kid in less than an inning. If he stayed for 4, then he probably wanted to take a good long look.
I believe your son is a HS sophomore. FYI, as far as NCAA rules go, parent and student are the same entity. The coach (D1/D2) can't talk to any of you in that setting (away from his campus). Mom would have put him in kind of a bad spot if she started up a conversation.