Sounds like you fully recognize the intangible values of a kid playing baseball with his high school friends. I understand your thinking that he may not develop or advance his skills much playing with his high school team. But I live in OC as well and there is no chance that he will not get to experience great competition and kids in his school and on the opponents that are not just as driven as kids on travel ball teams. Sure there are some kids who are just not that serious but they will drop out of the program by the time your son is on varsity. There is a ton of talented kids in SoCal and the high school games and rivalries are a huge part of a player' s experience.
I am not sure I understand the issue with the travel ball team and letting down the coach. The top travel ball teams in SoCal all shut down during high school baseball season. The only travel teams that I know of that play in the Spring are mostly filled with kids who could not make grades to stay on their high school teams or could not get along with their coach (or were not good enough to play on their HS team). I don't care how much talent those kids may have, that is not a group I'd want my son playing with. Your son has 6 months to play with his travel team in the Summer and Fall. That is more than enough time to improve and get exposure to college coaches.
Finally, I am not sure I understood the reference about you coaching with the high school coach? Are you an employee of the high school? Are you also a coach of the travel ball team?