I have seen several posts about talking to the AD but I don't think I have read any about talking to the coach PRIVATELY. It seems like all the conversations have been at practice or a game. I have never coached beyond travel level but I always tell the parents that if they want to have a conversation, I will be glad to talk to them but not at practice or at games. No matter how quiet you talk or even if you move away from everyone it is still a public conversation and everyone is just going to guess and rumor about the conversation.
Make an appointment with the coach to clear the air. I don't mean go and complain but rather offer "coach what can I do as a parent to help my son meet your expectations." Don't comment on coaching style or motivation. Your goal is for your son to get better not be the coaches favorite. Give the coach a chance to voice his opinions in private and be willing to listen. You don't have to agree but be willing to hear what he has to say. Don't offer your side of the story unless specifically asked. If the coach makes general statements "tell your son to hit his spots" then ask " Are there drills you recommend that he should be doing to help that?" Ask the coach "Is there anything I do as a parent to help you?" Again you don't have to agree but being willing to listen. You may say "no just cheer the team" or "he may say just tell the rest of the parents to shut up" you can offer "how about if I just stay out of those conversations and not add to the fire." I guarantee you will gain more from the conversation by just listening and letting the coach vent then you will trying to change him.
As far as what started the original problem:
1) Your son should have stayed until coach said it was time to go. Coach was disciplining the team and you asked your son to skip it. Win and lose as a team. My son has played more than one flawless game and ended up running poles after the game with his teammates for their failure to execute. I kept my opinion of this to myself and reinforced to my son that you share the glory and the faults as a team. No one, no matter how good, is above that on a team.
2)The comments between you and coach after that first game should have been resolved the next day between you and coach. You have let this go on too long to go back and say "hey coach I am not happy about what you said 3 weeks ago.
Any conversations you have are between you and the coach. Don't go back to your son or even other parents to discuss your conversation. They don't even need to know you had a conversation with the coach. If the coach is still unreasonable and retaliates against your son then you will have a clear cut reason to leave the team. Again sharing your reasons with anyone outside your family is counter productive.