I understand protecting your kid, but just to address a few things. Abuse is a very strong word. There is a very big difference between a coach leaving your son in for a little too long and abusing him. In todays climate, that word getting tossed around can cost coaches their jobs and reputation. Don't take it lightly.
The second thing is that 50 pitches is nothing. 50 pitches in one inning. Bad. 50 pitches in a game. Golden. Please do not pull your son off the field after 50 pitches. Not a single person that knows anything about baseball will agree with you. Little League is an 85 pitch limit and many think they can expand that a bit further. I think its just right.
To add to that. Arms are most durable in the elementary/middle school ages. When people say they threw a ball from 9am to 5pm all day, they weren't lying. I understand the concern with arm issues, but the odds of him developing shoulder issues at 11 would be most likely from a freak accident or the actual result of arm abuse. If you are that concerned, possibly play summer or fall without pitching. Arm injuries suck, but they are not concussions. They aren't dangerous to your health.
I'm not trying to tell you how to raise your kid, but you are not going to get a ton of people agreeing with you here. You said your husband was a pro baseball player. Trust his judgement. FWIW I think throwing more and playing everyday as a kid makes you better and helps produce a stronger arm.