I'm going to give you opposite advice. Little League can be magical (despite the politics, or maybe in spite?). The 12 year old year is the last year -- after this summer your son could move on to a higher level of travel ball. Our experiences are that if they want you when 12 they will likely want you when 13, especially a big kid who throws 70s already. My second son stayed loyal to the old friends in LL and was able to make it to the regionals. It was a lifetime of baseball memories, and something he had only dreamed of previously. After that summer he switched gears and has played high level travel since.
You are only young once! You age out so quickly from little league, and if your son is only 12 he has (at least) a couple of years before he would be on anyone's recruiting chart. Therefore practicing with a D1 coach who may not be there but a few years or at the facility may not matter just yet.
Is there a way to say no until next fall? Or join in after the state/regional tournaments once your LL season ends? There are PLENTY of competitive travel baseball programs, in fact, probably too many as it is watered down in many areas now. But there is only one Little League, in my opinion.