piaa_ump,
I think the OP is asking about a situation in which the pitcher is not ambidextrous.
There is a PBUC ruling 6.15 Ambidextrous Pitcher, which says each can switch once. But I don't remember seeing a rule disallowing the batter from switching (assuming he isn't delaying by stepping out of the box) if the pitcher hasn't switched.
Here's an article about a recent stand-off. In it, Paul Klemm points out that rule 6.15 is incomplete, since it doesn't specify which player must choose first.
Ambidextrous VenditteI think it was subsequently decided that an ambidextrous pitcher must choose first.
But back to the OP, I believe in OBR there is no direct prohibition on a batter switching if the pitcher doesn't, and he isn't delaying the game.