On this site, I quote only NFHS rules. If your area uses NCAA or OBR the answer can differ..
From NFHS rules the answer is yes, they can.
Here is the rule citation:
NFHS rule 3 Section 1 Article 1 (D) NOTE
(Condensed)
A substitute has entered the game when the ball is alive and
d. A pitcher takes his place on the pitcher's plate
So since all warm up pitches are taken during a dead ball, the player can be replaced prior to the umpire making the ball alive at the start of the inning without having to pitch to a batter. Only the starting pitcher must pitch to a minimum of one batter before being removed.