Sounds like you're trying to track the ball with your hand. You have to trust the break will be in a spot around the plate and let it come to you. Don't attack the pitch.
Couple of drills or things to focus during drills which may help.
1. Take your mitt off and have a partner toss balls at you and learn to catch the ball with index, middle fingers and thumb. Work on receiving the pitch softly and learn to trust your hands.
In this drill have the partner throw them straight at you to resemble fastballs and then throw them up with a hump so you are catching them on the way down.
2. Volunteer to catch as many bullpens as you can. Learn to trust the break and just recognize what you do wrong when you make a mistake. Then fix it.
3. Do as TR says - practice, practice, practice