The ump made the correct call.
If the first baseman is not near the bag and attempting to make a play on the runner then it is a balk. This assumes, of course, that the pitcher didn't step off of the rubber with his pivot foot prior to making the throw. If he stepped off first, no balk.