Well, the OBR and NCAA rule books are not so very clear on this issue. Fortunately, NFHS is definite that for the award to be TOP, it must have been the first throw by an infielder who has just fielded a batted ball.
There are a few resources that umpires use in addition to the rule book. One of those is referred to as Jaksa/Roder, and their interpretation matches the NFHS rule: the infielder must have fielded the batted ball.
However, all rule sets state quite clearly that if the batter and all other runners have advanced one base at TOT, then the award is unconditionally from TOT. And I imagine that in your situation, if there were any other runners, they too would have reached the next base before the bad throw was made. If so, your umpire was clearly wrong.
7.05(g) Approved Ruling is: If all runners, including the batter-runner, have advanced at least one base when an infielder makes a wild throw on the first play after the pitch, the award shall be governed by the position of the runners when the wild throw was made. Note the phrase "on the play after the pitch". Although the approved ruling doesn't directly refer to the situation in which some runner hasn't reached his advance base at TOT, it helps to clarify the rule makers intent.
In summary, for the award to be TOP, it has to be the first throw folowing the pitch. And if all runners have advanced one base at the time of the errant throw, the award is always TOT.