Stolen base ?

Here 's what happened, HS rules,1 out, man on 1st.
pitcher comes set, runner on 1st goes, for whatever reason pitcher never moves, runner now on 2nd. Pitch is now thrown, line drive right back to the pitcher, he catches it and throws to 1st.Ump calls him out, double play, inning over. I argued that runner had legally stolen 2nd and the only base he had to return to was 2nd. Of course he told me I was mistaken. Who's is correct??