Bigcat,
As Pirate Fan has pointed out, you made the correct call.
One other point to ponder is this: abandonment.
The Jaksa/Roder manual defines abandonment as follows: "A runner who discontinues his advance or return to a base, progresses a reasonable distance toward his dugout or defensive position, and indicates no intent to reassert his status as a runner, has abandoned his effort to run the bases. The cause of his actions are irrelevant."
As you can see in your play, abandonment was NOT an issue because as you say in your post the runner only took 4-5 steps and also the runner showed his intent to get back to second base once the ball was not gloved by the fielder. In 11 years, doing over 200 games per year, I have not once called a runner out for abandonment. There was one situation that I could have potentially called it per the rule. This is the situation:
Bottom of the 7th (7 inning game) home team down by 1. 2 outs. Bases loaded. Pitcher is unable to throw strikes and walks the batter. Home team is elated and everyone touches their correct bases except for R1 who gets half way and stops, then proceeds to celebrate with his teammates. Fortunately, the run scored before R1 actually abandoned the basepath. This becomes a time play, I never called the out and the visiting team was not paying attention as they could have appealed R1 never touched 2b which would have negated the run given it was a force out.
Just thought I'd throw in another possibility although I would like to reiterate that it would most certainly not come into play in the situation that you have presented.