Ill throw a curve in here and make a separate category for the position that has the highest difference between how difficult people believe it to be, and how hard it actually is to play it. For that odd category, I think first base and third base fit the best. Maybe its because you can see the greatest variation there from just ok players to truly great ones, imo.
I'm not going to argue about catcher being difficult to play. Very physically demanding with upper level skills required. The psychological makeup at catcher and pitcher are more demanding than any other position since they are in on every play. Shortstop ranks third in psychological demands, since you are going to make the most errors of anyone on the team.