I do get a chuckle out of the "just get a radar gun and a folding chair" comments. They are funny, but a bit far from reality. PG Staff pretty much nails it with his comments. I'll offer a couple examples of what constitutes a little bit more of what goes into scouting players.
Just today, I saw a kid that I'll be sending an email about, to the area scout I'm associated with. This kid is someone he saw two years ago, when he was scouting Lars Anderson. This kid did very well that day, and he began following him then. He's now a juco pitcher, a redshirt freshman who suffered an arm injury that shut him down last year. Today was only his third day back on the mound since that injury. He went 3 solid innings, which is what he was scheduled to go. He threw very well, so they ran him out there for a 4th inning. He isn't in game shape yet, and it showed in the 4th. He left a few balls up and they got hit hard. Till that time, he sat upper 80s, touching low 90s a few times. Had solid command of his FB, with some run to it. Curveball was outstanding today. Command to both sides in all counts, change thrown well too. Pitched aggresively inside. The other stuff that goes into it, though, was talking to his mother while he was pitching about her son's health, and also talking to his pitching coach about his progress and what might be expected as the year goes on. Also talked to the kid while he was getting wrapped with ice about a couple of key pitches, how his arm felt, and what kind of shape he considered himself to be in at that point. I already know a lot about the kid's character and signability, as I've known him for about 7 years. One of the biggest keys for me today was that I saw him compete better than I've known him to do when he was a bit younger. THAT is a little bit of what goes into evaluating players.
Here's another example of what a scout would look for. In December 2005, we were down in Australia with Bob Williams and Goodwill Series. One of our players was hit right on the ear flap with an upper 80s fastball. He went down like he'd been shot with a gun, and didn't get up for several minutes. When he finally got up, he wouldn't come out of the game. He walked to 1st, and then stole 2nd on the first pitch. A few pitches later, he stole 3rd and scored on a sac fly that was way too shallow for anyone to come home on. Next at bat, he smokes the 1st pitch he sees way out of the yard(remember, they're hitting with wood on Bob's trips). I was sitting next to Bob down the line past the dugout, and when the young man got done being congratulated by his teammates, Bob motioned for him to come down to talk. First question Bob asked was "What was the pitch you just hit out?" The player answered "Four seam fastball, sir, but the pitch that hit me was a two seamer." This is an example of seeing a player who has "IT", whatever IT is. You can't coach what this kid naturally has, and it was instantly obvious when you saw what he did, and heard what he said. That young man's name is Trayvon Robinson and he got drafted, straight out of high school six months later in, I beleive, the 10th round.
