I compared PGCrossChecker's "Draft 2006 State-by-State Follow List" for Georgia with the actual 2006 draft results. I only looked at high school players. PGCrossChecker (PGCC) did pretty well in the upper rounds and, not so surprisingly, not so well in the lower rounds.
PGCC broke down its pre-draft rankings into four groups:
Group 1 -- Projected rounds 1-3
Group 2 -- Projected rounds 4-8
Group 3 -- Projected rounds 9-20
Group 4 -- "Chance draft/Player to watch"
PGCC listed one HS player (Cedric Hunter)in Group 1 (projected in rounds 1-3). Consistent with PGCC's projection, he was, in fact, drafted in the 3rd round.
PGCC listed 6 players in Group 2 (projected rounds 4-8). Four of those 6 were, in fact, drafted -- 2 in the 2nd round (Brent Brewer and Tom Hickman) and 2 in the 3rd (Nick Fuller and Torre Langley). One of the 2 who was not drafted was Michael Demperio, but Baseball America's draft blog quoted his dad as saying, "We turned down the Braves' first pick in the second round." (He went to Texas instead). Assuming this to be true, at least 5 of the 6 players listed in Group 2 were evaluated/drafted as 2nd/3rd round talents by MLB teams. (NOTE -- The other undrafted player, Ben Paulsen, had originally signed with a JUCO, Young Harris, but around the time of the draft signed with Clemson, so there may have been signability issues there as well). Thus, the bottom line with Group 2 is that PGCC may have under-rated them, but only slightly.
PGCC listed 19 players in Group 3 (projected rounds 9-20). 7 of the 19 were drafted. That seems like a decent result for PGCC, considering the signability issues with HS players in those rounds. The 7 went in the 20th, 21st, 29th, 32nd, 34th, 35th and 48th rounds, respectively. Thus, with one exception (the 20th rounder), they were all picked in a lower round than projected. The biggest "miss" for PGCC in this group was an 11th rounder, Brandon Rice, who didn't appear on PGCC's lists at all, even as a Group 4 possible "chance pick".
PGCC did not fare too well with Group 4 ("chance draft"). They listed 53 players in this Group, of whom 5 were drafted. That's not bad in itself. But there were 12 other players who were NOT on PGCC's list who were drafted. To PGCC's credit, the 12 were all lower round picks (i.e., rounds 23 to 50). Of those 12, 9 had never attended a PG showcase. Of the 3 who had, 2 were rated by PG as "9's" and one as a 7.5. The 7.5 player was the very last pick in the draft (#1502). According to an article on mlb.com, that player's father is a part-time scout for the Cards, which is the team that picked him. (PGCC could easily be excused for missing that one).
Looking at total numbers, PGCC listed 79 Georgia HS players as possible draft picks (including 53 "chance draft/players to watch"). 30 Georgia HS players were actually drafted, of whom 12 were not listed as even "chance draft/players to watch" by PGCC.
The morals of the story --
1) PGCrossCheckers did a very good job at the upper levels of the draft. There was not a single Georgia HS player taken in the top 10 rounds who was not identified by PG.
2) The most significant "miss" was a player drafted in round 11 who was not identified as even a "chance draft" player.
3) PG didn't do all that well below round 20. There were a number of players who were obviously on the radar screens of MLB clubs that were not on PG's list. A disproportionate number of those players had never attended a PG showcase.
Thus, to address the question that started this post -- how useful are rankings? -- as to the PG pre-draft rankings, I'd say they're useful in identifying the top 10 round talent. After that, there are plenty of players who fall through the cracks and yet get picked up in the draft.