Unofficial Competition #1
The setup for the Official Competition in 2009 became the setup for the Unofficial Competition #1 in 2010. The rules here are straightforward: put the 22 professional forecasters (The Pros) into a league, and create a random draft order. And repeat this for one thousand leagues. The idea is that after so many leagues, we should reach consensus as to which Pro prefers which player. Here are the results:
| fan_id | points_ct | wins_ct | value_ct | fan_tx |
| 217 | 1491 | 420 | 6300 | Marcel |
| 108 | 1477 | 313 | 5782 | Chone |
| 115 | 1410 | 143 | 3344 | John Eric Hanson |
| 112 | 1322 | 12 | 967 | FantasyPros911 |
| 132 | 1314 | 16 | 866 | Fantistics |
| 299 | 1263 | 17 | 784 | Consensus |
| 118 | 1254 | 26 | 774 | Steamer |
| 131 | 1267 | 12 | 562 | Fangraphs Community |
| 105 | 1275 | 5 | 494 | Brad Null |
| 113 | 1291 | 4 | 448 | FeinSports.com |
| 116 | 1230 | 7 | 363 | KFFL |
| 120 | 1214 | 6 | 269 | Razzball |
| 126 | 1224 | 6 | 263 | Bloomberg Sports |
| 102 | 1147 | 1 | 48 | Ask Rotoman |
| 109 | 1110 | 0 | 23 | Chris Gehringer |
| 129 | 1144 | 0 | 17 | Wells Oliver |
| 106 | 1104 | 0 | 3 | CAIRO |
| 125 | 1123 | 0 | 1 | BigScoreSports |
| 111 | 957 | 0 | 0 | Fantasy Scope |
| 130 | 848 | 0 | 0 | Baseball Info Solutions |
| 127 | 1077 | 0 | 0 | Future of Fantasy |
This was a three-horse race, with Marcel winning 420 of the 1000 leagues. Chone was a very strong second, and last year's winner, Eric Hanson, is #3. Everyone else was pretty much an also-ran.
Unofficial Competition #2
In this competition, each league was made up of only two Pros, in a head-to-head matchup. Each Pro selected 275 players. Whereas in the previous competition, you would end up after 1000 leagues with about 100 players selected at some point, in this competition, the entire draft list is critical. Basically, each of these two competitions takes an extreme view as to the importance of the draft list. Here are the results:
| fan_id | points_ct | wins_ct | fan_tx |
| 115 | 14017 | 37 | John Eric Hanson |
| 299 | 14356 | 36 | Consensus |
| 132 | 13964 | 35 | Fantistics |
| 108 | 13990 | 33 | Chone |
| 113 | 13951 | 32 | FeinSports.com |
| 105 | 14061 | 30 | Brad Null |
| 116 | 13806 | 28 | KFFL |
| 118 | 13812 | 27 | Steamer |
| 131 | 13653 | 24 | Fangraphs Community |
| 120 | 13725 | 23 | Razzball |
| 217 | 13673 | 21 | Marcel |
| 102 | 13538 | 20 | Ask Rotoman |
| 112 | 13489 | 19 | FantasyPros911 |
| 126 | 13358 | 14 | Bloomberg Sports |
| 127 | 13389 | 14 | Future of Fantasy |
| 129 | 12836 | 11 | Wells Oliver |
| 106 | 12930 | 6 | CAIRO |
| 109 | 13094 | 6 | Chris Gehringer |
| 130 | 12667 | 6 | Baseball Info Solutions |
| 111 | 12313 | 2 | Fantasy Scope |
| 125 | 12623 | 0 | BigScoreSports |
Lots of strong showing, with Eric Hanson eeking out the overall win, with 37 wins in 42 leagues. Chone was also strong at 33 wins. We see that the Consensus (the consensus of the other 21 Pros) was extremely strong. Marcel was middle-of-the-road.
Unofficial Competition #3
In this competition, each league was made up of only one Pro and 21 random amateurs (The Joes). The idea is that this may better model reality, where you have one Pro in a league with random baseball fans. In order to simulate a Joe, I started with the consensus pick, assigned a dollar value to each player, and then randomly assigned a dollar value that was +/- 5$ within that initial dollar value. And for 5% of the players, I drastically reduced their value (effectively making those guys persona non grata to the Joes). Each Pro faced off against the same Joes and selected from the same draft position. There were 22 different leagues for each Pro. Here are the results:
| fan_id | points_ct | wins_ct | value_ct | fan_tx |
| 132 | 2283 | 20 | 230 | Fantistics |
| 115 | 2242 | 19 | 224 | John Eric Hanson |
| 299 | 2180 | 18 | 218 | Consensus |
| 113 | 2221 | 17 | 208 | FeinSports.com |
| 120 | 2184 | 17 | 204 | Razzball |
| 131 | 2174 | 16 | 202 | Fangraphs Community |
| 108 | 2168 | 13 | 183 | Chone |
| 105 | 2142 | 12 | 174 | Brad Null |
| 102 | 2103 | 11 | 164 | Ask Rotoman |
| 116 | 2117 | 11 | 164 | KFFL |
| 112 | 2082 | 10 | 157 | FantasyPros911 |
| 217 | 2078 | 10 | 153 | Marcel |
| 129 | 2078 | 10 | 152 | Wells Oliver |
| 126 | 2069 | 7 | 137 | Bloomberg Sports |
| 111 | 2034 | 6 | 122 | Fantasy Scope |
| 127 | 2014 | 6 | 114 | Future of Fantasy |
| 106 | 2019 | 5 | 113 | CAIRO |
| 109 | 1947 | 7 | 113 | Chris Gehringer |
| 118 | 1960 | 4 | 93 | Steamer |
| 130 | 1885 | 3 | 73 | Baseball Info Solutions |
| 125 | 1913 | 0 | 58 | BigScoreSports |
This time Fantistics won 20 of the 22 leagues, with Eric Hanson just behind in second place. The Consensus once again had a srong showing. Chone and Marcel were more middle-of-the-road this time. Also interesting is that The Pros, while only representing one of 22 teams in each league won 50% of the leagues. This was kinda fixed, seeing that I played with the 5% number I noted earlier to tip the scales toward The Pros. As we will see, that was a pretty good guess.
OFFICIAL COMPETITION
In this competition, each league was made up of only one Pro and 21 random Fangraphs readers (The Joes). The idea is that this may better model reality, where you have one Pro in a league with actual random baseball fans. Each Pro faced off against the same Joes and selected from the same draft position. There were 22 different leagues for each Pro. Here are the results:
| fan_id | points_ct | wins_ct | value_ct | fan_tx |
| 115 | 1662 | 17 | 210 | John Eric Hanson |
| 116 | 1588 | 16 | 201 | KFFL |
| 217 | 1535 | 15 | 187 | Marcel |
| 118 | 1617 | 15 | 185 | Steamer |
| 108 | 1535 | 15 | 182 | Chone |
| 132 | 1607 | 12 | 172 | Fantistics |
| 299 | 1535 | 12 | 169 | Consensus |
| 120 | 1584 | 12 | 166 | Razzball |
| 113 | 1514 | 11 | 161 | FeinSports.com |
| 112 | 1509 | 11 | 159 | FantasyPros911 |
| 105 | 1504 | 11 | 158 | Brad Null |
| 109 | 1510 | 12 | 156 | Chris Gehringer |
| 131 | 1461 | 11 | 151 | Fangraphs Community |
| 127 | 1462 | 12 | 150 | Future of Fantasy |
| 129 | 1462 | 9 | 140 | Wells Oliver |
| 126 | 1475 | 6 | 129 | Bloomberg Sports |
| 102 | 1366 | 5 | 74 | Ask Rotoman |
| 111 | 1361 | 3 | 59 | Fantasy Scope |
| 130 | 1319 | 3 | 52 | Baseball Info Solutions |
| 125 | 1331 | 1 | 37 | BigScoreSports |
| 106 | 1280 | 0 | 13 | CAIRO |
Eric Hanson, last year's winner, wins again! He won 17 of the 22 leagues, just ahead of KFFL's 16. In a three-way tie with 15 wins was Marcel, Steam, and Chone.
The 22 Pros won 45% of the leagues, even though they only represented 5% of the teams. With 484 teams (Pros + Joes) and 484 leagues, the average is for each team to have won once. And 21 of the 22 Pros reached that level. Only 75 of the 464 Fangraphs Readers (16%) won at least once. The key point here is that just about ANY system is better than NO system. Basically, a baseball fan knows enough information to be dangerous.
Overall Results
As you can see, Eric Hanson was in contention regardless of the kind of rules setup I had. Marcel on the other hand either did great, or was average. Which competition setup do you think is more relevant? Well, that's for you to decide, which is why I ran so many different kinds of setups. The reader is free to decide for himself what best models reality. Let me also present for convenience the overall results. What I did was that for each competition, I found the mean and standard deviations of the scores, and turned them into z-Scores. Below you will find these results as a simple average:
| ALL | Pros | H2H | Joes | Readers | fan_id | fan_tx |
| 1.31 | 1.31 | 1.32 | 1.31 | 1.28 | 115 | John Eric Hanson |
| 1.24 | 2.68 | 0.99 | 0.50 | 0.78 | 108 | Chone |
| 0.94 | 2.97 | 0.00 | -0.10 | 0.87 | 217 | Marcel |
| 0.78 | -0.08 | 1.15 | 1.43 | 0.60 | 132 | Fantistics |
| 0.71 | -0.12 | 1.24 | 1.20 | 0.54 | 299 | Consensus |
| 0.50 | -0.31 | 0.91 | 1.00 | 0.40 | 113 | FeinSports.com |
| 0.36 | -0.36 | 0.58 | 0.12 | 1.12 | 116 | KFFL |
| 0.29 | -0.41 | 0.16 | 0.92 | 0.49 | 120 | Razzball |
| 0.28 | -0.29 | 0.74 | 0.32 | 0.34 | 105 | Brad Null |
| 0.27 | -0.25 | 0.25 | 0.88 | 0.22 | 131 | Fangraphs Community |
| 0.04 | -0.02 | -0.16 | -0.02 | 0.36 | 112 | FantasyPros911 |
| -0.02 | -0.13 | 0.49 | -1.29 | 0.83 | 118 | Steamer |
| -0.37 | -0.55 | -0.82 | -0.12 | 0.02 | 129 | Wells Oliver |
| -0.40 | -0.42 | -0.58 | -0.41 | -0.18 | 126 | Bloomberg Sports |
| -0.42 | -0.54 | -0.08 | 0.12 | -1.18 | 102 | Ask Rotoman |
| -0.45 | -0.56 | -0.58 | -0.87 | 0.20 | 127 | Future of Fantasy |
| -0.59 | -0.55 | -1.24 | -0.89 | 0.31 | 109 | Chris Gehringer |
| -1.07 | -0.56 | -1.57 | -0.71 | -1.45 | 111 | Fantasy Scope |
| -1.24 | -0.56 | -1.24 | -0.89 | -2.28 | 106 | CAIRO |
| -1.26 | -0.56 | -1.24 | -1.68 | -1.57 | 130 | Baseball Info Solutions |
| -1.53 | -0.56 | -1.73 | -1.98 | -1.84 | 125 | BigScoreSports |
In some cases, like Eric Hanson, he simply showed consistent excellence across the board, always at around 1.3 standard deviations from the mean. Marcel showed the biggest range, either far above the mean at 3 standard deviations in one setup, or right around the mean in two cases. Overall, this put Marcel as #3 among the 22 forecasters, with Chone placing very strongly at #2.
Thank you again to all the Pros and Fangraphs readers that made this competition possible. Everyone really stuck their necks on the line and put in their time to submit their draft lists. See you next year!