Mechanically Separated Meat

CLICK HERE TO JUMP TO THE PRETTY COLOR-CODED FULL RESULTS

This explanation will look a lot like last year’s, but:

Every year since 2004 I’ve been hosting this Game of the Year poll for the users of some forums I read. There are a lot of GOTY polls out there, but this one I think is kind of special. Most polls, you’re given a list of four or five options and you’re asked to pick the one you liked best. This poll, people are given a list of a couple of hundred options, consisting of every new game released in the previous year– and asked to rate their top ten or twenty.

This does a few interesting things. First off, we get to see all the information about what people’s second, third etc choices are. Second off, because the second, third etc choices count, people are more likely to vote for the game they want to win, rather than the game they think is likely to win– they’re less likely to engage in “strategic voting”. Finally, because we have all this information, we’re actually able to provide somewhat reasonable rankings for something like the top hundred or so games of last year.

The full results– showing the exact number of voters who ranked each game first, second, third place etc– can be found here. In the meantime, the final results were:

1. Batman: Arkham Asylum (7772) *** GAME OF THE YEAR ***
2. Dragon Age: Origins (7019)
3. Borderlands (5579)
4. Call of Duty: Modern Warfare 2 (4458)
5. Left 4 Dead 2 (4295)
6. Assassin’s Creed II (4205)
7. Uncharted 2: Among Thieves (3858)
8. Torchlight (3792)
9. New Super Mario Bros. Wii (3386)
10. Resident Evil 5 (2941)
11. Street Fighter IV (2576)
12. Shadow Complex (2460)
13. Demon’s Souls (2342)
14. Halo 3: ODST (2104)
15. Brütal Legend (2008)
16. The Beatles: Rock Band (1991)
17. Infamous (1844)
18. Plants vs. Zombies (1773)
19. Scribblenauts (1752)
20. Prototype (1720)

The numbers in parentheses are the final scores each game got under the poll’s ranking system. Thanks if you voted, and some more elaborate analysis of the results (plus an explanation of the scores) can be found below.

NOTEWORTHY WINNERS

• GOTY 2009:
  

  #1, Batman: Arkham Asylum

• Top-ranked PS3 Exclusive:
  

  #7, Uncharted 2: Among Thieves

• Top-ranked PC Exclusive:
  

  #8, Torchlight

• Top-ranked Wii Exclusive:
  

  #9, New Super Mario Bros. Wii

• Top-ranked 360 Exclusive:
  

  #12, Shadow Complex

• Top-ranked DS Exclusive:
  

  #19, Scribblenauts

• Top-ranked PSP Exclusive:
  

  #46, Dissidia: Final Fantasy

• Top-ranked iPhone Exclusive:
  

  #56, Canabalt

• Best RPG:
  

  #2, Dragon Age: Origins

• Best FPS:
  

  #3, Borderlands

• Best Game Only Available Through A Console Download Service:
  

  #12, Shadow Complex

• Best “Indie” Game? (I’m not even sure I know what that word means.):
  

  #18, Plants vs Zombies

NOTEWORTHY LOSERS

• Best game of 2009 which somehow nobody considered to be their #1 pick: #30, Punch-Out!!
• Worst game of 2009 that at least one person considered their #1 pick: #248, Harvest Moon: Animal Parade (Only one person voted for this at all)
• Worst game of 2009: #284, Mario & Sonic at the Olympic Winter Games (Only one person voted for this; it was their #20 pick)

There were 17 games on the nominations list no one voted for at all. Also, FIFA 10– which was left off the nominations list by complete accident– probably deserves some kind of special “moral victory” award for the sheer number of people who were upset about its absence.

ALTERNATE SCORING METHODS

The rankings listed above are based on what was originally intended to be an approximation of Condorcet voting, but which I’m told is actually closer to the Borda count. In my Borda-ish voting method, each vote cast for a game gives that game a certain number of points. If someone ranks a game #1, that game gets 20 points. If they rank it #2, the game gets 19 points. If they rank it #3 the game gets 18 points… and so on. I have a script that checks a couple of alternate ways of ranking the same data, though.

For example, if we rank games only by the number of first post votes they got, we get a wildly different list, with the most obvious difference being Batman and Dragon Age swapping places:

First Past the Post

1. Dragon Age: Origins (134)
2. Batman: Arkham Asylum (119)
3. Uncharted 2: Among Thieves (111)
4. Borderlands (58)
5. New Super Mario Bros. Wii (45)
6. Left 4 Dead 2 (45)
7. Assassin’s Creed II (41)
8. Demon’s Souls (41)
9. Call of Duty: Modern Warfare 2 (32)
10. Street Fighter IV (25)
11. Brütal Legend (14)
12. The Beatles: Rock Band (13)
13. Shadow Complex (12)
14. Torchlight (12)
15. Resident Evil 5 (11)
16. Halo 3: ODST (10)
17. Muramasa: The Demon Blade (9)
18. Warhammer 40,000: Dawn of War II (8)
19. Red Faction: Guerrilla (7)
20. Forza Motorsport 3 (7)
21. Infamous (7)
22. Little King’s Story (7)
23. Machinarium (7)

Most years when I look at the first-past-the-post list a “cult” game usually emerges that received very few overall votes, but where an overwhelming percentage of those votes were #1 votes; this year there was no obvious leader in the “cult” category, although the jump in ranking for Demon’s Souls seems pretty significant, and Murasama (which jumps from #37 to #17) and Machinarium and Little King’s Story (which jump from #40 and #48 to a five-way tie for 19th place) seem worth mentioning.

I also did two more ways of sorting the rankings: an “approval” vote, where nothing is counted except the number of votes a game received (i.e. a first-place and a twentieth-place ranking count the same– all the matters is if the game was on someone’s list); and an instant runoff vote. Your eyes are probably starting to glaze over at this point, so I bolded the places where these two votes differ from the normal rankings:

FINALLY: PER-FORUM BREAKDOWNS

As mentioned before, this poll mostly exists for a handful of video game forums where some people I know post. Since a couple years ago when I started posting the results on this blog, I’ve tried to actually run some extra results, in each case counting only those voters who– as far as one could tell from looking at the logs– had come to the poll from one particular forum or other.

So, here you have it– these numbers aren’t totally accurate because my logging method is not entirely trustworthy, but here’s an approximate by-forum breakdown of these results. Links go to color-coded full listings.

CLICK HERE TO JUMP TO THE PRETTY COLOR-CODED FULL RESULTS

I’m just gonna copy and paste the explanation I gave last year:

For the last few years I’ve been hosting this Game of the Year poll for the users of some forums I read. There are a lot of GOTY polls out there, but this one I think is kind of special. Most polls, you’re given a list of four or five options and you’re asked to pick the one you liked best. This poll, people are given a list of a couple of hundred options, consisting of every new game released in the previous year– and asked to rate their top ten or twenty.

This does a few interesting things. First off, we get to see all the information about what people’s second, third etc choices are. Second off, because the second, third etc choices count, people are more likely to vote for the game they want to win, rather than the game they think is likely to win– they’re less likely to engage in “strategic voting”. Finally, because we have all this information, we’re actually able to provide somewhat reasonable rankings for something like the top hundred or so games of last year.

The full results– showing the exact number of voters who ranked each game first, second, third place etc– can be found here. In the meantime, the final results were:

1. Fallout 3 (8780) *** GAME OF THE YEAR ***
2. Left 4 Dead (6626)
3. Grand Theft Auto 4 (5032)
4. Super Smash Bros. Brawl (4321)
5. Rock Band 2 (3290)
6. Dead Space (3151)
7. Gears of War 2 (2942)
8. Fable 2 (2751)
9. Braid (2729)
10. Metal Gear Solid 4 (2666)
11. Little Big Planet (2520)
12. No More Heroes (2241)
13. Audiosurf (2152)
14. Castle Crashers (2083)
15. Valkyria Chronicles (2027)
16. Mario Kart Wii (2014)
17. The World Ends with You (2000)
18. World of Warcraft: Wrath of the Lich King (1914)
19. Penny Arcade Adventures: On The Rain-Slick Precipice of Darkness Ep. 1 (1910)
20. Sins of a Solar Empire (1850)

The numbers in parentheses are the final scores each game got under the poll’s ranking system. (The scores in general were a lot closer than last year–basically all the rankings 14-18 are within a couple votes of each other!) Thanks if you voted, and some more elaborate analysis of the results (plus an explanation of the scores) can be found below.

NOTEWORTHY WINNERS

• GOTY 2008:
  

  #1, Fallout 3

• Top-ranked Wii Exclusive:
  

  #4, Super Smash Bros. Brawl

• Top-ranked 360 Exclusive:
  

  #7, Gears of War 2

• Top-ranked PS3 Exclusive:
  

  #10, Metal Gear Solid 4

• Top-ranked PC Exclusive:
  

  #13, Audiosurf

• Top-ranked DS Exclusive:
  

  #17, The World Ends With You

• Top-ranked PSP Exclusive:
  

  #39, Crisis Core: Final Fantasy VII

• Best FPS:
  

  #2, Left 4 Dead

• Best RPG:
  

  #1, Fallout 3

• Best Sports Game:
  

  #27, Burnout Paradise

• Best Game Only Available Through A Console Download Service:
  

  #8, Braid

• Special “Cult” Award (see below):
  

  #26, Persona 4 & #15, Valkyria Chronicles (Tie)

NOTEWORTHY LOSERS

• Best game of 2008 which somehow nobody considered to be their #1 pick: #33, Spore
• Worst game of 2008 that at least one person considered their #1 pick: #179, Midnight Club: Los Angeles (Only two people voted for this)
• Worst game of 2008: #203, Mystery Case Files: MillionHEIR (Only one person voted for this; it was their #20 pick)

There were also ten games which were listed, but which no one voted for at all.

ALTERNATE SCORING METHODS

The rankings listed above are based on what was intended to be an approximation of Condorcet voting, but which I’m told is actually closer to the Borda count. In my Borda-ish voting method, each vote cast for a game gives that game a certain number of points. If someone ranks a game #1, that game gets 20 points. If they rank it #2, the game gets 19 points. If they rank it #3 the game gets 18 points… and so on. I have a script that checks a couple of alternate ways of ranking the same data, though.

For example, if we rank games only by the number of first post votes they got, we get a wildly different list:

First Past the Post

1. Fallout 3 (182 first-place votes)
2. Left 4 Dead (109)
3. Super Smash Bros. Brawl (42)
4. Metal Gear Solid 4 (42)
5. Valkyria Chronicles (39)
6. Persona 4 (39)
7. Grand Theft Auto 4 (35)
8. Dead Space (31)
9. Rock Band 2 (30)
10. The World Ends with You (30)
11. World of Warcraft: Wrath of the Lich King (30)
12. Gears of War 2 (23)
13. Little Big Planet (21)
14. No More Heroes (19)
15. Braid (15)
16. World of Goo (14)
17. Spelunky (12)
18. Sins of a Solar Empire (11)
19. Fable 2 (10)
20. Prince of Persia (10)

Every year when I do this there’s some game which scores horribly low in the objective rankings but gets a really startling proportion of first-place votes; last year the standout game in the “cult” department was Persona 3; this year the standout was, interestingly enough, Persona 4, which only got 87 votes at all, placing it at #26 in the overall rankings– but nearly half of those votes, a full 39, ranked it in first place, putting it in sixth place in the First Past the Post ranking above. Tying Persona 4 in the First Past the Post ranking is Valkyria Chronicles, which did a little better in terms of how many people voted for it (117 votes) but which still gets a pretty great cult ranking since one in three of those voters considered it their #1 game. (Honorable mention in the cult category should probably go to “Spelunky“, a wildly obscure but kind of awesome freeware pixel art game released in the last two weeks of December, which came in way down at 51st place in the overall rankings but managed to come in 17th in first-pace votes– with again nearly one-third of the people who voted for Spelunky at all rating it #1.)

I also did two more ways of sorting the rankings: an “approval” vote, where nothing is counted except the number of votes a game received (i.e. a first-place and a twentieth-place ranking count the same– all the matters is if the game was on someone’s list); and an instant runoff vote. Most years I’ve done this the Instant Runoff and pseudo-Borda rankings have been almost the same, but this time there were some interesting differences (with the biggest one being, for some reason I don’t understand, World of Goo somehow jumping a good seven spots in the rankings?!). Your eyes are probably starting to glaze over at this point, so I bolded the places where these two votes differ from the normal rankings:

FINALLY: PER-FORUM BREAKDOWNS

As mentioned before, this poll mostly exists for a handful of video game forums where some people I know post. Since last year when I started posting the results on this blog, I’ve tried to actually run some extra results, in each case counting only those voters who– as far as one could tell from looking at the logs– had come to the poll from one particular forum or other.

So, here you have it– these numbers aren’t totally accurate because my logging method is not entirely trustworthy, but here’s an approximate by-forum breakdown of these results. Links go to color-coded full listings.

CLICK HERE TO JUMP TO THE PRETTY COLOR-CODED FULL RESULTS

So for the last few years I’ve been hosting this Game of the Year poll for the users of some forums I read. There are a lot of GOTY polls out there, but this one I think is kind of special. Most polls, you’re given a list of four or five options and you’re asked to pick the one you liked best. This poll, people are given a list of a couple of hundred options, consisting of every new game released in the previous year– and asked to rate their top ten or twenty.

This does a few interesting things. First off, we get to see all the information about what people’s second, third etc choices are. Second off, because the second, third etc choices count, people are more likely to vote for the game they want to win, rather than the game they think is likely to win– they’re less likely to engage in “strategic voting”. Finally, because we have this information, we’re actually able to provide somewhat reasonable rankings for something like the top hundred or so games of last year.

The full results– showing the exact number of voters who ranked each game first, second, third place etc– can be found here. In the meantime, the final results were:

1. Portal (9532) *** GAME OF THE YEAR ***
2. Bioshock (8004)
3. Super Mario Galaxy (7968)
4. Mass Effect (5874)
5. Team Fortress 2 (5256)
6. Call of Duty 4 (5051)
7. Halo 3 (4848)
8. Half Life 2: Episode 2 (4660)
9. Rock Band (3788)
10. Guitar Hero 3 (2968)
11. Metroid Prime 3: Corruption (2948)
12. Assassin’s Creed (2937)
13. Legend of Zelda: Phantom Hourglass (2605)
14. World of Warcraft: Burning Crusade (2324)
15. Super Paper Mario (2215)
16. Pokemon Diamond/Pearl (2083)
17. Crackdown (1978)
18. Puzzle Quest: Challenge of the Warlords (1773)
19. Phoenix Wright: Ace Attorney – Justice for All (1621)
20. Zack and Wiki (1453)

The numbers in parentheses are the final scores each game got under the poll’s ranking system. (Bioshock and Galaxy were close!, as were Guitar Hero, Metroid, and Assassin’s Creed.) Thanks if you voted, and some more elaborate analysis of the results (plus an explanation of the scores) can be found below.

NOTEWORTHY WINNERS

• GOTY 2007:
  

  #1, Portal

• Top-ranked 360 Exclusive:
  

  #4, Mass Effect

• Top-ranked Wii Exclusive:
  

  #3, Super Mario Galaxy

• Top-ranked PS3 Exclusive:
  

  #34, Uncharted: Drakes Fortune

• Top-ranked PC Exclusive:
  

  #14, World of Warcraft: Burning Crusade

• Top-ranked DS Exclusive:
  

  #13, Legend of Zelda: Phantom Hourglass

• Top-ranked PSP Exclusive:
  

  #56, Castlevania: Dracula X Chronicles

• Top-ranked GBA Exclusive:
  

  #100, Legend of Spyro: Eternal Night

• Best FPS:
  

  #1, Portal

• Best RPG:
  

  #4, Mass Effect

• Best Sports Game:
  

  #27, Forza Motorsport 2

• Best Game Only Available Through A Console Download Service:
  

  #49, Sin and Punishment

• Special “Cult” Award (see below):
  

  #25, Persona 3

NOTEWORTHY LOSERS

• Best game of 2007 which somehow nobody considered to be their #1 pick: #19, Phoenix Wright: Ace Attorney – Justice for All
• Worst game of 2007 that at least one person considered their #1 pick: #187, Hammerfall (Only two people voted for this)
• Worst game of 2007: #208, SNK vs Capcom Cardfighters DS (Only one person voted for this; it was their #19 pick)

There were also five games which were listed, but which no one voted for at all.

ALTERNATE SCORING METHODS

The rankings listed above are based on an approximation of Condorcet voting. In my pseudo-Condorcet approximation, each vote cast for a game gives that game a certain number of points. If someone ranks a game #1, that game gets 20 points. If they rank it #2, the game gets 19 points. If they rank it #3 the game gets 18 points… and so on. I have a script that checks a couple of alternate ways of ranking the same data, though.

For example, if we rank games only by the number of first post votes they got, we get a slightly different list:

First Past the Post

1. Portal (176 first-place votes)
2. Super Mario Galaxy (170)
3. Mass Effect (105)
4. Bioshock (88)
5. Rock Band (48)
6. Team Fortress 2 (44)
7. Call of Duty 4 (39)
8. Halo 3 (27)
9. Persona 3 (20)
10. World of Warcraft: Burning Crusade (14)
11. Metroid Prime 3: Corruption (12)
12. S.T.A.L.K.E.R.: Shadow of Chernobyl (10)
13. Half Life 2: Episode 2 (10)
14. Zack and Wiki (9)
15. Uncharted : Drakes Fortune (9)
16. Phoenix Wright: Ace Attorney – Trials and Tribulations (8)
17. Forza Motorsport 2 (7)
18. Legend of Zelda: Phantom Hourglass (6)
19. Pokemon Diamond/Pearl (6)
20. skate. (5)

Every year when we do this there’s some game which scores horribly low in the objective rankings but gets a really startling proportion of first-place votes; this year the standout game in the “cult” department was Persona 3, which only got 78 votes at all, placing it at #25 in the overall rankings– but 20 of those votes ranked it in first place, putting it in ninth place above.

I also did two more ways of sorting the rankings: an “approval” vote, where nothing is counted except the number of votes a game received (i.e. a first-place and a twentieth-place ranking count the same– all the matters is if the game was on someone’s list); and an instant runoff vote. Almost every time I’ve ever done this the Instant Runoff and pseudo-Condorcet rankings have been almost the same, but this time they were actually kind of different. Your eyes are probably starting to glaze over at this point, so I bolded the places where these two votes differ from the normal rankings:

FINALLY: PER-FORUM BREAKDOWNS

As mentioned before, this poll mostly exists for a handful of video game forums where some people I know post. This year, I decided to actually run some extra results, in each case counting only those voters who– as far as one could tell from looking at the logs– had come to the poll from one particular forum or other. Meanwhile, as coincidence would have it, a few days into the vote one of the posts from my blog– where I had also posted about the poll– got linked by Digg, and as far as I can tell from the logs a group of the Digg users actually clicked over to the next post and voted in the poll.

So, here you have it– these numbers aren’t totally accurate because my logging method is not entirely trustworthy, but here’s an approximate by-forum breakdown of these results. Links go to color-coded full listings.