Video game publications, websites, and communities have a tough job at the end of every December: selecting their top game of the year. Will it be Uncharted 2 or Assassin’s Creed 2? Is it an exclusive? Is one console’s version better than another? Is it an original game or a refined sequel? There are plenty of factors that are looked at when determining whether one game is considered the “GOTY”.
I have often wondered how some of my favorite publications have selected their top games. In many cases they rely on a point scale awarded to the individual editors/community members’ selections. For simplicity’s sake this seems like a clean and standard method that has made the rounds: Each voter selects 5 games. The top game gets 5 points, the second gets 4, and so on down the line. When all votes are tallied the highest scoring game generally wins out. (This is in fact how SideQuesting is selecting our GOTY for 2009.)
But then I noticed some variables starting to pop up. For one, the voting method is inherently flawed. In a 5-point scale, there is a greater chance for ties to happen. And, is a game that gets 17 points really better than a game that gets 16 points? Also, point scale methods seem to have a “gray area” where the amount of voters plays a role in making or breaking the system. Too many voters = too many games with low point totals. Not enough voters = one-sided voting towards one or two games. These issues were just the tip of the iceberg.
It’s time to get down to business.
Let me go off on a tangent first. Initially my goal was to set up a perfect formula for helping deduce an actual Game of the Year for publications & communities. I came to realize that, well, I never usually agreed with a GOTY for whatever reason, whether it was an accurate point total or a game that I hadn’t played yet. No matter what happens, someone’s GOTY isn’t going to end up at the top, or perhaps even make the cut.
In other words a Game of the Year is an almost personal selection. Communities and publications select them as a culmination of what a group has played throughout the year, but they will never be more valuable than an individual’s selection. I enjoy when websites post their Editors’ own personal picks as it allows me to sort of align to those writers whom I share similar interests with. If I agree with Joe Schmoe on 8 of 10 games I’m much more interested in what he has to say about game selections down the line.
Regardless, we still may need to create a better formula that helps put a face on a publication’s selection… if anything else than to at least give more credence to a staff pick. We need to start with the variables: What can cause a selection to be skewed?
The variables
1) While console-specific websites will obviously be able to focus on one platform, their GOTY is in comparison only to the other games on that platform. Playing games on competing platforms may change what one player likes/dislikes in a year. Liked The Conduit? After playing Modern Warfare 2 one may decide that the FPS controls of The Conduit aren’t optimal, thus pushing the game lower on their scale below Muramasa.
On the other hand console-agnostic sites have to deal with an issue of too many games and too many platforms. There are great games on the XBox and PS3, but what about the iPhone? Android? Facebook? Flash- and web-based games?
2) As noted above, the most common way of selecting a top game is usually to poll the site editors or community members. Therein lies a perpetual struggle: Do the opinions of the editors who live, eat, and breathe video game reporting have more weight than those of the fans or community? The editors typically see and play more games than community members. And even then, who’s opinion is more valid: the editor that played 50 games but not Assassin’s Creed 2, or the editor who only played 6 games, one of which is Assassin’s Creed 2?
3) In some cases, a game will appear on everyone’s list at various levels yet never end up cracking a top spot because another will have two or three high ratings. Is a game that appeared only twice at #1 a better game than one that appeared 7 times at #4 or #5, even though it may have a few more points?
4) While it may be a decent task to narrow down a list to five games, it can often be a fight to select between 2 or 3 for our favorite of the year. “Did I like Uncharted 2 better than New Super Mario Bros.?” Sometimes the difference between our personal #1 and #2 games isn’t as clear.
5) Finally, is it okay to select a game as a GOTY if the player/editor hasn’t completed it yet? Sometimes games are great throughout. Sometimes editors never get a chance to complete very many games throughout the year, only playing 60-75% of the way through some.
The Formula
Once we have a set of variables — I’m sure there are more, but we’ll stick to these for now — we can begin to create the formula that solves for them. Ideally, 7-10 is a great number of voters to help really get a good range of games and gamer types (casual, core, fanboy, etc). Anything less or more seems to leave awkward ties or holes in lists.
*Note: While scribbling notes on a chalkboard ala my 7th grade math teacher Mrs Crabtree, I ended up creating a second formula that seems to work to an extent as well. Instead of tying it to this article I’ve added a post in our message boards… found here!
First, we’ll use the 5-4-3-2-1 scale as our starting point. The top pick receives 5 points, the bottom gets 1.
Knowing that there may not be a clear delineation between selections in a top five list, we need to create a method that allows voters to adjust their pick’s point totals. In comes “the bonus point.” By allowing the voters to place one extra point anywhere on their list, the ability to make up for some of the gray area is resolved. Can’t choose between two games for the top spot? Give them both 5 points. Really loved your top game over all of the rest? Give it 6 points instead of 5. This allows for some level of indecisiveness AND clarity to be accounted for.
Next we need to resolve the issue of games that are listed multiple times. This “frequency” issue is common for games like Borderlands that appear often but never at the top. In this case, the 7-10 voter total really shines. For a game to be considered “frequent” it needs to appear in at least half of the lists. For each time it is listed after that it receives an extra point. This allows for games that more voters play to show higher on the list. Games like Battlefield 1943 gain higher rankings, and may crack the overall top five.
The last bit of the formula left to fix is to help leverage the votes of those who have played more games than others. We can call this this the “Ryan Gan clause”. SideQuesting staffers Ryan Gan and Aaron Kirchoff had played more games in 2009 than anyone else on the website. They have a great understanding of why they choose their top pick over their second or third. So how do we make sure that their point totals (and those of other avid gamers) are given a fair amount of relevancy? With a staff of 7-10 we can select the folks who have played the most games and give them extra points: two for their top pick and 1 for their second. Selecting who gets the extra points is relative. If there are two people who vastly outplay the other gamers then those two would get the extra points. The best rule of thumb would be to keep the top players to 33% of the voters or less, otherwise the “minority” of experts starts to become the median amount. Hey, playing a lot of games pays off.
Essentially, this formula can account for several of the variances that occur within a publication’s selection process. While it may not create a surprise grand champion, it might be able to resolve the small gap between two or three games’ point totals. There is just one problem: It’s too damn complicated. In fact, it’s too complicated for even at SideQuesting us to use. It’s borderline Calculus II. An advanced formula still doesn’t account for everything, like HUMAN EMOTION or the ability for people to change their minds during a discussion (see: our GOTY podcast). I guess that is why going to a simple 5-point scale (while not the most accurate) provides an easier way to bypass all of that. And, it just works. Period.
In Conclusion…
Using a point system takes away the human aspect of selecting an outlet’s top game. While I personally chose Uncharted 2 as my game of the year, Modern Warfare 2 may have earned the top point total. Yet after a lengthy discussion with the Editorial team we may push for Assassin’s Creed II to take the top spot. There really is no way to accurately choose a Game of the Year without some flubbing here and there. Using a formula may seem like a good shortcut, but it’ll never replace personal preference.
2 Comments