Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.
Published on January 31, 2008 By Draginol In PC Gaming

According to GameRankings.com, a site that simply takes the review scores from the major gaming publications and averages them, the top 14 games of last year (as many as I can fit into a screenshot) were:

 

2008-01-31_1543
Comments
on Jan 31, 2008
Number three! Very, very nice.

I just beat my first game of Dark Avatar. What a brilliant game!
on Jan 31, 2008
Big pat on the back, Draginol and team. You folks deserve it
on Jan 31, 2008
I don't understand how someone can take gamerankings and similar websites seriously. It's one thing thing to be nominated by editorial staff, it's another thing to be nominated by a statistical flux. You can have a thousand websites like that with a thousand different "tops".
on Jan 31, 2008
Congratulations! A position well earned


on Jan 31, 2008
I'd don't like FPS and I would have bought Bioshock but the copy protection scheme put me off. So for me you guy's are number one.

on Jan 31, 2008
Voting was rigged! We need a recount! (sarcasm)

Congrats. That position is well earned.
on Feb 01, 2008

I don't understand how someone can take gamerankings and similar websites seriously. It's one thing thing to be nominated by editorial staff, it's another thing to be nominated by a statistical flux. You can have a thousand websites like that with a thousand different "tops".


It compiles the average grade given by those thousands of sites. How can you say that a grade given by an editorial staff is significant, but that the average grade of THREE editorial staff members is not?
on Feb 01, 2008
It's great to see Stardock up there showing their stuff against those mega publishers like EA and THQ.
on Feb 04, 2008
[quote="AlmostDecent"]How can you say that a grade given by an editorial staff is significant, but that the average grade of THREE editorial staff members is not?[/quote]
Long answer:
http://www.edge.org/3rd_culture/lanier06/lanier06_index.html

My answer:
I said "It's one thing thing to be nominated by editorial staff, it's another thing to be nominated by a statistical flux." Nomination for game of the year is (when it's done by people) a symbolic/conceptual gesture. It's different from grading game in terms of digits. Digital grades are meaningless no matter how many of them you take.

But I'll try to answer you question's substance. Let's say one guy says that Pathologic is a good game. Probably, you can then deduce what kind of game it is by looking at statement and personality and background of that person.

Let's say you take 2 more people. One says that a game is horrible, another says the game is average. This is not, really, a digital scale, but for the sake of argument we convert it to one.

Bad = 1, Average = 2, Good = 3.

(1 + 2 + 3) / 3 = 2.

So, according to our crude aggregation the game is now Average, but the dominant majority of our sample doesn't agree with this. By averaging three grades into one we just lost some very important information.

This is a simple example, but it only gets worse in real life. The scales are meaningless from the beginning. One journalist thinks that 7 is a good game, another thinks that 9 is a good game, while 7 is a mediocre piece of crap. So 2 journalistscan agree that the game is good, they can agree about the extent to which it's good, but assign different grades on 1-10 scale!

Then we get the reader to do the same thing in reverse. One thinks that 7 is average, another thinks that it's good. Which one is true?

But wait, we skipped a couple of steps! The system must first select which websites to use for averaging and which formula to use to calculate the average. It can be mean, it can be median, it can be rating formula for figure skating (which discards the best and the worst grades), it can be _anything_.

Etcetera.

This is, of course, a description of grading woes. There is also a simple (and very unpopular) truth which precedes that argument: a game cannot be summarized by a single digit, period.