LOL.  I was just about to type something similar.  Civ 5's initial reviews where so positive, I am not sure if the reviewers are either retarded, where playing a completely different game, or payed off by Take 2.  I would say most fans of the series where disappointed with the game. 

I find it's best to ignore the scores and read the actual text of a review. The text tells you specifically what the reviewer liked and disliked which you can then use to make a judgement about the game. Scores are completely abitrary, a 7 for one person might mean the same thing as a 9 for someone else.

I find that reviewers are wrong a lot.

BTW FFXIV did not get a free pass Tridus.  It got the same review scores Elemental did, despite probably having more payola.  Game apparently is a complete disasterpiece.

On ag.ru (biggest russian game reviews site) it lists both reviewer's score and users average score for games, and in most cases scores are close (+/- 10). There were some cases when review and users score differed much, but usually its due to reviewer's bias to game/genre/company. However reviewers usually try to judge game by some rules, while users scores are completely arbitrary. 100/100 or 0/100 are VERY rare for reviewers, but users use these scores all the time.

So yeah, I'd say neither of these are truly objective, but I'm still cautious about anything receiving lower than 60/100 from both review and users.

Evidence?

There's no doubt that business interests can play a part on occasion, but this conspiracy that there's a wholesale arrangement between devs/publishers and game reviewers is bollocks.

It's known that in at least one case, game materials were tied to review scores for a AAA-game. 

I don't think there's a wholesale conspiracy, but I do think payola is alive and well for the biggest games, and that mid-major devs such as Stardock are probably the losers in all of this.

I know of the case with gamespot and Kane & Lynch (though really, who pays attention to gamespot) and a recent example of a lad's magazine reviewer being told to give something a high score, but like you say, that's hardly wholesale.

I think people need to make peace with the fact that reviewers might just not agree with them.

Part of it is just legacy/expectations. Reviewers fall victim to this in a big, big way. If a game is from a company that is beloved for producing great games, and the game is highly anticipated, then it gets stellar reviews no matter what the case. The most recent example of this is Civ 5. It is not a bad game, don't get me wrong, it got my $50. However it is not the 10/10 that most sites gave it. Compared to Civ 4 it is lacking in some key ways. It needs some work to be as good. However it was anticipated and Firaxis is beloved so it got through the roof ratings. If you go read user reviews, and some of the less "mainstream" reviews you'll find that indeed it is a good game, with some flaws that need fixing and it gets the more 7-8/10 that it should.

Valve would be another great example of this. There seems to be nothing those guys can release that every reviewer doesn't claim to be the best thing EVAR. Again, it is a case of them wanting it to be great. Because Half Life was such a cool, genre defining game, now everything they do is rated as wonderful, no matter what. A fairly recent example would be Left 4 Dead 2. The original L4D was a fun game, but I don't think quite worthy of its universal praise. However L4D2 was really nothing but a cash in. Released a year later, almost nothing new, it was an expansion back in all but name. Not bad, but not worth $50. However it again got universal acclaim, because it was from Valve. Had it been form someone else, it probably would have been looked down upon as a ripoff.

Reviews are more useful in the case of less anticipated releases. When it is just a game that is coming out from a developer that isn't really special, they often are far more accurate. They evaluate what the game is, not what they want it to be.

However some people, Valve, Blizzard, Rockstar, etc, just don't get real reviews. Their reputation influences things and so the reviewers inflate things without realizing it. The have mentally prepared themselves for the game to be awesome, and so it is to them even if it really isn't.

It is. It would have been a bad game if it was released in 2003 (ie: before WoW). Compared to modern MMOs? It's a complete disaster. I really loved the part where if you crash while doing a timed quest (ie: all of them) you fail it. And for the ones that have a cooldown? Oh well, come back in two days.

The game is so crash prone that I actually wound up making a NEW CHARACTER in open beta (which was actually a demo, I mean they explicitly barred open beta users from submitting bug reports so it was never intended to be a 'beta') so I could attempt the starting quests again. Release fixed very little about the crashing. The control scheme is completely out to lunch and they made no effort to come up with an intelligent system for the PC (it's really aimed at the PS3). Target something, pick a spell, then... target again? WTF? You know what I'm targetting! I never did figure out how to move the world map to actually find things.

It seems largely single player focused with text messages limited to such a short length that you can barely say anything (think: twitter). Performance is bad. Did I mention the crashes?

I really have almost nothing good to say about it, because I couldn't get far enough into it to actually find out how anything past the early game is like. Hell, even their patching system is so terrible that after hours of waiting I had to go to a website and just download the last files (even some of the reviewers commented on that)... and this is on 25mbps fiber.

Square really has declined in quality since the merger.  Then again, most Japanese companies have really declined.  Even Capcom has in my eyes, though I've heard good about DR2.

I was going to write a post just like this, but i dont have to.

Probably beating a dead horse here, but there has been discussions and issues for years surrounding these beta's, and what it means to be in a beta and players using them to test the game. I admit, I am trying out the game too but I can ignore things that are prone in betas like falling through the world, crashing (crashed a couple times in FFXIV myself), and sometimes missing features though it sounds like all those missing features in beta were still missing on release. Still, SE throws out a beta, doesn't given English speaking players a forum or any means to whatsoever to provide feedback and still call it a beta. There was no way that game was going to change the way it needed to be changed when they only took the feedback from FFXI fans who got in on the beta early and worship the ground SE stands on.

While I don't necesarily think that many of the things you talk about Civ 5 are as big a deal as your making them out to be (other then the AI, which is a big deal but many people don't care about AI as can be seen by the love for MoM or MoO 2) I do agree with your general point that reviewers often don't get that deep into a game before they give it a good review. The most infamous example of that might be Black and White, a game that was released to much hype where you played a god raising a giant pet who helped you rule your kingdom, kind of part god game, part pet game. The first few hours of the game were incredibly interesting and original so initial reviews were glowing. At least a few reviewers were later forced to apologize for their A+ score when they realized that the game wasn't very deep, had poorly designed mechanics, and got boring pretty quickly once you got past all the cool stuff.

Most of the time though you don't get to see apologies for overhyped games, people just disagree and move on.

I'd always put more weight with player reviews, since people generally buy the type of games they like (i.e. TB strategy game lovers will buy TBS games, FPS shooter lovers will buy shooter type games, etc), and give more accurate (practical) info about the game itself. Whereas, formal reviewers are more than likely "paid professionals" who are likely just looking to rate the game by rankings as to certain categories of which the categories are themselves weighted. For example, "reviewers" probably over-rank or over-weight "graphics" as to TB games which, frankly, are not as important to TB type games, and should probably be 4th or 5th as far as weighted categories. In any TB game the AI should always be #1. Graphics would be more important in a FPS game.

Also, I wouldn't be surprised if larger, well known companies, put more pressure on or "lean on" reviewers more than smaller indy companies do. A reviewer would likely be afraid of the 900 pound gorilla but not the 90 pound weakling in the room. So, "politics" most likely plays a part in "paid reviewer" ratings.