Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.
What constitutes a fair review?
Published on October 4, 2006 By Draginol In GalCiv Journals

I don't like negative reviews of things I make. And no matter how far into the fetal position they put me, there's really nothing that can be done except hope that your product or service is good enough that, on the balance, users and reviewers will respond favorably.

The reviews on Galactic Civilizations II have been very positive overall. But we also got our share of reviews that I either wasn't happy with or felt were unfair.  But what can one really do?  In my case, if I felt a review was unfair and was able to document specific examples in the review or put together some other coherent argument as to why the revie was unfair, I'd contact the review site/magazine/whatever.

But one thing that must be done with great care -- don't attack the integrity of the reviewer or magazine publicly, because that's a lose-lose situation.

The average GalCiv II review was roughly a 4.5 out of 5.  But at the outter edges there were two reviews that were under 3.5 out of 5.  Conspiracy? No.  Were they fair reviews? Impossible to say.

Recently there was a hub-bub between the company that makes Sword of the Stars and Tom Chick.  Back in 2005, Tom was contracted for a lump sum to help write the Galactic Civilizations II manual since Tom had spent the last 4,000 years ribbing me for how craptacular our user manuals were.  Roughly a year later, Chick, as one of the best known reviewers in the industry, gave a negative review to Sword of the Stars. Kerebos, the game's developer felt there was a conflict of interest.

Maybe I'm jaded but I just don't see it.  That would be like me saying that because we spent hundreds of thousands of dollars with PC Gamer (or CGW or whatever) in advertising that PC Gamer that they should knock down similar games.

From a purely business point of view, games don't compete in the sense they do in other markets. That is, a person who likes GalCiv II is likely to give Sword of the Stars or Space Empires V a serious look at. Buying GalCiv doesn't preclude a purchase of other games that are in the same remotely similar genre. 

More to the point - when it's time to put a product on retail, the # of units they'll take is largely defined by how well other games that are similar did. 

Ever sell a house? It's a lot like that.  You WANT your neighbors who sell their houses to get as much money as possible because that sets up how much you are likely to make selling yours.  If other houses do poorly on the market, it affects you. 

It's not commonly known but one of the beta testers for Galactic Civilizations II was Soren Johnson (designer of Civilization IV).  Some of the features GalCiv II has in common with Civ IV came form Soren's suggestions.  The point being, game developers usually see themselves on the same team. We want Civ IV to succeed and they wanted GalCiv II to succeed.  Similarly, we want Sword of the Stars to succeed and we want Space Empires V to succeed.

So while it is frustrating when your baby gets a negative review, it's never a good idea to impugne the character or reputation of the person doing the review.  I've written my share of frustrated emails but they've always focused on either the review or that the reviewer's perspective came across as someone who just doesn't normally like the type of game they were reviewing (i.e. a first person shooter guy getting assigned to write a 4X strategy game and saying it's "boring" and giving it a low score).

Sometimes it may not seem this way but we developers are on the same side.  We want other games to succeed.  Moreover, the best known reviewers may not always agree with us on our games but they do have a high degree of integrity in my experience.


Comments (Page 1)
3 Pages1 2 3 
on Oct 04, 2006
Hah! I knew Tom Chick couldnt be trusted! No really, I think the whole thing is just silly, but I doubt there is anything anyone can do or say except to just sit tight and ride out the storm. "Can't we all just get along?"
on Oct 05, 2006
Back in 2005, Tom was contracted for a lump sum to help write the Galactic Civilizations II manual since Tom had spent the last 4,000 years ribbing me for how craptacular our user manuals were. Roughly a year later, Chick, as one of the best known reviewers in the industry, gave a negative review to Sword of the Stars. Kerebos, the game's developer felt there was a conflict of interest.


I don't see there being a conflict of interest here either. If this was a conflict of interest, then the vast majority of people in the business world would be guilty of it.

Tom, avoiding reviewing Stardock products, where there might be a conflict of interests is more than enough.

Kerebos seem to be looking for someone to blame, perhaps a attempt to bash the 'perceived' competetion. Out of 20 reviews the game did not get over 80%, its a promising looking game but needs more work/depth to be a true classic.

The GC2 manual is very good, it gets all the necessary info. across in a easy to read manner and makes good use of tables, lists, screenshots to split the text up. I hope this controversy does not effect u employing him to work for u in the future.

on Oct 05, 2006
Kerebos seem to be looking for someone to blame, perhaps a attempt to bash the 'perceived' competetion. Out of 20 reviews the game did not get over 80%, its a promising looking game but needs more work/depth to be a true classic.


That's my impression as well (that Kerebos is looking for a scapegoat), which if at all true, would be silly in the extreme. No game has ever been brought down by a single bad review. Most people I know don't judge a game based on just one review--they base it on reading a lot of reviews, including ones by fellow gamers.

Sword of the Stars is no exception to this rule. It's garnered an average rating of around 70%, indicating it's a decent game, but there are definitely things it could do better. In fact, almost every single review of the game (at least that I've read) say something to the effect that SotS is overall quite fun and has great potential, but it has a few major flaws preventing it from becoming a benchmark/marquee title (such as GalCiv 2 or the first two MOO games). Tom Chick--regardless of any past or future work he has or will perform for Stardock--has nothing to do with how well the game turned out.
on Oct 05, 2006
Sword of the Stars was a great idea let down by a horrible interface. It wasn't awful, but I don't think the developers should go spreading blame for bad reviews on others when they were the ones who made it so clunky.
on Oct 05, 2006
Kerberos (see the spelling   ) are slightly paranoid. If you've been on their forums at the beginning you'd know what i mean. What's worse, i couldn't mention Sins of a solar empire on their forums...that subject was forbidden! (both teams used to work together before). Now that Stardock is publishing SoaE, They'll even see more conspiracies...

Now...maybe i'll be banned from their forums but i don't really care (not that i won't buy their game).
on Oct 05, 2006
by the way...said review seems to have been hacked as i cannot have access to it...

 
on Oct 05, 2006
Kerberos (see the spelling ) are slightly paranoid. If you've been on their forums at the beginning you'd know what i mean. What's worse, i couldn't mention Sins of a solar empire on their forums...that subject was forbidden! (both teams used to work together before). Now that Stardock is publishing SoaE, They'll even see more conspiracies...


So u r not allowed to mention other games on there forum?

Compare this to Stardock who not only allow other games to be discussed but join in the discussion with POSITIVE comments.

I have read Frogboy's and other members of the Stardock team comments on games in general and there have been 100% glowing in there praise of them. I presume if anyone at Stardock does not like a particular game they just avoid saying anything about it.

Well I do not agree with everything Frogboy says about games, I am grateful he steered me towards Dominions 3, which is a fantastic game.
on Oct 05, 2006
So if Miramax paid a movie reviewer thousands of dollars to write a screenplay, there would be no conflict of interest suspected when he gave the thumbs down to the next Paramount movie coming out opposite a Miramax film?

Buying Civilization IV would have precluded me from buying Galactic Civ, because I only have the time for one. The reviews for GCII were better. Better reviews for your opponent's games may increase the size of your market, but it will always reduce your market share. You might not go as far as to buy bad reviews for your competitors, but you'd be crazy to buy them good ones, right? But if their success really has only a positive effect on you, you wouldn't mind.
on Oct 05, 2006
Quite a few magazines link their opinions to certain products directly to the amount of advertisment space those companies buy. It is a sad fact, but it does happen. I'm certainly not saying this is what happened here but I can understand people being a little edgy about this whole thing.

Anyway, relying on a single review is not the best way to proceed and I found that out the hard way. The review said that the game was good although it suffered from a few minour bugs. The game was so rife with bugs, glitches and unimplemented features (for which they still advertised, though) that a few weeks after the game was released, major outlets set a precedent by offering a refund or a trade-in. By my knowledge no game was ever treated in such a fashion with stores using the 'you opened it, you bought it'-principle. It goes to show that a single reviewer can be way off course.
on Oct 05, 2006

So if Miramax paid a movie reviewer thousands of dollars to write a screenplay, there would be no conflict of interest suspected when he gave the thumbs down to the next Paramount movie coming out opposite a Miramax film?

Rogert Ebert wrote screenplays that became movies while also reviewing other movies.

on Oct 05, 2006
Media = propaganda.

Propaganda is the message of those who pay.
To those who will buy it.
So to say that those who will rate it.
Don't tell the message of those who sell it.
Is wrong.

I'm very sorry. About this.
It's the hype who kills the game.

And the saddest thing about it is, they can sell shit.
And sell it for plated gold. And it will be bought.
It's sad.
on Oct 05, 2006
Sounds like the makers of Sword of the Stars want some free publicity to me, and what better way to do that by getting the attention of all your customers. Sad.
on Oct 05, 2006
http://www.gamespot.com/pc/strategy/galacticcivilizations2/review.html?mode=web

You can see all the reviews there, it does seem like those two UK mags are much, much lower than everything else.
on Oct 05, 2006
Heh, according to Wikipedia Ebert's two film credits are Beyond the Valley of the Dolls (1970) and Beneath the Valley of the Ultra-Vixens, "a sexploitation satirical film starring Uschi Digart and Kitten Natividad written and directed by breast fetishist Russ Meyer." Valley of the Dolls was written very shortly after he started reviewing movies in 1967, but I hope Ebert included a disclaimer when reviewing any other breast fetish movies released in 1979!
on Oct 05, 2006
Hey, no one ever claimed he wrote GOOD movies!
3 Pages1 2 3