Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.
What's involved in making a fair computer AI
Published on March 31, 2006 By Draginol In GalCiv Journals

Every so often someone, usually someone who just got beaten, will start screaming that the game "cheated!".  It's a lot like the neighbor kid who screams "cheater" every time he loses at a game.

One of the reasons people like Galactic Civilizations II's AI is because it doesn't cheat.  But that generalization is only true to a point -- it doesn't cheat until AFTER "intelligent" (aka "Tough" in the overall difficulty slider).  Few players can beat it at that difficulty level and that's where meticulous care was put in to make sure that it's playing exactly the same game as the player.  Beyond tough, all bets are off.  At that point, we're talking about players who have either found an exploit or are simply incredibly good at strategy games and our goal then is to make sure they have a challenge. 

At Intelligent, it's playing its best game, same rules, same money.  At genius, it gets a 125% modifier to its bonuses (economically mainly and ability bonuses).  In practice, it just makes them a little bit spunkier as no AI can be as efficient as a human in a crunch so this is the level I recommend for people who are really good at the game who can beat intelligent.  And the last one is called "Incredible" where we're doing everything we can to beat the player.  200% modifier to bonuses, the intuition algorithms are set to be very lucky, etc. Basically, the player's a god at this point and we're doing all we can do beat the player, no holds barred.

The difficulty levels (from the manual) are:

  1. Fool.
  2. Beginner
  3. Sub-Normal
  4. Normal
  5. Bright
  6. Intelligent (this is where the AI is playing the same game you are with no bonuses or penalties with its best algorithms)
  7. Genius
  8. Incredible

In most games, the AI begins cheating right away or at such a low level that even casual strategy gamers have to raise the level well beyond anything that is remotely "fair" in order to be challenged.  In Galactic Civilizations II, it's not until near the end that the game is tilted against the player.  Again, this is in the user manual so this isn't some state secret. 

In the development journals, we talked about the goal of having the "intelligent" AI being able to beat 95% of players.  I think we succeeded.  In v1.1, our goal is to maintain that 95% (as people get better at the game some will eventually be able to beat the "intelligent" AI so we have to evolve with them).

So where do people scream foul and why?

  1. Claim: "The AI knows where the good planets are".  This is largely a carry over from Galactic Civilizations 1 where they did know where the good planets are. As any experienced GalCiv 1 player can tell you, that advantage was really pretty minor in the bigger scheme of things.  A good player can colonize as fast as the AI in GalCiv 1.  In GalCiv II, it's even easier because if you have Stellar Cartography you can see the planets on the mini-map. 

    However, we knew that many people used the GalCiv 1 AI advantage to say "the AI isn't totally uncheating" so a lot of work was put into having the AI design, build, and scout out planets. And I mean a lot of work.  To a user, scouting is easy, they "look" at the mini map. But having the AI do it intelligently took some work.  Even things like how many scouts to build, when should it build them, where should it send them, involved a lot of work to do it well because every move at the start of the game is crucial.

    In 1.0X of the game, we even put in additional checks to make sure that this was the case since there were cases where the bIsKnown flag could be uninitialized and have some junk in it (and as any C/C++ coder can tell you, if you're doing a if(bIsKnown) and there's garbage in there it'll return true.

    But this scenario has been played out over and over again, tested, verified by us, other players, etc.  Anyone with the demo can test it for themselves too. It's very easy to verify that the AI does not know where the good planets are.  Now, at "Incredible", it's hard to say.  The ability bonuses and some of the simulated intuition stuff gets murky when you're multiplying everything by 200.  But that's not material to our discussion since the AI at Genius and Incredible is getting advantages over the player and has always been stated as such.

    For instance, at 200 bonus, the sensors are probably so powerful that they have no real fog of war. 
     
  2. Claim: "The AI builds too fast."  This part almost always rises out of, sorry, "newbie" players who don't realize you can quick buy things. Decent human players do this, and you can bet the AI does this when it can.  The more money it has, the more often it can quick buy things.  On the planet screen, that's what the "Buy" button means.  A player starts out with 5,000bc.  You can literally build something every turn for dozens of turns without any special advantage.  At the highest levels, where the AI gets money, it can go even longer. 

    This particular claim I find especially annoying because it sounds a lot like the neighbor kid screaming "cheater" because they don't know how to play the game effectively yet.  It would be much better for the person to simply ask on a forum and then someone could tell them how it's being done (or read the manual).
     
  3. Claim: "The AI trades with each other too much."  This is another one of those "I'm losing so the game must be cheating" type things.  In 1.0 to 1.0X the AI literally does a ulPlayerIDToTradeWith = rand()%(ulNumberOfActivePlayers);.  IF it's the human player, it pops up a dialog and offers to trade X for Y. ELSE it just simulates the same thing with the other player deciding whether they want to make the trade or not.  In 1.1, it still does much the same thing but I made it a lot less likely for the military techs to get traded if they're expensive since it was very annoying having each player having the same military techs. 

    The point being, the AI isn't making a distinction between the human player and another computer player.

Now this isn't to say there isn't some bug or something.  But I haven't seen it and I haven't seen one other than the bIsKnown initialization issue and even that would likely only show up on a saved game and was fixed weeks ago in 1.0X (and I was never able to reproduce it, I just saw that it wasn't being properly initialized and that the variable wasn't being loaded correctly from saved games).  So it's possible that there is some bug. 

For example, I haven't looked into this because I didn't write the code but the AI may know where galactic resources are.  I say that because during the beta, galactic resources showed up on the mini-map.  IF that's the case, we'll likely have galactic resources show up on the mini-map again (assuming they don't already) because there really shouldn't be a "Race" for them, they should just be known IMO.  But as a practical advantage, it's pretty minor. By the time you've found your planets, you've probably also found the galactic resources.

Consider some of the things the AI has to do, in order to play by the rules and still be effective:

  • It has to design its own ships. You think you have trouble deciding whether to put an extra sensor or life support unit or weapon on a ship? Try being a computer player trying to fit every last thing on the ship without cheating. Think of all the possible combinations of fitting stuff onto a ship to use up every last bit of space.  Think about what weapons you would put on and what defenses you would put on.
  • Similar to the first point, having to figure out which weapons and defenses to focus on. Not only does it have to focus on them and research them but it has to figure out when to research them and do so without being scripted.  It has to know how much offense and defense to put on.
  • It has to know whether it should be focusing on range or speed based on different galaxy sizes, different numbers of players, etc.
  • It has to know what to build on a planet and when.  Should it use that special food tile for a farm or put a factory on it anyway because it needs the production more than it needs the money?
  • It has to know when to know when it should be building colony ships as opposed to freighters as opposed to constructors.  In Civilization IV, my favorite TBS, the player might build a dozen or so "workers" throughout the game. Workers don't get consumed when they're used, they stick around the whole game.  But in GalCiv, you have constructors which are consumed and you have freighters, which are consumed, and you have transports, which are consumed which means a LOT more decisions have to be made on what to build on a planet.  And some of these units are expensive.
  • It has to do a lot of work on the diplomacy, there's a lot of factors to take into consideration.  Because the game has no political borders which in other games prevent players from doing a sneak attack, GalCiv's AI has to be on the look out for all kinds of threat patterns and do so intelligently.  Plus, you don't want players going to war with you one turn and then asking for peace the next. There has to be a good, logical, intuitive reason why it goes to war.
  • The game has fleets AND those fleets are bound by logistics. In other games you either have armies/fleets or you don't.  If you have them, there's no practical limit to how much stuff you can cram in them so they can just go around wreaking havoc.  In the games that don't there's no effort in having to put together such an army.  But in GalCiv II, the AI has to decide how big of a fleet to make, when it should be used, what ships to put in it.  Plus it has to research techs to increase those logistics abilities. When should it research them? Why?
  • The tactical battle AI could take pages to explain.  In most space games, they don't have an open surface so there's plenty of choke points and a very limited # of places a fleet can go.  In most land based games, you have mountains and oceans which create choke points as well.  But in GalCiv II, it's open space which means that it's very very easy for a human player to outflank or use mobility to win the day.  The AI has to deal with that kind of thing. It has to know when to flee or avoid a target, to circumvent an opponent to go to a more valued asset.
  • The AI has to contend with special invasion ships. In other games, any unit can invade. In other games, one can build a transport and send it over and not worry about having to escort it.  In GalCiv, the AI has to decide when and how to escort a ship.  Escort too much and you lose the initiative. Escort too little and you lose billions of troops in space.

And it goes on and on and on.  So when you look at the claims people have made above, they're such small potatoes compared to what is involved in writing a computer AI that is playing by the same rules. 

If I were going to have the AI cheat, I'd do things like give it more space on its ships. That would be a wonderful advantage.  Or I would teleport ships outside the fog of war so that it would be easier to coordinate attacks.  Or I'd bump up its logistics ability so it could fit just a bit more in the fleet.  Or maybe I'd sprinkle its worlds with just a few more special tiles.  But I don't.  Because the AI doesn't cheat.  But it would be so much easier if it did.  And so you can probably see why I get so annoyed when some new user or hack reviewer tries to build their own credentials by making cheating allegations.  Because I could have saved a ton of work if I had had the AI cheat.

And claims like "The AI colony ships knows where good planets are" are so annoying because first, it's not true and secondly, even if it was true, it's such a trivial advantage compared to the real tough stuff that players take for granted.  As most veteran GalCiv II players can tell you, they typically out-colonize the computer AI because human intuition is better than simulated intuition anyway and that would still be true even if it knew where the best worlds were.

But let's say that the game cheats. Let's say it cheats like crazy.  That would put it on par with pretty much every other strategy game out there.  Hearts of Iron 2 is a really good strategy game isn't it? Very popular in Germany?  Yet its AI isn't even remotely playing the same game as the human player. Does that make it a bad game? No. 

The reason Galactic Civilizations II's AI doesn't cheat isn't because it makes it more fun or it's a good marketing tool, it doesn't cheat because for me, as a developer, the whole point of the exercise of writing a strategy game is to write the AI.  That's why our graphics usually suck in our games because I tend to only care about the computer AI.  

The point being that there's no real business justification for me to be coding on the game other than I love coding computer AI.  It's what I love to do.  I love hearing about other people's strategies. I love making tweaks and enhancements to the AI.  I love adding in new features.  But I also get very annoyed when some user or "reviewer" makes "it cheats" claims without even talking to us.  Odds are, they either misunderstood something.  But let's say they found a bug (which isn't likely but for the sake of argument) unlike the AI, we're only human. Who knows, I could make a mistake.  The AI doesn't escort its ships as well as I try to code it, it's possible that I missed something.


Comments (Page 1)
5 Pages1 2 3  Last
on Mar 31, 2006
That's why our graphics usually suck in our games because I tend to only care about the computer AI.


All I can say is...
I'm glad it's only usually, I'm very satisfied with this one, and hoping for more of the same from Society... building design... yummy! Are you doing the AI defense in that game as well? (Since that's the only AI part I can think off.)

That aside, I'd like to say that it was a pleasure to be schooled by your AI for the first few games while I was getting the ropes (still can't believe I lost to the Bright AI so badly... the shame). It was quite fun, and I'm hoping for more similar experiences as you continue to improve it.

PS: I was wondering how long it would take for something like this to come up. What's with all the sudden AI cheating thread... didn't we go over this ages ago?
on Mar 31, 2006
With all the new AI revisions and all - when are you going to take a break Brad? There's only so much off the boards that any man can take. Don't take all the AI blame personally, people just need something to vent over.

The AI is still the best out there right now, and you're trying to make it better. Much appreciated.
on Mar 31, 2006
You forgot one more item. The AI has to do all that FAST. Not many folks would like waiting 5-10 each turnn on the AI!
on Mar 31, 2006
Yiou have done a great job on the AI, and the entire game Brad!
on Mar 31, 2006
We need a checkforBSinforumthread()

There's a percentage that no matter how well you explain it, there'll still be new posts about "AI Cheating" and my favorite one the AI is a tard and they're playing on EASY for crying out loud. (Easily fixed by playing on harder difficulties ). I don't believe a large % of posts. Intelligent is not too easy. It's also too easy to lie about what difficulty you're really playing.

The one about trade really burns me up to no end too. I thought it (the game was fine where it was) didn't need any changing. What I thought was a shocker that late game minors that "appear" have all the techs. Other than that. I didn't care if the AI traded or no. I didn't trade it my weapons techs thats for sure. When I did make a trade I thought I got a fair shake, now I'm fairly sure I get shafted everytime a trade deal is made (either the computer AI offer or what I have to add extra to get a trade ). Ugh.
on Mar 31, 2006
About the 'fair' tech trading, I've written up on why it is needed extensively. I don't want to go off topic so I'll just post a link to that last one...

Link

on Mar 31, 2006
All I can say is that the AI improvements in the beta have made it a whole new game

-Dewar
on Mar 31, 2006
Yep, the AI is pretty good in 1.0X, at least it makes me think about what i'm doing early to mid game. Can't wait till 1.1 comes out and challenges us further.
on Mar 31, 2006
PS: I was wondering how long it woud take for something like this to come up. What's with all the sudden AI cheating thread... didn't we go over this ages ago?


German print magazines have been panning the game pretty universally. The worst review came in from GameStar, which is the biggest print magazine in Germany for PC game reviews. They gave it a mere 62% and the reviewer justified it by saying that the AI cheats ( and starting positions are almost always bad... not that they didnĀ“t give Civ4 10/10 points for starting positions, in comparison to 3/10 for GC2 ).
on Mar 31, 2006
I'm using 1.0x also and I'm generally impressed with the AI. The list for 1.1beta looks good so I'll have to try that after my game is over.

The main weakness I see in the AI is that it does a poor job of choosing its battles, which unfairly handicaps it as it throws away ships in hopeless battles. This is mainly a problem when I have a fully loaded military starbase helping me out.

Can the AI see that the fleet it is attacking is under a military starbase bonus?

I'd like to see the AI be able the size up defenses and know it has no hope of winning unless it builds its own base for supporting the attack. That's what a clever human would do.
on Mar 31, 2006
German print magazines have been panning the game pretty universally. The worst review came in from GameStar, which is the biggest print magazine in Germany for PC game reviews.


Someone better get translating Brad's rebuttle into German then
(No, DON'T use those online translators. They're butchers)
on Mar 31, 2006
Good read. I'd put this in the FAQ.
on Mar 31, 2006

-----------------------------------------------------------------------
AI suggestions...


Thanks so much for posting this, Brad. I have been using the sensors = +4 racial pick for the last few games and

I will verify that the AI is "not" cheating.

I'm glad you brought up the AI issues, there are some areas where I feel you all have been doing a "great" job on

AI, and others I have some suggestions on how to improve it without too much work. You touch on these in your post

and I appreciate the chance to make some suggestions:

You wrote:

"It has to design its own ships. You think you have trouble deciding whether to put an extra sensor or life

support unit or weapon on a ship? Try being a computer player trying to fit every last thing on the ship without

cheating. Think of all the possible combinations of fitting stuff onto a ship to use up every last bit of space.

Think about what weapons you would put on and what defenses you would put on."

I suggest that the AI can do this by using a library of "canned" designs that help it out. For example, here are

two for early game:

Ship Defender 11: 1 laser, 1 titanium armor, small hull.
conditions: totalplanetAttack<3,planetDefense<1,anyOpponentHasAttack,opponentAttack<=2,opponentAttack!=missile.
Techs required: laser 1,titanium armor.

Ship Fighter 31: 3 laser Mark III,1 titanium armor,small hull,impulse II.
conditions: undefendedPlanet,opponentAttack<3,opponentDefense!=shield,iWantToExpand,opponentDistance<10.
Techs required: Impulse II,Laser III,enhancedMiniature, Titanium I.

The computer every so often (like when it is time for a world to build) would look at existing conditions, and

would only consider building designs that meet those conditions. If it hits a situation where, say, he can't

defend a world because, for example, in the early game, he sees that an opponent has developed attack capabilities

but he has no or inadequate defense, then he would put an immediate priority in researching the required techs so

that he could switch building.

I would be more than willing to send in some of my successful ship designs for different situations, I am sure

others would as well.

You wrote:
"Similar to the first point, having to figure out which weapons and defenses to focus on. Not only does it have to

focus on them and research them but it has to figure out when to research them and do so without being scripted.

It has to know how much offense and defense to put on."

"It has to know whether it should be focusing on range or speed based on different galaxy sizes, different numbers

of players, etc."

Both of these points can be handled using the conditions I mention above. You just need to build ships that are

appropriate for the situation you are in. Ships get outdated so quickly anyway (particularly with tech trading

which is why I LOVE the option you have implemented to disable it) that you just need to focus on current goals

for what to build. You can always upgrade successful ships later.


You wrote:
"It has to know what to build on a planet and when. Should it use that special food tile for a farm or put a

factory on it anyway because it needs the production more than it needs the money? "

This actually should not be difficult, I have build orders that just need to be slightly tweaked for special

tiles. You might want to decide what a planet is going to specialize in (special tiles, PQ, etc. help here) and

then have specialized build orders for each one, making sure you match tiles to their bonus except in special

cases. I can't tell you how many times I have captured an opponent's world to see special tiles mismatched to the

building placed upon them. I then have to demolish these buildings. I mean, come on, this can't be that hard!!!

You wrote:
"It has to know when to know when it should be building colony ships as opposed to freighters as opposed to

constructors. In Civilization IV, my favorite TBS, the player might build a dozen or so "workers" throughout the

game. Workers don't get consumed when they're used, they stick around the whole game. But in GalCiv, you have

constructors which are consumed and you have freighters, which are consumed, and you have transports, which are

consumed which means a LOT more decisions have to be made on what to build on a planet. And some of these units

are expensive."

I actually think that the AI does an exceptional job here, generally better than most human players. This is one

way the AI really shines. But it would not be difficult to take it to the next level based on what some of the

smarter ones of us are doing. Extra colony ships, and there is nowhere else to colonize? Take them home and use

them as ferries from worlds that have low approval to either seed worlds with small pop, or to the world where you

will build your econ capital that you want to grow to obscene levels. Or upgrade them to a constructor and start

mining that resource that you are close to!


I think the AI is doing a good job on the other things you mention (for example, diplomacy), with the exception of

military tactics (and just tell me what computer game can do that well!). Two issues about tactics:

1. The major thing and easiest to fix is to protect its transports better. It should be really simple to say that

every transport needs to have at least one military escort fleeted with it (it would also be useful to have the

computer use much faster transports, you don't have to go very far into the game before you can easily make

transports speed 7 or better). Don't take them into range of opponent's ships at the end of their turn

unescorted. And it should only send them when it has identified that a planet can be taken.

2. Computer units seem to be aimless. It looks so much like, "Oh, boy. I am a fleet. Let me go into the

territory of another player and see if I can do some mischief. Wow, look at that planet. Oh, well, too well

defended, let's not try that. Hey, see that mining starbase over there? No, it's too strong. Any transports in

view that I can take out? Any freighters? Darn, my opponent is too smart. I'll just go impale myself on that

starbase. I won't be able to kill it, but at least I will wound it a bit before I die...". Why not have a plan

that you try to execute? Like, I look for an opportunity. For example, I want a planet. Let me see, which of

their nearby planets is not well defended. Ok, there is a planet. Let me achieve enough force to take it. If

that means I have to travel in more than one fleet (because of logistics points) I can still move as one

"effective" fleet. I build my "mega" fleet, put them together, now send them over all at once. As they get

close to the planet, I reevaluate my plan. Uh-oh, there is a military starbase that adds 5 defense to ships at

that planet. Whoops! That changes my plans a bit. Do I have enough force to take out the military starbase?

Darn, no. Is there another target that I do have the force to take out? No. But luckily, I brought over 3 Fast

constructors, because, as a smart AI, I want to be prepared. If I build a military starbase here now, I can

counteract the effect of the other one, and win the day.

on Mar 31, 2006
The only comment to the above is erm... now would you like to write the other 9 AI as well?

Seriously though, it's nice that you're giving all this strategy, but the AI is design NOT to be scripted. To do the things that you ask, build orders, constructor and fleet strategy, it basically means it HAS to be scripted. So while you think that particular AI will do GREAT, a player will laugh and blow the 3 constructors up with a hit and run assault. Now you will have to go back and asign escorts to your constructor fleet, before you know it, you need to gather together 4 navies just to take that one planet, mean while he just took half your empire through delay tactics. Worse, before you could do all that, they will probably see that colony ship ferrying people and know, "ah ha! that's going to be their econ capital, I bet he's building the econ capital right now, let's take it". Etc... etc... etc... When you start scripting the AI, a smart player will find it and exploit it, there's no way around it except not to make it scripted, force it to learn and adapt. It puts the "I" in AI, and is one of the things that keep the GalCiv2 AI way above the rest of the AIs in my book.


Requiring every transport fleet to have an escort fleet is just silly, simply because escorts really don't help transports all that much. What transports really needs is 1) speed, and 2) a fleet that wiped the surrounding area clean. The problem with "escorts" is that they are sickening weak, and they are always on the defense, which is a very bad thing considering how combat works in the first place. A hit-and-run ship could easily kill the escort, and thus the entire fleet. The best thing for the AI to do with the price of an escort is to start building "Spy" ships, cheap, disposable scout ships with a ton of sensors designed to keep an eye on it's enemies. That should keep down the amount of guessing that it has to do. Simply fly one into the area before a transport would, and if it gets taken out, yes you need to escort the transports, or rather, you need a fleet to come in and wipe the area clean. If not, you can see everything anyways... fly away.
on Mar 31, 2006
This particular claim I find especially annoying because it sounds a lot like the neighbor kid screaming "cheater" because they don't know how to play the game effectively yet. It would be much better for the person to simply ask on a forum and then someone could tell them how it's being done (or read the manual).

Well keep in mind that the neighbor they're yelling at (strategy games) is the worst cheater in town, from the omniscient AI in pretty much every one (I'm thinking of an Ion Cannon in C&C, specifically) to direct bonuses to production to stronger units to whatever. Even if Galactic Civilizations was a so-so game (which is purely hypothetical, of course ), I'd still be playing it all the time. Civilization IV lasted a month before I stopped because I didn't want to play against an AI with double the income simply because it existed. I played Crusader Kings for a few weeks before I couldn't stand the AI's invulnerability to debt, amongst other things. I tried Hearts of Iron II again recently, but stopped a few months into the war because I couldn't keep my allies from making strategic blunders that would make Hitler look like Napoleon. (What's with Paradox and AI anyways? )
I think that a lot of these people are used to turning on the medium-level difficulty on the first try and dominating it while they get used to the game. I guess when it doesn't happen they think that the AI must cheat at lower levels than usual instead of simply being well-programed for once.

But as a practical advantage, it's pretty minor. By the time you've found your planets, you've probably also found the galactic resources.

Not necessarily - when I started I just built a scout or two to the planets specifically, ignoring all the space around the direct routes between systems in the initial rush. Plenty of resources in there to worry about. I've jumped on the sensor-ship train since then, but I see a lot of scouts in AI navies, so they may still be a little behind in this regard.
That said, I haven't noticed an all-seeing AI. In fact, I've seen resources inside their territory that I've quickly snatched up before they got the chance. They had other, non-resource starbases too so I doubt that they didn't have the ships available, and I've seen a different AI do something similar from time to time: Two resources near the edge deep within Torian territory, one on the very edge of the map, taken by the Drengins. One (the edge one) was also left alone during the Torian-Drengin War, although the other (within the same sector) was destroyed. The Torians just didn't see it, Brad's right, QED.
5 Pages1 2 3  Last