Brad Wardell's views about technology, politics, religion, world affairs, and all sorts of politically incorrect topics.

image

Disclaimer:

For the first series of these I’m going to put these up so that those of you who have no idea who I am don’t get too concerned that there’s this guy endlessly criticizing the game (and other people’s games) in the most unprofessional way. 

Who I am: I’m the founder and CEO of Stardock. I wrote the original Galactic Civilizations for OS/2 back in 1993 largely as an AI simulator in which humans were allowed to interact.  It took off from there.  I designed GalCiv I and GalCiv II.  For GalCiv III, I consulted and helped here and there but most of my time was spent on Ashes of the Singularity and my day job of running Stardock in which the game stuff is a minority of our overall business.

To be candid, I’m here for you guys.  I write games for people to play. I don’t care how popular the genre is.  I want to play what I want to play and hope you feel the same.

Now, let’s get started.

Start-up

There are two pieces of starting up the game

  1. The first is that pretty little start-up window.  I’d like to see it have a different graphic than the main loading window.
  2. The main loading window

I lowered the load time of #1 by 2.7 seconds on my machine.  I did this by having the language validation put onto another core and moving other validation to being a background thread becuase it doesn’t matter when it happens.

I lowered the main loading by 11 seconds on my machine by having the main game textures loaded on different cores.

Note: If you have a dual core machine there’s nothing I can do for you.  I’m not changing algorithms here (that is called optimization).  I am re-architecting it to run in parallel.

Steady-State

Before you get the next update, load up the game and open up task manager.  On my machine, a couple of my cores are pegged.  I’ve fixed this by using different APIs for measuring time (they’re less precise but much cheaper) along with reducing the number of threads that game sets up for its job system which I’m in the process of scuttling entirely to replace something more similar to what we did with Nitrous over at my other job at Oxide Games.

Bottom line, this updated GalCiv III architecture which will likely show up in 1.84 should use a lot less CPU, be a lot faster and be more stable.  I hope to see this released as an opt-in early next week.

Game Setup

image

 

Memory Use on start of game (via task manager)

  1. v1.84: 2.9GB (my modified version)
  2. v1.83: 4.8GB (current Steam version)

How: Updated the DDS file compression to use one that is new to DirectX 10 (I was able to avoid a DirectX 11 one and get nearly all the benefits). I am still in process of doing this so this will keep getting better.

Total Time from launching the game to being in the game via “quick start”

  1. v1.84: 22s (my modified version)
  2. v1.83: 36s (current Steam version)

So on my machine, 14 seconds of boot time reduced.  This is due to re-architecting, I didn’t “optimize” any algorithms.  I’m just using your hardware a little more completely (parallel computing, my friends at AMD and Intel would be pleased).

Defaults I recommend

  1. Tech Brokering should be disabled by default

Features I’d like to see

  1. I’d like to be able to add components to Mercenary Ships. I think this is very VERY Important. Literally, let me just click upgrade and add and remove individual components.
  2. I’d like to be able to do the above to any ship in fact.
  3. Planets that aren’t class 0 should be given some very subtle atmospheric effect
  4. I think embassies should have to be founded by Diplomatic Envoys. Gives players a lever to improve relations.
  5. GNN report on where the universe stands that comes up every say 20 turns on a particular topic.
  6. In diplomacy, Civs with neutral and then less than neutral relations should get their own circle.  Right now, a player with unfriendly relations occpies the same ring as someone with cordial which isn’t helpful.

Observations

  1. I haven’t gotten to play a lot of GalCiv III since it launched.  It is most definitely a lot better of a game than when it was released.  It wasn’t “bad” when it shipped but I felt it was a bit too much like GalCiv II with a re-skin with a lot of rough edges.  Today’s game shows a lot of nice, and often subtle, refinement. 
  2. Playback speed of battles should be 2X by default
  3. Too many cigar shaped ships.

 

Part 2: Duel

I feel like the performance is good enough to really start digging into the AI strategies.  I already fixed a couple of things I thought were not ideal. 

Disclaimer:  Writing good AI is more art than science.  People who know nothing on programming imagine that we can somehow just brute force calculate out some perfect strategy.  No, you really can’t.  People will pick short terms with decent AI over massively long terms with great AI.  My job is to make a really good AI with short terms. 

Every few years I do a rating of the AIs I’ve written.  Here is how I’d rate them on a scale from 1 (terrible) to 10 (ideal). And my opinions change a bit over time as I learn new things.

Galactic Civilizations OS/2: 9

Galactic Civilizations I: 8

Galactic Civilizations II: 6

Galactic Civilizations II: Dark Avatar: 8

Galactic Civilizations III: Twilight of the Arnor: 7

Galactic Civilizations III: at launch 4

Galactic Civilizations III: at end of 2015: 6

Galactic Civilizations III: today: 5

For other references:

Most other 4X games I play go between a 3 and a 6.

Weaknesses:

  1. The governor system really hurt the AI.  I can only imagine what they do to humans.  But it’s not ideal.
  2. The ship design system needs to be improved. Before Crusade.
  3. The planetary improvement system needs to be improved: before Crusade
  4. The strategic fleet AI needs to be improved. before Crusade
  5. Spacing of stars in star generation needs to be changed (hurts AI, makes game less interesting)

Other problems:

  1. Catering to forum “experts”.  I see a lot of code in here that was put in clearly to assuage people on the forum’s pet peeves. It’s good to listen to feedback. It’s bad to implement their advice as they suggest. Computer players aren’t people. When you try to make an AI play like a human like this, you end up with a very exploitable AI. 
  2. Exploits.  I see a lot of systems in here that I, as a player, could exploit.  Any time I play a game, I exploit it.  It’s a game.  I can choose not to exploit it but the more often I have to do that, the more fun I lose.  You can’t plug exploits easily.  You need to have a better AI system in which exploits just don’t come into the picture in the first place.

That said..

I’m going to do a 1 v 1 game and exploit the hell out of the AI as best I can.

Lots of improvements made today.   But also some significant game design changes to how the economy works that is sure to create some discussion.

The area I’m not that familiar with (in terms of exploits) is diplomacy stuff. Feel free to chime in below with any good diplomacy exploits you want me to look at.

Big problems

So 400 turns in on a massive map I’m running into a couple different problems here. 

First, the AI is reducing its priority on building colony ships even though there are many many worlds available (I modified the internal settings so that there are thousands of worlds to test this). 

Second, the AI is slowly reducing its priority on technology which is causing it to fall further and further behind me.

Part 3: LESS CPU Usage

image

So after my tweaking, the idle CPU time is, well, idle.  I mostly play GalCiv III on my Microsoft Surface Pro 4 so I’m pretty sensitive to battery use. 

Sheer Performance difference:

image

Thanks to Jon (a GalCiv player who likes to play insane maps with 50+ players) I have access to saved games with hundreds (plural) of hours put into them.  I used these saved games to help carve down some of these costs.

Now, my machine is a monster so my turn times are already short.  On his machine, it was taking multiple minutes to get through a turn while on mine, even on the public version, it’s “only” 13 seconds.  So then it’s question of whether it’s a CPU or memory thing. 

 

Part 4: A better, smarter AI

I ended up having to jettison a lot of the way the AI decides what ships to build.  It’s not that the code was bad or that a lot of work hadn’t put into it.  The problem was a fundamental issue of strategic priority: What should the AI do at a given time?

As the game progresses, more and more things compete for resources and this is where AI’s tend to fall apart in complex strategy games.  GalCiv III struggles with this as well.

For example, what type of ship should an AI facing 50 opponents, 14 it’s at war with that also has 4 unclaimed resources within range and 9 nice planets but is also facing money issues but has 3 trade route slots available and has a very vulnerable enemy that can be attacked if it builds a starbase to increase its range unless, instead, it decides to produce a fleet of longer range Frigates?

What I can tell you is that building a colony ship kept getting  knocked lower…and lower…and lower in the priority list.  The result was that as the game went on, the AI, which expanded well early on, would get slower and slower at further expansion.

There was no way to “tweak” this.  It had to be re-done and moved away from bottom up priority system and instead be looked at from the opposite direction: A top down.  The AI player needs to know what its goal is, globally, and pursue it decisively.  That’s where AIs are good.

Another area it struggles with is when it should, post-haste, send a unit to a troubled area versus rallying them up.  I see people on forums saying that the AI “just” needs to build bigger fleets.  Duh.  They’ve put a lot of effort (almost an insane amount of effort) into trying to do this well.  In GalCiv II, my code for handling this was a lot less sophisticated in the sense that the answer was ALWAYS just build the damn fleets. No nuance. Just do it. 

Test game

image

 

Sunday night check in:

Just before midnight, I have my second check-in (first check in was all performance, this one is balance and AI)

 

# Memory #
New Compressed Textures: (DirectX 10)

Starbase minimum distance reduced from 5 to 3 tiles
AI tends to focus more on tech (generally) than previously
AI will build survey ships late in game
AI provided with a much wider variety of ship designs to choose from with the various strategies
AI weights units much more closely than previously in order to not make one virtually impossible to build
Opposing culture diplomatic penalty reduced from 1.25 to 1.1 if AI is benevolent
Opposing culture diplomatic penalty increased from 1.25 to 2 if AI is mercilous
Opposing culture diplomatic penalty increased from 1.25 to 2 if AI is cruel
Close to victory diplomatic penalties increasedf rom 1 to 5
Ripe for conquest diplomacy penalty reduced from 0.75 to 0.1 if AI is benevolent
Ripe for conquest diplomacy penalty increased from 1.25 to 1.25 if AI is aggressive
Minimum planet class to be a manufacturing world reduced from 7 to 4
Max number of colony ships the AI will construct at once increased from 2 to 3
Max slider focus reduced from 95% to 75% (to deal with coercion penalty)
Coercion forgiveness eliminated (there is always some coercion if you move off from center)
Maximum coercion reduced from 50% to 35%
Colony Capital production points reduced from 5 to 1
Civilization Capital production points reduced from 5 to 1
Influence Radius root increased from 1.75 to 4 (i.e. influence ZOC will start out much much smaller now)
Tech inflation penalty significantly reduced
Penalty for switching a constructor to a colony ship increased from 50% to 75% (yea, you know who you are)
Pirates moved further away from starting location
Increased the variance for the various settings (minors, , pirates, etc.)
Modified the governor script to provide a bit more flexibility based on planet bonuses when building improvements
Colony capital maintaince eliminated
Colony capital hub  provides a 2 (was 1) level bonus to adjacent buildings if they are population or manufacturing or research
Civ capital hub provides a 3 level bonus (was 1)
Reduced maint. slightly on research buildings
Rare habitable planets decreased from 30% to 25%
Nebula and black holes and anmalies made much less common on all settings
Occasional planet frequency amount doubled from 1X to 2X
Common stars modifier increased from 0.9 to 1
Common planet frequency amount increased from 0.75X to 3X (this also fixes a bug where common was less than occasional)
Abundant stars increased from 1X to 1.5X
Planet Frequence increased from 0.25X to 3.5X (also fixes bug)
Tiny map size decreased from 30 Sectors to 18 sectors
Tiny map base objects increased from 32 to 120 (i.e. tiny maps will be a lot more filled)
Small map decreased from 40 to 32 sectors
Small map base object count increased from 64 to 150
Small map max habitable planets increased from 32 to 40
Medium map size decreased from 60 sectors to 40 sectors
Medium map Base object count increased from 110 to 200
Large map size decreased from 75 to 60 sectors
Large map base object count increased from 155 to 300 (i.e. over twice as many stars/planets/etc. in a particular size)
Huge map size drecased from 95 to 70 sectors
Huge map Base count increasded from 250 to 350
Gigantic map size decreased from 180 to 80 (but number of objects remains the same -- more stuff)
Immense decreased from 240 sectors to 120 sectors (but same object count)
Excessive decreased from 290 sectors to 160 sectors
Max camera zoom on excwssive increased from 16000 light years to 30000 light years
INSANE map size decreased from 380 to 320
INSANE map object count INCREASED from 1200 to 1400!
INSANE map maxhabitable planets INCREASED from 600 to 640 thanks to memory reduction in this build
Max camera zoom increased from 18.2k light years to 40,000 light years.
Sensor cost components reduced slightly
Cargo module cost reduced from 27 to 10 (cheaper colony ships -- but remember, planets now rely more on population for their production)
Tiny hull cost reduced from 11 to 10
Small hull cost reduced from 25 to 20
Medium hull HP increased from 100 to 120
Large hull cost increased from 128 to 300
Large hull HP increased from 250 to 600
Large hull storage incresed from 100 to 120
Large hull logistics cost increased from 7 to 10
Huge hull cost incresd from 432 to 500
Huge hull HP incresd from 500 to 800
Huge hull storage increased from 250 to 300
Huge hull logistics increased from 10 to 16
Cargo hull cost reduced from 42 to 24
Cargo hull HP reduced from 10 to 1
Cargo hull logistics incresed from 5 to 6
Max AI improvement queue reduced from 3 to 2

# Bugs fixed #
Fixed double counting of trade routes for AI diplomacy purposes
Placed a critical section around the D3DX11CreateShaderResourceViewFromMeory call (random, late game crashes)
Placed a critical section around m_textureSets.PushBack( pGC3TextureSet ) (fixes random load game crashes on large games)
Fixed bug that could cause the AI to see tiles in the opposite map position of where they were exploring
Fixed a bug that could cause the AI to reduce the priority weighting on colony ship production based on the # of targetable planets


# Optimization #
Removed legacy stopwatch code from the sound system
Moved old profiling junk
Reduced CPU usage of the particles thread
Reduced CPU usage of main graphics thread
Removed redundant debug output

## AI ##
Added UpdateColonies and UpdateShipyards as seperate functions
Eliminated the UpdateColoniesShipyards (readability)
AI builds scout ships later in the game to keep exploring very large galaxies
Dramatically reduced AI >normal economic bonuses (no longer needs them)
Re-designed the way the AI expands into the galaxy with colony ships so that its expansion no longer stalls on large maps
Dramatically reduced the amount of time an AI will interact with a player if they are not within ship range
AI doesn't try to bribe other players to go to war unless they and the other players have a decent military already
AI will no longer give tech gifts to players with a different ideology (i.e. no more Drengin giving you tech if you're a nice person)
AI now scales the number of similar ship projects based on number of colonies rather than a flat number
AI pumps up colony ship production priority if there are a lot of nice planets laying around
AI pumps out fewer Constructors in general
Re-designed the eway the AI determines which planets should get a shipyard to be more (way more) intelligent
Updated AI's handling of adjacency bonuses to be more intelligent
AI values planet quality more when picking planets to colonize
AI somewhat more heuristic on tech research
AI somewhat quicker to war

 

This will take some testing.  When you boost perf like this, you uncover timing issues that were well hidden before. 


Comments (Page 1)
4 Pages1 2 3  Last
on Nov 25, 2016

If the startup process is not already doing so, perhaps file validation could be cached by using a simple timestamp. Then it wouldn't have to validate anything at all 99% of the time.

on Nov 25, 2016

Thank you for this update. I know you were dithering over excluding DX10 and doing only DX11. It seems you were able to stay in DX10.

 

My question is, can and will you eventually optimize Gal Civ III to DX12 or is it even doable?

 

I am far from anyone with knowledge, but isn't' DX11 able to be 'better' than DX10 for '.....'everything'?

 

I love you post's, here, please keep them up!

on Nov 25, 2016

Fascinating insights, and quite (actually very) happy to see another update, even with the excitement put safely away in the vaults of secure storage, which power lies in intricate timer based locking-mechanisms!

Personally enjoy the old style "battle speed". Because when you tinker with the ship designs yourself, watching them in (slower) action, never gets old, somehow. Not sure if this makes this one a member of the "flashy explosions and bright colors" club, but certainly a fan of a certain scene in "Ben Hur" (1959).

 

Thanks, for taking the time to put this out. Personally see any updates as early Christmas presents, regardless of the season. Might be a side-effect of sticking to the series for a while. Hard to say. 

on Nov 26, 2016

Updated

on Nov 26, 2016


Spacing of stars in star generation needs to be changed (hurts AI, makes game less interesting)

This is a concern of mine as in, smaller maps (huge or below) I agree on max settings there is a star every x tiles throughout the whole map, less gaps for the black holes.  With that being said on Insane maps (Gigantic and above) the stars start to space way apart with only a cluster here or there causing the experience to vary drastically.  My main concern is that this doesn't minimize habitable planet generation as it has already been morphed since just before release to 600 planets (base) max without modding.  This is a far cry from where Galactic Civilizations II was in abundant (average habitable per star not total although on the largest maps 600 habitable planets was doable I usually saw 450-550 planets).  

I know there is performance concerns with this or multiplier concerns I just don't want this to be morphed even further without modding as I don't like modding and prefer to play the game as intended.  

Thanks.

P.S.

Turn time up to 1 minute 50 seconds from clicking turn to when Auto Save completes.  Plus very choppy ship movement when revealing fog.

Insane - Map, Abundant, 94+ Factions (some have been killed off) plus minors.

AMD FX 9370 16 GB Memory

 

on Nov 26, 2016

Really insightful post, particularly when it comes to designing 'good' AI.

 

I'm not sure this is an exploit per se, but the way the AI wages war, particularly in large maps and particularly when there is a significant gap between star systems has been problematic in my experience. I experience this behaviour as the AI declaring a war on me, but they will not actually send fleets to my territory, or if they do, they will often target planets of very low value. Another criticism of AI war/fleet behavior regards how the AI will build a significant fleet of transports and wars ships, and have them orbiting a shipyard. As war occurs, the AI will send piecemeal fleets out from this shipyard/rally point and as they lose, will often fail to make use of a significant reserve of inventory when they're being pressed.

 

Also -- single transport fleets.

 

The planet/AI governors are miles better than they were at release. The planets build out decently all things considered. One thing I find odd in my last experience however, is they don't seem to make great use of specialist buildings, like capitals for example. I realise that the governor system is probably a very difficult thing to design and tweak, but it's also worth pointing out.

 

I'm still finding that the even without tech brokering, it's really easy to snowball the AI mid-late game by manipulating trade deals. It can be so one sided (and a little satisfying at times), but perhaps AI diplomacy could be improved further. While they're really easy to manipulate, they seem a bit incapable of manipulating back with diplomacy. Furthermore, the AI offers frankly really horrible trade deals that I always pass over, because they're not worth my time as a player when I can refuse the offer, go back and get a much better (and often completely one-sided) deal for myself.

 

These are some things off the top of my head that I have noticed. I haven't played for a little bit and I also try not to engage in any armchair AI discussion because I ultimately know jack about AI programming/design. Overall, I think Galciv III's AI is compares quite well to to all the current 4x games around. But then again, that seems to be a fairly low bar if I am understanding Brad's points of contention. I really do appreciate further improvements, because great AI makes these games really lively and fun to play, and I really have no one I play them with. 

 

*EDIT*

 

Also with diplomacy, alliances and proxy wars feel non-existent. I do not know what is occurring under the hood with the AI, etc, but I can convey the feeling where an alliance with an AI partner does not culminate into anything meaningful in terms of actions on the board. For example, I can never rely on an AI to fight with me when under attack. Also, because of the way I've observed the AI waging war, making a proxy war happen doesn't seem to result in any meaningful changes in the galaxy, although perhaps other players have had different experiences. Lastly, I abuse treaties not because they provide a mutual benefit that helps me out as well as a friendly empire, but because I can farm techs and money which helps me build up the snowball.

on Nov 27, 2016

In regards to planet/AI governors, I would like to designate which planets will become financial/manufacturing/research capitals. I would also like to designate whether worlds will build Precursor Worlds DLC specialty buildings when/if they become available.

on Nov 27, 2016

One thing I would like to see is a 'preference' that the core ai have towards a win strategy and then have the ai work towards those goals at least initially. An example of what I am talking about:


Drengin: They should use diplomacy to get get or force tech deals that allow them to build bigger, stronger ships with better weapons. Perhaps give them a flavor towards logistics, Hull size and weapons techs.

The Drengin should ignore warmonger penalties if ever implemented and continue their path to victory through conquest first. 


Altarians: They should focus on a Science victory (which they are doing somewhat now), they should force diplomacy deals which give them techs to further this end. (I will give you help in the war with the Drengin if you give me the tech on building these buildings). 


Yor: Like the Drengin they can have a warlike path to victory since they hate all organics. Secondary agenda/victory choices should be influence or Science victories. Diplomacy deals should flavor those paths if possible. 


As far as Exploits I use, I (always) focus on trade that improves logistics and hull size. These two techs far outweigh all others in terms of shear power they give to any player. Might makes right and bigger fleets with bigger ships will win the day. The ai should NEVER give these to the (human) player unless they are getting a one sided deal in their favor. 

I like the agenda system of Civ VI and it works so you never know what the ai will want/do. We do not have a warmongering penalty in Gal Civ and the one in Civ VI is broken (one or two wars and everyone hates you), This kind of system has potential and could be used to warn the player that 'these ai may unite and declare war on you because you are a warmonger' <<< That kind of information is invaluable! 

on Nov 27, 2016


Weaknesses:

The governor system really hurt the AI.  I can only imagine what they do to humans.  But it’s not ideal.

So don't have the AI use the governor, but that doesn't mean you have to remove it.

on Nov 27, 2016

bump

on Nov 27, 2016

change log added.

on Nov 28, 2016

very nice

on Nov 28, 2016


Starbase minimum distance reduced from 5 to 3 tiles

Yet another concern, star base spam is high in many games having 7 economic star bases around every system, this would only increase it as well as increasing the amount of bonus's

Penalty for switching a constructor to a colony ship increased from 50% to 75% (yea, you know who you are)

so....

Tiny map size decreased from 30 Sectors to 18 sectors
Tiny map base objects increased from 32 to 120 (i.e. tiny maps will be a lot more filled)
Small map decreased from 40 to 32 sectors
Small map base object count increased from 64 to 150
Small map max habitable planets increased from 32 to 40
Medium map size decreased from 60 sectors to 40 sectors
Medium map Base object count increased from 110 to 200
Large map size decreased from 75 to 60 sectors
Large map base object count increased from 155 to 300 (i.e. over twice as many stars/planets/etc. in a particular size)
Huge map size drecased from 95 to 70 sectors
Huge map Base count increasded from 250 to 350
Gigantic map size decreased from 180 to 80 (but number of objects remains the same -- more stuff)
Immense decreased from 240 sectors to 120 sectors (but same object count)
Excessive decreased from 290 sectors to 160 sectors
Max camera zoom on excwssive increased from 16000 light years to 30000 light years
INSANE map size decreased from 380 to 320
INSANE map object count INCREASED from 1200 to 1400!
INSANE map maxhabitable planets INCREASED from 600 to 640 thanks to memory reduction in this build

Another huge change.... I like the Insane map....

Just concerns, I'm sure the changes are to address performance issues, it's just sad to see a game that has been out for 18 months potentially getting some of these changes.  I trust you have your reasons and I'll wait to see it in action...  By far the largest concern is the star base spamming....

on Nov 28, 2016

Wow, that is a lot of work.  I like it.  Everything makes a ton of sense to me.  The only exception is the starbase spacing.  That seems kind of extreme.  I agree with Seilore that it will increase starbase spamming, but I like starbases, myself, so I have a different reaction. 

on Nov 28, 2016

Very nice

4 Pages1 2 3  Last