[2022] Stellaris Dev Diary #239 - AI++

A lil somethin somethin: You can find the details for this event on the announcement page here.

written by Offe and Guido

Доступно на русском в ВК/Read in Russian on VK[pdxint.at]

Hello and welcome back to another update on the Stellaris AI. This is Guido again. Today I’m here with my fellow human Offe who also enjoys doing organic things. Like generating energy through processing photosynthesised light in the form of matter via ingestion. I like bacon and ice cream. Everybody likes bacon and ice cream. So Offe, please, take it from here.

Hello, it is me, Offe!

I’m a 28 cycles old Human manufactured and operated up here in the north. I’ve worked here at the Arctic office for two years and recently joined the Custodian team as a junior programmer. Guido and I have previously worked on other projects together and he has taught me a lot about game development, but most importantly I learned some tips on how to improve my diplomatic interaction protocols. Where I would often use phrases like “it’s an absolute disaster”, he would instead prefer “This is pretty good, but it can be even better!”. This may prove to be important later on.

I would like to say Thank You to all the people out there who took time playing on the open beta and provided us with feedback and bug reports. If you ever find the AI in a situation where it is doing something strange, please bug report and most importantly attach save games, it helps tremendously! For example, two separate issues were found and addressed with the new job changes.

And lastly, this dev diary will contain older changes and screenshots that were made long before the beta, but also new changes which were not part of the beta, meaning that you still have some new changes waiting for you in the 3.3 release.

Changes to pop job system
I will start with this change since it will also directly affect players and not only AI!

How it used to work:

Each time something important would happen on a planet, such as a pop is grown, a district/building gets constructed or an upgrade finishes, every single pop would update their desire (also known as weight) to work each job. Then all pops would be unassigned of their jobs, and all of them would be put back on a (potentially) new job.

Now there are some pros and cons with this approach. The good thing is that we are not doing any calculations when we don’t have to, since if nothing changes then we don’t update any of the jobs. However, the downside is that if you have scripted conditional job weights, for example, based on how many amenities there are on a planet, it will cause mass migrations of pops between jobs when the system eventually does update because all pops move at the same time.

In the current 3.2 system the most obvious problem is for hive mind empires where pops will mass move to the maintenance drone job when the planet amenity level is low, and then during the next update, all of them will leave due to having way too many amenities causing a perpetual ping pong effect.

This also affected non hive mind AI empires because in 3.2 the AI would prioritize a job producing a resource during a shortage across all its planets. For example, during an energy credit shortage it would prioritize the technician job on all its planets, causing every single job to be instantly filled. This would likely cause a shortage of some other resource such as minerals, resulting in most types of AI empires to get stuck in a ping pong behaviour once they had entered a resource deficit. This also had the unfortunate side effect of AI starting constructions that were not really needed, but the sudden shift of pop jobs made it appear so.

How it works in 3.3:

• During each monthly update, update the jobs on all planets
  
• Only remove or add maximum of one pop per job during the update

Many of you are now probably immediately clenching your fist in anger while picturing your poor CPU melting, as scripted calculations based on number of pops in stellaris can be very CPU demanding. But I have some good news for you, first of all in 3.2 there were some redundant calls to the job weight calculation. By removing them where possible, we could already reduce the amount of job weight calculations by about 75%.

Furthermore, we are now reusing job weights between pops that are of the same species and share the same job. Meaning if you have 40 pops working as miners on a planet, and they are all of the same species, the scripted job weight calculation will only be performed once instead of 40 times as in 3.2. This comes with some limitations though, as it is no longer safe to base job weight on individual pop data, such as which faction they are in or their happiness. In the end the vast majority of all job weight calculations were removed while still updating jobs every month.

With the new system it allows you to write a scripted job weight calculation that depends on itself without causing ping pong behaviour. For example, jobs that produce amenities can now base their job weight on the planet’s amenity level, or the enforcer job can now base its job weight on the crime level.

The intention is that you will not notice any difference from the system in 3.2 other than some jobs like enforcers and maintenance drones having a more reasonable amount of pops working that job.

Jobs for your pops
In 3.2 AI would look at the number of free jobs on a planet when deciding if it needs to build new jobs. So if there were for example 3 free jobs then the AI would clap its hands together and call it a job well done and move on. At the same time the planet could have huge numbers of unemployed pops rioting on the streets.

This scenario comes from the fact that not all pops can work all jobs, so while there are technically free jobs on the planet, that doesn’t mean that the unemployed pops can actually work those jobs.

In 3.3 we are changing the way that the AI is looking at planets when it is deciding what jobs to create. Instead of looking at the number of free jobs on the planet and then creating more when this number is low, the AI will now look at actual unemployed pops and make sure to create a job that the specific pop is actually able to work.

This solves a variety of issues present in 3.2 where AI doesn’t make good decisions for pops such as slaves or robots, this is something we will continue looking at but it is a big first step in the right direction.

AI scaling economic subplans
Scaling subplans was something we mentioned earlier as a planned feature for the future, well the future is now so strap yourself in!

In 3.2 we got rid of the old economic plans which had a predefined early/mid/late game strategy and introduced the shared base plan which doesn’t look at what year it is, but rather looks at what state the empire is in.

Now when I first saw Guido’s new economical plans I immediately thought wow this is pretty good, but it can be even better! So I started working on the scaling sub plans which aims to remove all upper limits of production (previously mentioned 500 alloy per month cap in 3.2) but still provide the AI with a responsive plan that adapts to the current state of the AI economy.

How the system works as for 3.3:

The base economic plan is now very small, it sets a minimum target for all types of strictly needed resources such as minerals, energy and food (such as +20 monthly income). Once these targets are met, then a small amount of CGs, alloys and science targets are added.

Once all of the above base plans are satisfied we then enable the scaling sub plan, which is just like any other economic plan except that it will add itself each time it is fulfilled, an unlimited amount of times. The scaling plan contains a small amount of energy/minerals but primarily contains alloys and science. This means that the more mature the AI economy becomes, the focus on base resources becomes smaller and the primary focus will shift to military and science production.

Additionally we have added 3 separate conditional scaling sub plans which we enable for materialist, militarist(and total war empires) and spiritualist empires that add additional science, alloy or unity targets to their economic plan as a first step to making AI economy more distinct from each other.

Grand Admiral hive mind reaching a monthly income of 3k alloys and 22k science in one test run by year 2422. (Screenshot from before the unity rework)



AI district/building specialization
One of the big advantages that fellow Humans like you and I have over the AI is that we can easily make long term strategies which are based on assumptions and goals. So we may have a long term strategy to turn a planet that we have not yet colonized into a factory world. As mentioned in answers to the last AI dev diary questions, the economic AI is stateless which means that it has no notion of past nor the future, it only looks at what it has right now and what it can do to satisfy it’s economic plan. This makes it very good at adapting to the situation it is in, it will keep a close eye at the current economic situation and immediately react to any shortages but lack some of the long term planning capabilities that we have.

So how can the AI make specialized worlds without planning for the future? Well one straightforward way of doing it is simply by switching places of districts that we have already built in the past. So if we compare two planets where both of them have 5 mining and 5 energy districts each, we can gradually specialize the planets by replacing the districts one pair at a time until we end up with one planet with 10 energy districts and another with 10 mining districts.

This approach works quite well in practice and is also very dynamic in the sense that it allows the AI to make hybrid planets in the early game which becomes more specialized over time as the empire expands.



AI consumer goods vs alloy production and planet designations
In 3.3 we are adding an AI system where the AI will manually pick a planet designation instead of using the default scripted planet designation system which is the same one as the player gets if you do not change it yourself.

The AI system looks at the available designations for each planet and calculates how many resources it would get each month from choosing the designations. It then scores each designation by judging how well the gained resources fits into the AI’s economic plan, giving extra score to designations that align with its economic goals.

Normally it is very easy to pick the designation, for example, a planet with only mining districts on it will clearly have the mining designation. However, other designations such as Factory/Forge world are more complicated and the AI needs to carefully assign these designations in a way that keeps the economy balanced.

For non hive mind empires science and alloy production is the biggest AI economy challenge we have faced so far, since the AI needs to produce both resources independently of each other to meet their economy plan targets even though they are produced from the same district in three different possible ways. The current system is a step in the right direction but this is definitely a tricky problem that will require additional fine tuning in the future.

AI alloy spenditure
Now that AI adjusts its alloy and consumer good production separately it was time to tackle how AI spends its alloys.

In 3.2 the AI really liked defense platforms, and keeping them up to date by upgrading them any time it was possible. Not only is this a massive drain of alloys, it would also more or less permanently fill the production queue in the shipyards with upgrades which meant that in some cases it wasn’t able to build any new ships even if it wanted to.

Further there was an issue where the AI would get blocked from building any modules or upgrading any starbases if there was an open module slot in which it wasn’t possible to build anything according to the AIs starbase templates. For example, the AI has dedicated shipyard starbase templates and if it has open slots in it then it would really like to build the titan assembly module on it. But if it wasn’t researched yet then the AI would get blocked here, preventing construction of new starbases.

In 3.3 the AI alloy spending priority goes something like this:

• Build new ships until we reach fleet cap
  
• Build starbase modules
  
• Build new starbases
  
• Upgrade starbases
  
• Upgrade ships (and defense platforms) if it gives a +30% fleet power bonus, and upgrade the entire fleet this ship is in while we are at a shipyard anyway. Saving both alloys and time!
  
• Build defense platforms as a last resort

AI tech picking
The AI has scripted weights for each tech in the game, this gives it some direction as to what technology to pick next every time a research is completed. Both in terms of which technologies are more powerful but also taking into account AI personalities, militarist empires are for example more inclined to research weapon tech.

In 3.2 the majority of techs had some modifier on it which increased the chance of it being selected by the AI, but when you prioritize everything, well then you prioritize nothing. For 3.3 we went through all the techs in the game and remade the AI priorities from scratch, emphasizing techs that will help the AI scale into the mid and late game. For example, resource production boosting techs, pop growth techs and resource producing building chains are now more encouraged.

Additionally AI will now look much more favourably on techs that are cheaper compared to the other options, this allows the AI to more quickly cycle through the available options and find the techs that it really likes.

AI superfluous destruction
This one is short and simple. AI will now delete stuff if it gives jobs, housing or building slots that we do not need. Meaning, if we for example have more free jobs and housing than provided by an energy district we will simply delete it to avoid paying the upkeep cost and freeing up this slot for something else in the future.

This scenario most often happens when an AI empire invades another planet and purges their pops, so determined exterminators will now be able to repurpose the conquered planets into something that aligns with their economy!

AI rogue servitor and bio trophies
While there has been a lot of focus on the AI’s ability to compete economically with the player in this dev diary, one of the primary objectives of the AI initiative is also to enhance the role playing capabilities of the AI.

In 3.3 we are adding additional AI support for the rogue servitor civic and how they handle their bio trophy pops. The AI should now build an organic sanctuary on each planet that has an upgraded capital structure causing their bio trophies to spread to other planets. And they should build additional sanctuaries on planets with a lot of complex drones.

Additionally we have addressed a group of related bugs where the AI was unable to build special types of buildings like gaia seeders, spawning pools and chambers of elevation.

AI comparison
As a final note we would like to share some comparison graphs between the 3.2 and the 3.3 AI. Please note that what you are about to see is based on one single test run on ensign and one test run on grand admiral. This comparison is not meant to be interpreted as evidence but as an indication of what has changed between 3.2 and 3.3.

In any AI playthrough there is a huge variance in the AI performance due to random factors such as how they pick techs, traditions and ascension perks. The experiment setup is also used for internal AI testing only and not representative of an actual playthrough.

Experiment setup:

Tiny galaxy

• 1 AI empire
  
• All test using the United Nations of Earth empire
  
• Mid and late game years set to 2575/2600 so they don’t trigger
  
• The map is the same between the 3.2 vs 3.3 comparison, but NOT the same between the ensign and the grand admiral test

.
Let’s first look at the comparison between the 3.2 and 3.3 ensign difficulty:



Up until year 100 the military power is roughly the same, but from that point on the results of the work we put into mid and late game AI scaling starts to really show. This allows the AI to act and react in a lot more interesting ways in the late game than before.



1) Around year 150 the 3.3 (“develop”) AI reaches the 32/32 starbase capacity due to having researched all techs in the game, resulting in the slowdown of the military power development.

2) 3.2 AI gets stuck in an economic death spiral for about 30 years shortly after year 100, AI eventually manages to escape the death spiral and then has massive economic growth and is able to build up to the 32/32 starbase cap quickly due to having saved up alloys for 30~ years.

At year 200 the gap between both AI military strength gets smaller since neither AI is really building that many more ships due to having maxxed out starbase capacity and already way above their fleet cap resulting in very expensive fleets. The power gap at year 200 is mainly due to 3.3 AI having superior technology.

However, it turned out that for GA difficulty the AI wouldn’t correctly apply the increased buff from trade value. Now, when it does, the AI takes a good step in the direction of making it more challenging for players.

Overall the GA and ensign test show a similar pattern where the first 100 years are roughly the same and then the difference becomes substantial. However, in the GA test the upper limit of 3.3 AI scaling can be seen around year 150-200 as the military growth curve tends to flatten out at this point when reaching the starbase cap.

That's it for today's Dev Diary, thanks for reading! Have a question about the AI in 3.3? Ask it here[forum.paradoxplaza.com].