[2019] Friday Facts #286 - Pollution cleanup

A week in the office

This week is another week of typical bug fixing, so I thought we would make a one-time change of style and do a day-by-day account of what exactly that means for us.

Monday

We had a typical day of bug fixing for the most part. After our weekly Monday meeting, where we discuss the development plan and the Friday Facts plan, everyone settled into the normal bugfixing work, which boils down to something like this:

1. Find a bug to work on - Either by looking through the forum or the Automated crash reports.
2. Try to reproduce the problem, or someway figure out what went wrong.
3. Try to fix the problem. Sometimes this is super easy, other times it can be hours/days.
4. Check the solution is effective at fixing the bug.
5. Check the solution didn't break something else. For this we have over 1,300 integration tests which cover a large amount of the code base. We also have a server which runs tests for all platforms whenever anything is pushed to the master or any pull request.
6. Commit the solution with a changelog entry to master and push!
7. If applicable, move the bug report on the forum to 'Resolved for the next release'.

It was about 4pm when we had to decide about a release. There wasn't anything super game breaking, but there was a Introduction Campaign script problem which we had solved, so we decided to start the deployment process.

To be succinct, the deploy process is as follows:

1. We enter a console command on the deploy server.
2. The deploy server does its automation magic, builds all the platforms and uploads it everywhere, updates the locale from Crowdin, etc.
3. A human finishes the non-automated steps, namely posting the Steam changelog and setting the release live on Steam.

This typically takes about 90 minutes.

The 0.17.10 deploy process executed flawlessly.

However quickly we saw a massive spike in crash reports, seems we had broken something with GUI's in multiplayer. After the first hour, we had over 100 crash reports, so it was definitely a big problem.

It was just a simple oversight, so it was a one-line fix. We quickly spun up the deploy script again, the final tally of crash reports was over 300 by the time the hotfix was out.

Tuesday

Tuesday started in the usual way, with most of the team arriving in the office by 11am, and Ben arriving after a long bus ride from Germany. Just before lunch kovarex opened a discussion in the office about the pollution balance, prompted by this forum post.

It seems when we normalized the burner energy efficiencies, we didn't consider that it would affect how much pollution the burners would generate. We talked for some time about what to do, the effect of pollution on the strategic gameplay decisions. For instance the decrease in pollution from Steel furnaces to Electric furnaces is an important factor in a players choice of which furnace to use.

The starting point is that we don't really have any super solid information on pollution and what the real values in the game are. We were discussing that its some value per tick per amount of energy consumed, but different entities also do it different ways etc. So the idea was floated to add pollution statistics. This would show where the pollution is being generated and absorbed, in just the same way as Item, Electricity, Fluid statistics etc.

By the end of office hours kovarex had the pollution statistics in the game, with some finalisation planned for later on.

Otherwise, a pretty typical day of finding bugs, fixing bugs, pushing fixes, etc. Rseding spent some time on the 'Back button' logic of the Manage Mods GUI. It is not just as simple as 'A button which goes back', as we have the neat system now that it will show a prompt about any unconfirmed changes still in the GUI, and what you want to happen.

In the evening, kovarex finished off the pollution statistics, adding statistic for the tree absorption and the tile absorption. Very often kovarex will work a 'night shift' for a few hours after he has wrestled his 3 young boys to bed. Another change that was prompted while looking at all these statistic GUIs was the statistic smoothing logic. The way it was before, all statistics had a smoothing of 0 except for the Item statistics, which meant they were extremely spiky.

We decided to enable the smoothing for all statistics apart form the Electric network statistics (as by its nature it is smooth). We think the result is much nicer:

Wednesday

A pretty typical day of bugfixing. Kovarex finished off the pollution statistics, and started on normalizing the pollution values. The internal prototype value was renamed to emissions_per_tick_per_watt, and the old prototype value of emissions will be internally converted if it is present. Another change was to amend the tile prototype pollution absorption definition. Currently it is called ageing, what does it even mean? So its now renamed to pollution_absorption_per_second , which gives some hint as to what it does.

The next step in this reworking was to show the pollution as a 'x/s' value, as opposed to the current unexplained number. When shown in this format to the player, the numbers were very large and not 'clean', such as '83.33/s'. The goal was to 'normalise' the values, so we have clean numbers such as 1/s, 5/s, while keeping the balance relatively the same. The general result is that all internal values are roughly divided by 60, and there were many migrations which needed to support this:

• Amount of pollution on each chunk.
• Amount of pollution absorbed by spawners.
• Amount of pollution absorbed by trees.

The result is something like this:

After lunch, posila added a new graphics setting for the macOS version: Render in native resolution. The game will by default tell macOS to render the game in the native 'Retina' resolution, however some people were having performance problems on older and weaker MacBook's, so we added the setting so it can be turned off. It is tough developing the game sometimes, a lot of macOS players were telling us for a long time to support the high DPI retina screens, and that they can't believe that we haven't done something as basic as setting the config flag. So we added it for 0.17, and we get another group of players complaining that we turned it on. Sometimes you can't win.

Thursday

After finalizing the pollution migrations and ensuring all the tests pass, the pollution changes were merged to master. The plan was made to make a release which will include the changes, so if there are any bugs we can fix them on Friday before the weekend.

With it merged to master and the values somewhat figured out, this is what the new pollution graph will look like in a typical factory:

Another nice addition, was adding the information about what biters will spawn and how much pollution the spawner will absorb when it sends a unit to attack:

This allows players to directly estimate the magnitude of the response of the enemy based on his strategic decisions. For example, you can roughly estimate how long the stockpiled ammo will last in a mining outpost, based on the mining drill pollution generation and spawners attack generation. We are not expecting people to do it regularly, but it is always nice if there is a way to inspect the mechanics of the game closely for those who care.

It is also on Thursday that we normally prepare the FFF, which means writing up the topic, preparing GIFs and images, finalizing the features and applying a lick of polish. Sometimes the deadline of FFF is tight, but it works well to push us to get things finished in a reasonable time. We've experienced the problem where if you have infinite time to work on something it ends up taking infinite time to finish.

About 16:30 we decided we should get the deploy started, and made the internal announcement. This internal announcement lets everyone on the team know, gives enough time to finish and push any work, and invites them to speak up if there is a reason delay the release. One of the final things for the 0.17.12 release was merging a branch by Dominik related to modded underground pipes, specifically moving the underground connection support from the PipeToGround class into the FluidBox. When we have larger changes like this, we process it through Pull requests. Rseding as our nominated code reviewer will go through the PR for any bugs or issues, and if it all looks good, he will handle merging it to the master branch.

So it all looked good for the PR after some minor cleanup, so it was merged, and after tests passed we started the release process. At just about 20:00 CET the whole deploy was completed, 0.17.12 was now available for all experimental players.

Friday

In a way, the plan worked, as after the release we had some feedback about the 'things we missed':

• Migrating the map setting of how pollution increases the evolution factor. (pollution was increasing evolution 16.6 times faster than it should for existing saves).
• The emissions of the Steel furnace were too low compared to 0.16.
• Changed the map_settings_example.json to not have (now) ridiculous pollution values.

Having your evolution factor increase at 16x the normal rate? That's the thrill of playing on experimental!

We also made a plan to merge a batch of Introduction Campaign fixes and changes. We wanted to do it early in the day, so there is time to test it and make sure it works, and also that there will be enough time to do the fixes before we want to release. Even further, we need enough time that is there is a catastrophic bug introduced, we can do another release in the same day.

Typically on a Friday, the FFF is in the stage of finalization. The last parts of the topics are filled in, the images uploaded, and final editing starts. This week is somewhat special, as I am writing this day-by-day.

Its just past 5pm now, most of the team is winding down for the weekend. The count of bug reports on the forum is 366, which lower then the 372 of last week. It seems we might be past the peak of the curve, where the bug reports no longer appear faster then we are able to close them. We have 0.17.13 out, over 100 bug reports resolved in the last 7 days, and a few new features for you all to enjoy. I'd say its been a pretty good week. The last order of business this week is publishing the Friday Facts.

This latest release has a new version of the Introduction campaign, we would appreciate if everyone could give it another playthrough so we can get as much feedback as possible, and ideally have someone unfamiliar with the game to try it.

As always, let us know what you think on our forum