almost 4 years ago - jtoddcoleman - Direct link

Hey folks,

We put a new version up on Test last night -- this has a number of performance improvements, and a few known issues that we should probably highlight.

the Good

- First, we found a few improvements in the rendering pipeline. This should give a frame rate boost on all machines.

- Grass cards have been improved to look better (no razor edges along the bottom) AND to perform better. You can see the difference by turning your settings back to "Basic" ... but frankly, the overall performance improvement to this system means that you might as well run in Medium or higher.

- We added a new culling system to help performance as well. "Culling" is a way of removing items and effects from the scene to improve framerate -- and, in theory, you shouldn't notice anything but increased performance if we are removing the right things. (again, that's in theory).

- We fixed the framerate stutter that happened when you looked at harvesting interactables,

- Finally, we managed to get unity garbage collection under control! This one has been a long time coming, and what it means, in layman terms, is that you should see far less hitching as you run around the game. Pay attention to the "SPK" (spike) counter in the upper left; you'll probably see it hit 4 or 5 when you first log in, as the world loads around you, and then it should settle down and stay at or near 0. That's the hope, anyway, so let us know if that is what you are seeing.

the Bad

- In practice, the culling algorithm is making decisions for you about what you do/don't want to see, and algorithms are notoriously prone to error. We've exposed settings for this in the preferences screen, so please try out different settings to cull things more / less aggressively and let us know if you can find the sweet spot.

- Eventually, we need the culled objects to fade in and out of view rather than just popping. Once fading is in, the culling of objects will be less noticeable.

(We have more advanced culling in the works, as well, but even this first iteration should pay dividends!)

- Some items that were created in previous builds look like they have bad mouse-over information. Mouse-over popup info for new items is working fine, and the old items are (I think?) just as functional as before, but the info being shown for them is incorrect. The easy fix for this issue would be to wipe items, but that kind of sucks for testing, so let's see if we can hold off on doing that unless it becomes absolutely necessary.

& the Ugly

- There is a bug (potentially related to culling, but maybe not?) where you collide with something on the server that you don't see on the client, and after a moment it causes your character to pop/sync/teleport. This is especially bad around outposts and hero statues/respawn locations, two locations where I'm sure you're really delighted to be unexpectedly yanked through space.

- Frostweaver targeting seems borked.

- There is a particular power that seems to be causing a server exception. Bad, bad, bad.

- Some state seems to be causing powers to stop working in a zone? This bug seems wiley, but thankfully rare.

- There was an edge case that caused wall upgrades to be lost on a server reboot. We believe this issue is now resolved, but please be on the look-out and let us know if it happens again.

In all of the above cases, we're looking at the Ugly bugs first, and we'll move on to the Bad once those are done. As always, we working to get the issues resolved as quickly as possible.

"Why did you prop it, with known issues?"

That's a great question, thanks for asking.

Two answers to that: first, we don't always know all the issues -- or, we sometimes know an issue is there (as we did with teleport) but it turns out to be much worse in practice than we expected it to be. and second, sometimes we don't know about the issues until the build is tested in front of real players. Crowfall is a pretty beefy game at this point, with a lot of interlocking systems, and even with internal testing + our overseas testing firm that does a full set of tests every night while we are sleeping, we still don't catch every issue every time. There isn't enough time in a 24 hour cycle to touch every single [ power, recipe, loot roll, monster, victory card, building, interactable, etc. ] in the game.

It's probably worth reminding everyone that this is the Test server, and there are always bugs on the Test server. The more tolerant we are of pushing builds out to Test with known bugs, the more builds we can push! and the opposite is true, also: if we want to ensure more quality, it means we prop less often. In this case, we've gone several weeks since our last update, and that means hundreds of fixes waiting in the wings... and in the meantime, no way for players to harvest rare resources. It seemed like a good idea to get it out and let you guys test it over the long weekend, even at the risk of the overall experience not being 100% playable.

If you find these kind of issues too frustrating -- and believe me, I can certainly understand if you do -- then I would caution you to wait until the game migrates to Live -- where the issues will tend to be bad rather than ugly. For those of you who are willing to help us test, know that we appreciate it greatly, as the faster we can find the fixes the faster we can get them resolved and get this thing out to Live.

Hope that helps shed a little light on the method to our madness, and see you in Test, if you brave it... and if not, I hope you have a great memorial day weekend!

Todd

almost 4 years ago - - Direct link

Hey folks,

We put a new version up on Test last night -- this has a number of performance improvements, and a few known issues that we should probably highlight.

the Good

- First, we found a few improvements in the rendering pipeline.  This should give a frame rate boost on all machines.

- Grass cards have been improved to look better (no razor edges along the bottom) AND to perform better.  You can see the difference by turning your settings back to "Basic" ... but frankly, the overall performance improvement to this system means that you might as well run in Medium or higher.

- We added a new culling system to help performance as well.  "Culling" is a way of removing items and effects from the scene to improve framerate -- and, in theory, you shouldn't notice anything but increased performance if we are removing the right things.  (again, that's in theory).

- We fixed the framerate stutter that happened when you looked at harvesting interactables,

- Finally, we managed to get unity garbage collection under control! This one has been a long time coming, and what it means, in layman terms, is that you should see far less hitching as you run around the game.  Pay attention to the "SPK" (spike) counter in the upper left; you'll probably see it hit 4 or 5 when you first log in, as the world loads around you, and then it should settle down and stay at or near 0.  That's the hope, anyway, so let us know if that is what you are seeing.

the Bad

- In practice, the culling algorithm is making decisions for you about what you do/don't want to see, and algorithms are notoriously prone to error.  We've exposed settings for this in the preferences screen, so please try out different settings to cull things more / less aggressively and let us know if you can find the sweet spot.

- Eventually, we need the culled objects to fade in and out of view rather than just popping.  Once fading is in, the culling of objects will be less noticeable. 

(We have more advanced culling in the works, as well, but even this first iteration should pay dividends!)

-  Some items that were created in previous builds look like they have bad mouse-over information.  Mouse-over popup info for new items is working fine, and the old items are (I think?) just as functional as before, but the info being shown for them is incorrect.  The easy fix for this issue would be to wipe items, but that kind of sucks for testing, so let's see if we can hold off on doing that unless it becomes absolutely necessary.

& the Ugly

- There is a bug (potentially related to culling, but maybe not?) where you collide with something on the server that you don't see on the client, and after a moment it causes your character to pop/sync/teleport.  This is especially bad around outposts and hero statues/respawn locations, two locations where I'm sure you're really delighted to be unexpectedly yanked through space.

- Frostweaver targeting seems borked. 

- There is a particular power that seems to be causing a server exception.  Bad, bad, bad.

- Some state seems to be causing powers to stop working in a zone? This bug seems wiley, but thankfully rare.  

- There was an edge case that caused wall upgrades to be lost on a server reboot.  We believe this issue is now resolved, but please be on the look-out and let us know if it happens again.

In all of the above cases, we're looking at the Ugly bugs first, and we'll move on to the Bad once those are done.  As always, we working to get the issues resolved as quickly as possible.

"Why did you prop it, with known issues?"

That's a great question, thanks for asking.

Two answers to that: first, we don't always know all the issues -- or, we sometimes know an issue is there (as we did with teleport) but it turns out to be much worse in practice than we expected it to be.  and second, sometimes we don't know about the issues until the build is tested in front of real players.  Crowfall is a pretty beefy game at this point, with a lot of interlocking systems, and even with internal testing + our overseas testing firm that does a full set of tests every night while we are sleeping, we still don't catch every issue every time.  There isn't enough time in a 24 hour cycle to touch every single [ power, recipe, loot roll, monster, victory card, building, interactable, etc. ] in the game.

It's probably worth reminding everyone that this is the Test server, and there are always bugs on the Test server.  The more tolerant we are of pushing builds out to Test with known bugs, the more builds we can push!   and the opposite is true, also: if we want to ensure more quality, it means we prop less often.  In this case, we've gone several weeks since our last update, and that means hundreds of fixes waiting in the wings... and in the meantime, no way for players to harvest rare resources.  It seemed like a good idea to get it out and let you guys test it over the long weekend, even at the risk of the overall experience not being 100% playable.

If you find these kind of issues too frustrating -- and believe me, I can certainly understand if you do -- then I would caution you to wait until the game migrates to Live -- where the issues will tend to be bad rather than ugly.  For those of you who are willing to help us test, know that we appreciate it greatly, as the faster we can find the fixes the faster we can get them resolved and get this thing out to Live.

Hope that helps shed a little light on the method to our madness, and see you in Test, if you brave it... and if not, I hope you have a great memorial day weekend!

Todd

 

almost 4 years ago - jtoddcoleman - Direct link

...it's a great question, we tend to not give too much direction because (1) we need help with coverage on all the things, and (2) you guys aren't employees, so we hate to ask for things that feel like "task assignment".

that said, in this particular case, a lot of what we've been focused on are performance issues. so any help with trying out the new culling system, and finding particular areas or situations that cause fps drops (other than the obvious, like "a ton of players using powers in the game area") would be greatly appreciated.

Todd

almost 4 years ago - jtoddcoleman - Direct link

Let's talk about that.

Internally, we cut new builds all day, every day. There isn't just "one version" of the game internally; we have different groups working on different builds (like right now we have a "culling version" and "new player experience version" and "fix powers" version) and these are all duplicates (branches) of the game that different teams work on independently and we can bringing them up separately and testing them in separate environments.

You have to have separate development tracks because otherwise when someone is doing open-heart surgery on one system, every

almost 4 years ago - jtoddcoleman - Direct link

Aha, this is helpful because I can leverage my previous post to answer! See my note above about branches -- the patch notes were erroneously generated to include some fixes that were in branches not included in this particular merge.

Todd

almost 4 years ago - jtoddcoleman - Direct link

I'll pass that along to our engineers, I know one of the initiatives they are still working on is character loading optimization.

Todd