devtrackers.gg
It starts in 30 minutes (10am PST). I'll post any interesting tidbits here!
Expecting details on how they handle constant changes to content and how they kept up with constant changes in the living stories seasons.
External link →It starts in 30 minutes (10am PST). I'll post any interesting tidbits here!
Expecting details on how they handle constant changes to content and how they kept up with constant changes in the living stories seasons.
External link →What do you mean by 2 seconds? Do they test all skills and mechanics of the game in 2 seconds...? Also, THANK YOU for this information. I was very interested.
on average, every 2 seconds we are finalizing a test result. The tests take about 5 minutes to run each, but because we run them in parallel, the test completion rate is compressed significantly.
One of the interesting key points was the sheer number of groups that need to ingest these test results.
There's the product management team as a whole that need certain subsets of data.
The engineering team needs to know when things break due to code issues (features not working as intended).
The design team needs to know when certain actions aren't able to be completed (say due to a portion of a map being broken).
However, the results of tests are largely informational, NOT part of the main release process (i.e. all tests must pass or they absolutely cannot release). They aid the build process, not block it.
Automation results are still a key part of the release process, just not in a way that blocks developer workflow. There are far too many people contributing to each branch to stop all development when a test breaks.
As someone who works in QA, I would have loved to be at GDC to hear this.
I'm on twitter (@lancehit), and I'll definitely post any links to the talk there, once they're available.
If you don't mind me asking, how are the tests written? In a custom test suite? Or a popular framework?
Are the tests written in BDD or TDD style? How do you guys go about testing something in isolation? I can imagine mocking important points of your infrastructure would be difficult.
We built a thin (200ish lines) harness in python that has a TDD style markup. Its very similar to something like nUnit or visual studios mstest harness. Has basic fixtures and a bunch of specialized attributes so we can do test selection and reporting.
Testing certain stuff in isolation can be hard. Some areas of the game are easier than others, and we work with the other programmers to improve those areas so they can be tested with automation.
We don't really mock the infrastructure, what we do is use specialized maps that reduce the amount of noise the tests are exposed to, and many of our tests use test content that can give us the ability to skip prerequisites, etc.