devtrackers.gg
over 1 year
ago -
Play New World
-
Direct link
Transcript (by Youtube)
| 0s | Another real hot topic in game development that New World isn't immune to is, when is it good enough? |
|---|---|
| 7s | At some point, you know, you really have to push things out by certain deadlines, otherwise you're never really going to get something out. |
| 20s | Welcome to this episode of Forged in Aeternum, where we talk about everything New World. |
| 24s | This time we're going to talk about finalizing a release. |
| 27s | I have with me Michael Corpora from our QA team and Derek Hernandez from Live Operations, but who used to be a release manager and we'll be speaking specifically about prerelease things. |
| 41s | So let's get started with something that all of the players have already experienced, right? |
| 47s | Which is the Brimstone Sands release. |
| 50s | Okay. |
| 50s | So everybody's already developed. |
| 53s | Now we're into what phase, testing? |
| 56s | Like, what happens once everyone's done with those features and they go here? |
| 60s | Yeah. |
| 60s | So that's an exciting part for us because that pretty much means that we get to go into our final testing period where kind of like we fork our activities, we begin some of our preparations where we start working with - more closely with - our QA partners and with our Community team members. |
| 81s | And on one hand, we start ensuring that everything has come together collectively, works well together, is as we expect it to. |
| 94s | On the other hand, at the same time we start preparing our PTR - or public test server builds - where we invite our players to help test and provide feedback along with us to make sure that not only this content meets our own expectations, but it meets the expectations of our players too, and it gives them a chance to provide their feedback and their input based on what they see of the content that we've just recently developed, and based on their results and based on what QA finds as well, we're able to make some of the changes before we actually finalize everything at the last minute, and then we hopefully have made a new fantastic game build for everyone to enjoy next. |
| 140s | So then testing kind of takes two routes, right? |
| 142s | So you mentioned the public test server, or the PTR, but QA - you also have to shift what you're doing from development testing to release testing, right? |
| 152s | Right. |
| 153s | Prior to going out to our public, it's important that we look at the game as it is as a whole. |
| 160s | Prior to that point, we're mainly focused on new features, but at that point where we branch, we start finalizing toward release, we have to look at the change that's occurred across the entire game. |
| 173s | So we have a very wide spectrum test that we begin at that point prior to PTR in hopes to ensure that, you know, game modes, things that necessarily weren't focused for the release, are still functioning properly as well as of course that the integration of those new features are in a place where we can put it out to our players, we can get actual good feedback because they're not completely broken, although, you know, those early builds go into the PTR, sometimes, it's a toss up depending on how much time we've had in order to look at those features to make sure that the integration is good and that everything is stable. |
| 213s | Stability going into PTR is a big thing. |
| 216s | If we're not stable, then not only we're not going to get feedback on those features, but now people are going to have less of a good impression of the upcoming thing that we're trying to release. |
| 226s | Yeah, and that - so we got all of that testing with Brimstone Sands. |
| 231s | We did have some very quick turnaround changes that we did after we released with that one, but that one had a pretty good solid long time on the PTR and we also had a lot of time with testing with it and I would consider that a very successful release. |
| 246s | And then we had another release right after that that did not really hit the PTR, right? |
| 252s | That was the Winter Convergence Festival. |
| 255s | So what is like why, why do we make those decisions to withhold something from the PTR if then when we put it out there, there might be some performance issues like what we saw with Winter Convergence Festival? |
| 268s | Definitely. |
| 269s | That's always a good question and there's no real true answer for that. |
| 274s | And that's something that we're constantly asking ourselves, you know, whether or not we should or shouldn't put something that's brand new on the PTR. |
| 282s | One of the things that we like to do as game developers is probably very cliche, is surprise and delight the players. |
| 288s | And for that point, new content or new events really are really cool things to jump in-game and really shake up the day-to-day humdrum. |
| 300s | And uh, there's kind of two opposites pulling at each other here where we want some things to be surprising and some things to be brand new to all the players. |
| 311s | But on the flip side, we want things to be stable. |
| 314s | We want things to be performant. |
| 316s | We want things to be enjoyable and balanced. |
| 319s | And then so sometimes, especially for live events that are more temporary content, we may make the concession to say, All right, let's do some more extra testing internally. |
| 331s | Let's get some more play tests internally. |
| 333s | Let's work closer with QA to ensure that we try to copy all edge cases down and stuff like that and let's, you know, sneak in an extra activity for the players. |
| 344s | Sometimes those plans don't exactly work out right, and then we sometimes get an extra bug or two. |
| 350s | But that's unfortunately the risk that we have to do in trying to weigh those two options. |
| 356s | But yeah, that's something that's that's very, you know, cognizant question that we're always asking ourselves. |
| 365s | And based on feedback of what something works, something that doesn't work will inform our next choices too. We didn't do it for one event and we might do it for another event based on, you know, what lessons we learned. |
| 380s | And as a team, we're always learning from our mistakes and learning from what went well, what went wrong, and trying to improve on the next point for sure. |
| 391s | So then when do we know that the Winter Convergence Festival is ready? |
| 396s | That's a great question. |
| 399s | So I love the idea of being able to delight our players, right? |
| 403s | But my QA center says, err on the side of caution. |
| 407s | Put it all in the PTR. |
| 408s | I put it on the PTR, put it all out there! |
| 411s | Because even when we test these things internally, we do quite rigorous testing, not just within QA, but within the entire dev team, right? |
| 420s | There are things that we'll find, you know, in the case of Winter Convergence there, most of the issues we actually did find internally, but the fixes we put in place were not altogether sufficient. |
| 432s | And having the exposure on PTR, in particular in terms of scale, is always going to eliminate things that we just can't hit in terms. |
| 440s | Like the way that players are going to play. |
| 442s | Exactly. |
| 443s | Exactly. |
| 443s | Or abuse the game such as it is, you know, which happens. |
| 448s | We try to - we try to, you know, get all that stuff out and identified. |
| 452s | Yeah, but it's a fine line I think and. |
| 457s | And that really kind of leverages, too, just another real hot topic in game development that New World isn't immune to is, when is it good enough? |
| 469s | And as as a game you really want to make sure that everything is perfect and you know, anything creative, really, you want to hold that baby until you know that it's ready and you're comfortable for it to, like, leave the nest. |
| 483s | Right? |
| 484s | But at that point, it'll never really be ready. |
| 487s | You always find a part to polish up to fix. |
| 490s | And so at some point, you know, you really have to push things out by certain deadlines. |
| 495s | Otherwise you're never really going to get something out. |
| 498s | And then so at that point, based on testing, what we see, based on the feedback, we do sometimes have to make judgment calls to figure out, all right, do we have enough time to to make this change? |
| 509s | Do we want to make this change for next time? |
| 511s | And you have to weigh that against the priority of that issue that you've discovered. |
| 515s | Exactly. |
| 516s | Like, for instance, the things that were mitigated. |
| 518s | Were they mitigated and like substantially enough that we feel confident in releasing this and then maybe doing a fast follow with a fix or not? |
| 526s | And so we're using like limited information, essentially, since we don't have that testing from the PTR at that point. |
| 532s | Right? |
| 532s | Because that's pretty late in the game. |
| 534s | And sometimes we actually have fixes that we're pretty confident in based on what we see in the code and what we are able to do in test internally. |
| 543s | But then sometimes there are still edge cases or there are other means to still do this bug that we never really see internally, just due to the amount of people. |
| 552s | No matter how many testers we can throw at it, it still doesn't match the thousands and thousands of players that are able to experience that change at any point in time. |
| 560s | So, uh, so think of Dr . |
| 563s | Strange and the multiverse, right? |
| 565s | Yeah. |
| 566s | Different possibilities. |
| 567s | Yeah. |
| 568s | "Good enough" is one of the most challenging concepts I think that I've ever had to embrace in my career. |
| 573s | Because, yeah, we find these things, we, we put in fixes for them. |
| 576s | We do test explicitly against the, the thing that we found that was wrong and we test around that of course. |
| 584s | But there's always that point where we're ready, we're expecting to release, where we need to release. |
| 594s | We can't take all the time that we would maybe want to take. |
| 601s | So we're, we're making the best of the time and resources that we do have available to identify issues and generate solutions that we feel are good enough to release to our players. |
| 616s | And that's done in a pretty short period of time. |
| 619s | Like when you consider the total development cycle. |
| 621s | What you're talking about is not like a massive part of it. |
| 625s | That's really like a very truncated but very testing-focused piece that happens just before we do a release. |
| 634s | Right? |
| 634s | Yeah. |
| 635s | Like there's, you know, spot testing throughout and we test the builds in general, but this is really the time where that build gets our full focus. |
| 643s | Yeah, absolutely. |
| 644s | Like depending on the piece of content or feature, certain things could be in development for a year or months or just weeks, too. |
| 653s | So depending on the release, things kind of are in quasi-existence. |
| 660s | And in these targeted releases is when we actually bring everything together and we see, do they work actually as expected together? |
| 670s | Does this look well together? |
| 671s | Does it play well together? |
| 674s | Then sometimes, you know, the things that we do find don't work well together and we do make the decision that like, hey, this isn't good enough and we do need to spend a couple of extra time that we originally anticipated in order to to make this good enough or, you know, to make this at our benchmarks. |
| 694s | So in that case, you know, we would make the decision to delay in order to make sure that we deliver something of quality where we can and where we know. |
| 705s | Yeah, we've shown that we're yeah, we've shown that we're not afraid to make that decision, you know, because it benefits the players, really. |
| 711s | So that's a very super high stress time. |
| 715s | Everything is moving super fast. |
| 716s | I know that. |
| 717s | And then we release a build and then we get feedback from players. |
| 720s | So I mean, that's really a very high level, but also kind of in-depth way of explaining how we finalize our builds, our major releases before we get them out. |
| 731s | Yeah, definitely. |
| 732s | So with that, let's talk about our Community Question. |
| 737s | What's something that you can't wait to see on the PTR? |
| 741s | So make sure to like and subscribe. |
| 744s | We love to see you guys interacting on these videos and telling us what we can do next. |
| 749s | Thanks for coming. |
| 752s | Our vision for how we perceive rewards in the game is that players should be making measurable progression every time they play the game. |
| 761s | Yeah, even on the 100th time there it is. |
| 763s | Even the 100th time. |
| 764s | So like when you're doing something, if it's 100th time you're on an Expedition, you should still feel like you're making progress towards some goal or have the opportunity to get something really cool from it. |