[2022] Status Updates - Untradeable Items Bug

over 2 years ago - /u/JagexLight - Direct link

Hey everyone, I've made this post to update you quickly on the status of our investigation into untradeable items being bugged following the engine changes due to this week's game update.

This is really unfortunate and I want to make sure we're communicating as clearly as possible with you as we work to resolve this issue. Please check back here for any further updates. The team is working as quickly as possible.

(12.30) We've shut down the game after seeing reports of untradeable items being bugged.
(12.55) We can confirm a rollback is necessary and the team are now starting the process. The rollback will take us back to the time of the game update at 11.30AM. We apologise for the inconvenience caused.
(13.05) Previous rollbacks have taken us a couple of hours to perform. We will keep you updated and provide you with an ETA when we have one. There were a few minor issues we noted with the minigame during the update, so the content developers are investigating what they can coldfix while our server team is working on the rollback work.
(14.52) Good progress has been made on restoring the game to just after today's update. (11.30am approximately). During this time we've also resolved some of the issues you raised with Guardians of the Rift. We'll provide an ETA once we're comfortable the game is good to come back online!
(16:22) The team has completed content fixes during the process of the rollback and server downtime. We are now in the process of testing the worlds before they come back up to ensure that they are working as intended. We appreciate your patience throughout this time. Any ETA is dependent upon how the testing goes. If testing goes well, the game should be up within the next hour. A more accurate ETA will be provided when we have it.
(16:37) Good news guys. TESTING IS CONCLUDED. It went well. Worlds are coming back online shortly. I am aware I never gave you an ETA, I'm sorry about that. Didn't want to get your hopes up and then disappoint. Let me know your feedback for GOTR and I really apologise for the inconvenience

External link →

over 2 years ago - /u/JagexLight - Direct link

Hey everyone, I've made this post to update you quickly on the status of our investigation into untradeable items being bugged following the engine changes due to this week's game update.

(12.30) We've shut down the game after seeing reports of untradeable items being bugged.
(12.55) We can confirm a rollback is necessary and the team are now starting the process. The rollback will take us back to the time of the game update at 11.30AM. We apologise for the inconvenience caused.
(13.05) Previous rollbacks have taken us a couple of hours to perform. We will keep you updated and provide you with an ETA when we have one. There were a few minor issues we noted with the minigame during the update, so the content developers are investigating what they can coldfix while our server team is working on the rollback work.
(14.52) Good progress has been made on restoring the game to just after today's update. (11.30am approximately). During this time we've also resolved some of the issues you raised with Guardians of the Rift. We'll provide an ETA once we're comfortable the game is good to come back online!
(16:22) The team has completed content fixes during the process of the rollback and server downtime. We are now in the process of testing the worlds before they come back up to ensure that they are working as intended. We appreciate your patience throughout this time. Any ETA is dependent upon how the testing goes. If testing goes well, the game should be up within the next hour. A more accurate ETA will be provided when we have it.
(16:37) Good news guys. TESTING IS CONCLUDED. It went well. Worlds are coming back online shortly. I am aware I never gave you an ETA, I'm sorry about that. Didn't want to get your hopes up and then disappoint. Let me know your feedback for GOTR and I really apologise for the inconvenience

External link →

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Isthatyouson

ty for wasting my time

I'm extremely sorry for the inconvenience caused.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by lowcalredbull

Thanks for taking the time to update us. Any idea how long this will take?

I don't have this information - no ETA yet. Typically they will take a few hours to resolve though so don't hang around if you need to get something sorted. I'll make sure to update the OP post when I have an ETA.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by scarvalh0

I just tripled my bank at the dual area 20 min before the update. I hope I don’t have that progress lost LMAO.

As far as I'm aware, the rollback will bring you to just after the game update at 11.30am. I will update the OP post if this changes.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by SalahScoresAgain

Thank you for the fast response. Does this mean that all KCs for the boss will be reset back to 0?

I believe so yes, I am extremely sorry for the inconvenience caused. We're working on getting your save to back to just after the game update so 11.30am.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by BoxOfDemons

Did the team give you an eta on when the game will be back up?

I don't have an ETA but as soon as I do I can update the OP. They will typically take a few hours in the past. The team is looking at the point of how to roll back the clan login servers as well.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by ShivAGit

Each of these weird bug things suck individually and I feel bad for the Customer service people, but at some point Jagex need to own up to it and stop. Whether thats better QA, more focus on code reviews or just anything, kind of embarrassing that it's more likely than not recently that things will f**k up on release of content thats hyped up.

Thank you for your feedback, it's definitely noted. A rollback is definitely not what we wanted for today! We'll absolutely be reviewing in the appropriate meetings - no blame really, but to ensure that we learn and improve from this. I apologise for the inconvenience caused.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by LivingLegend8

Wait… how long ago is the rollback??

The game has been shut down since 12.45 or so I think. We're working on rolling back to 11.30am

over 2 years ago - /u/JagexLight - Direct link

Originally posted by lllmatic

How many hours is getting rolled back?

Back to 11.30am of today

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Lazy_Inferno

ETA?

I don't have an ETA yet but I'll provide in the OP if I get one. Typically these have taken a couple of hours in the past. Currently just letting the team do its work and will update you on any status changes as I hear them

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Osmium_tetraoxide

No worries. Bad things happen in software development. I hope that a postmortem happens at Jagex and something changes to the process so big bugs like this don't escape.

Maybe twice a week releases so the random changes like death piles happens before any quests/major pieces of content so people who do quests on release are less likely to have to do it again. But whatever the outcome it'd be nice to see what is learned in any postmortem and hopefully things like this happen less often.

Change management is hard to get right and at the end of the day we lose 30 minutes of progress, disappointing but oh well.

Thank you for your feedback and politeness. The engine doesn't get updated very regularly and it was planned to be updated this week, so we felt that it either went into this engine update or players were left unsure of when it could come in. Engine changes are less frequent than our regular updates. We will definitely review this and see what we can to do improve. I apologise for the inconvenience caused and the progress lost, it really sucks

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Noksdoks

Do we have to do the quest again?

Yes, I apologise for this. Your progress should be set back to 11.30am

over 2 years ago - /u/JagexLight - Direct link

Originally posted by I_Ride_A_Nimbus

^This.

I saw plenty of comments on how XYZ isn't being done yet because of so much QA being done. Well now we have QA being done that doesn't seem to matter (IE we're having downtime for every update, sometimes massive downtime like Leagues) AND we get told fixes can't be made because of this ineffective QA.

I don't want to speak for other people. It was a lot going into this week's update, extremely ambitious and it is very likely our resource was focused on ensuring the minigame was as tested as possible. I'm sure this will be reviewed by the team and I understand the frustration. I apologise for the progress lost, we'll get the game back online as soon as possible.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Reubachi

Why on the updates page is this not listed under "issues and hotfixes"?
https://secure.runescape.com/m=news/guardians-of-the-rift-launch?oldschool=1

I get that it's right above it as a blurb, but I'd think this qualifies as something that needs to be documented under a category called "issues and hotfixes". Otherwise, once this is resolved, no one will be able to confirm that there was a rollback.

I've got the thread linked at the top of the newspost, does that work OK?

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Selesia_read_it

u/JagexLight from the RS3 community: we care about both games, if you're going to turn off the login servers, could you kindly inform us too....

Hey there. I'm so sorry about this. I have asked one of our engine team and they confirmed the issue you're experiencing is likely not connected to the OSRS stuff going on. We would inform you if we knew that was happening.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by DrowZi

Any available eta?

Not yet but I'll update as soon as I know of one.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Bunburial

you guys know the E in ETA stands for "estimated" right

Working on getting this to you as soon as possible. We're relatively close to the testing stage now so I'll post the next update soon.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Hinty-RS

Appreciate the updates, but you’re providing an estimated time only once you know when the time will be? Isn’t that counteractive to the purpose of an estimate? I appreciate not wanting to set false expectations, but an estimate should be just that, and not a last period countdown.

I totally understand this. The team is hesitant to give you an incorrect time but I've pushed a bit more for a 'best estimation'. We're relatively close to the next update (nearing the testing phase), and when we're in it I'll update the OP.

over 2 years ago - /u/JagexLight - Direct link

Originally posted by Ergunno

I appreciate you giving us updates on this, Light, but what's the point of an ETA if we won't get one until you're comfortable the game is good to come back online? We're gonna get an ETA once you guys are ready to put it live again?

I would've expected an ETA of like 6 hours, and then either pulling it back or pushing it forward depending on the progress you've made thus far. This is me-problem, but with your approach, I've just found myself refreshing this page and the twitter a whole bunch of times trying to get an ETA on when I can actually play again. Getting an ETA half an hour before the game's back literally defeats the purpose of an ETA at all if the downtime is several hours.

You're getting better as a team of informing us about downtime at least - it's actually been in the blog post for once.

I totally hear this feedback, I would love to keep you as informed as possible. We're relatively close to the testing stage and then I'll be able to give you a slightly better update via the OP. The team is very hesitant to give time-frames in case we miss them

over 2 years ago - /u/JagexLight - Direct link