[2019] [Discussion] Riot Support says we should discuss a reworked leaver system

Both u/HubblePie and u/sinderling touched on the heart of the problem here:

1) How do you determine intention based on behavior?

(To a lesser degree, there's the additional problem of "How do you execute (1) in such a way that people cannot game the system?", but let's focus on the main problem here)

It is really really difficult to be absolutely certain of what someone else's intention is, when all you can observe is their behavior. And I'm not just talking about software and games here - this is a good life lesson to remember when you're dealing with problems between you + your friends/family/coworkers/etc. In League specifically, did someone disconnect because they ragequit or because their internet went out? Was someone's score atrocious because they intentionally fed or because they simply made some mistakes or had an off game? Side note, the term "soft int" pisses me off because it literally means "intentionally feeding, but behaving in such a way as to be indistinguishable from someone having a bad game" - unless you are psychic and privy to someone else's mental state, you cannot claim that someone is "soft inting" by definition *.

From an engineering perspective, we don't necessarily need absolutes. We can create systems that work under degrees of uncertainty (like a machine learning model trained to detect the likelihood that someone disconnected because they rage quit?). In this case, the problem can be amended to look like:

1) What degree of certainty of intention is acceptable for our system to reward/punish a player?

From this perspective, if you consider what happens in the case of false-positives and false-negatives, I think you'll begin to understand why Riot currently has these things set up in the way they are. Consider disconnects in this case - if our "rage quit" detecting system hits a false-negative, a rage quitter gets off with little punishment; on the flip side, if this system hits a false-positive, an innocent player gets punished unfairly! The expected value will change a bit depending on false positive/negative rates, but the core idea is that false-positives suck immensely more for a player than a false-negative, so whether it be rage quitting or inting or anything else, we want to err on the side of caution and not punish innocent players.

All this said, I do want to explicitly address that players' feelings of frustration at the limitations of our current systems are definitely valid! And the teams in charge of these systems work hard to improve them every day. That's why I typed this all out - so players better understand the magic that's happening behind the scenes and we all can have more productive conversations constrained by the limitations of the systems we're dealing with. I hope this helps, and I'm happy to have more conversations are these topics here or wherever/whenever else they pop up around here :)

(*caveat - if you think you have data-driven ways to determine if someone is soft-inting, you more than likely a) using information outside of our system, and thus would not be feasible to incorporate into a soft-inting-detecting system, or b) describing diverging behavior between soft-inting and non-soft-inting, which circles back to the lesser problem of "not making the system gameable", i.e., soft-inters will stop doing whatever new behavior we are looking out for, which brings us back to where we started)

"Appealing their leaver punishment" feels like a good amendment to make to the system, but I think it reduces to the same problem we originally have: how do you determine if someone left for legitimate or malicious reasons? And when you start to think about specific things someone could show to demonstrate their innocence, how can we make sure those things can't be faked by malicious players? It's tough, it really is...

I like how "not gaining/losing LP if you get flagged by the leaver system but your team still wins" sounds initially, so I'll iterate on some thought I have about the specifics below.

1) if you leave in the middle of the game and don't come back, regardless of your intention, you've put your team at a significant disadvantage. The fact that they overcame that and won is awesome, but we can't really give them more LP for overcoming that hurdle (that's just ripe for LP abuse). In this case, I think it makes sense to punish the player for how they punished their team, because it disincentives both malicious leaving (don't ragequit!) and other types of leaving (make sure your internet is stable, don't queue up if you have 15 minutes before you have to go to school, etc). Some things are out of your control (internet goes down), but I would like to point out that we have loss prevention systems to save your LP from the BIG things that are out of your control (internet goes down because, e.g., a shark chews through the undersea fiber optics cable that connects Australia to our OCE server )

2) if you leave a game and do come back, we can't just say "no worries, all is forgive", because, again, you've kinda sorta put your team at a disadvantage, regardless of intention. So maybe we say that you won't lose LP if you play as usual and help your team win. But here's the tricky part - now we're back to trying to determine someone's intention, and do it in such a way that malicious players can't abuse it. How long does a leaver need to play after they come back? Do they need to farm X cs or get Y k/d/a or end at Z level compared to others on the team? When we think about it this way, you can start to see how rage quitters can toe the line and behave like they're intending to play legitimately. Also consider that X, Y, and Z can vary across skill levels - iron players might never reach X cs even if they never disconnected in the first place, or diamond games might not get so bloody that a reconnected player could improve their k/d/a. All this to say is, maybe we could define "legitimate behavior" we want to encourage if someone leaves and rejoins, but as soon as everyone knows the secret sauce, it's no longer effective.

I know I just said in my previous comment that "we want to err on the side of caution and not punish players for false positives", but in the case of leaving a game, it's important to consider impact of the behavior, not just the intention behind it.

oh boy, I see where I mess up here. I'm sorry for not being more clear in this regard! 😅 we do have a team whose purpose is researching and solving these problems, and they do hire smart PhD CS and data science folk and they do use ML algorithms to solve these problems, so you're not wrong in that regard!

My intention was to highlight some domain-specific constrains when dealing with the general idea of "punishing leavers", as well as show the different trade-offs that we might consider when building and fine-tuning the systems we use to execute this idea. We all want the same thing - punish the toxic players, and don't punish the non-toxic players! And while no system is perfect, acknowledging that fact does not make your pain around this system any less legitimate. My hope is that when players read what I've written, they understand a bit more about what's going on behind the scenes and "why things are they way they are", but also that Riot hears their pain and is working to fix those issues! It seems like I've neglected that latter part, and for that, I am sorry.

I wish I could talk in more specifics about the system, but I don't work on that team and I don't know the specifics. 😞 (let me reach out to the team tho, see if there's some more insight they can provide!) I do know that the system isn't a black-and-white punishment system (so you're spot-on with the granularity idea!), and I do know that the team who works on it does experiment with the system and does research on ways to improve it! I just want you to know that 1) we hear ya, 2) we know it sucks (we feel ya!), and 3) we're always looking for ways to prevent these things from happening, or reduce the pain when it's hard to prevent, so we appreciate players like you who care and give us this feedback! 😊

Hey u/AtakuHydra, hope you're doing well!

I chatted with some friends on the team that owns Leaver Buster and learned some cool new things about it. Way back in patch 9.6, they released some big changes to the system that should encourage players to reconnect to their games and help the team win. Not every game that you disconnect->reconnect->win will forgive you leaving, but in some cases, winning the game is the difference between being marked a leaver or not. They wouldn't go into the specifics of how it works (in case I accidentally leak something 😅 ), but the general idea is that the best way to not get marked as a leaver if you disconnect from the game is:

1) minimize time disconnected

2) get back in the game as soon as possible

3) play the game

4) do your best to win the game

I hope this answers some of the questions you had! If you're on NA, let's do an ARAM together sometime :)