[2020] High MMR matchmaking comparison EUW / NA / KR

Interesting analysis! I have some more detailed takeaways, but here is a quick TL;DR:
EUW did have a configuration difference that was optimizing for faster matches at higher MMR for a little looser matchmaking. The actual Match Quality is way closer when measuring by the metrics we actually use to make games (not LP), but there's some room for improvement so we're going to shift it to be closer in line to other regions.

- First, cool idea and thanks for bringing this stuff up!
- So as you know, we don't match on LP, we match on MMR. The two numbers correlate pretty well up at the top of the ladder, but there's a lot of ways it can differ from region to region. So while the numbers can look pretty jarring here, they're much closer in reality. For example, looking at only players with Master or higher MMR, the 90th percentile of worst games still had both teams within ~1 win's MMR of each other in all regions. Differences between teammates were also extremely similar across all regions.
- EUW was configured slightly differently because of some changes from back when Position Ranks came out in NA and KR
- The reason basing this analysis on visible rank makes the problem look a little worse is that it assumes all Master/GM/Challenger players are a uniform skill distribution across all regions. In some regions, the ladder is very top heavy, with the best players having a fairly sizable lead. Some regions are middle heavy, with some outliers on the high side taking the Challenger spots. So one GM player may be substantially different than another GM player, MMR-wise, and averaging out based on tier can make a match look worse than it is.
- While LP and MMR correlate pretty well at the top of the ladder for accounts with a lot of games, this is less true for newer accounts. A Diamond I player with a lower game count may be actually a Diamond I player, or a Challenger player still on their climb. Some regions have higher incidences of multiple accounts, and that can make any kind of LP-based matchmaking analysis a bit misleading.

I don't want to let those quibbles take away from how useful this was, though, so thanks again!

-

Posted above! Thanks for flagging me

It actually did before I finished writing that post - the change itself is pretty simple.

We have a matchmaking rating (MMR) that we use as a current estimation of your skill. When we make a game, we try to find players with similar MMR, but also account for factors like how long they've been waiting, if they're in a premade, etc.

So defining the "quality" of a match is something we approach from a lot of different angles, but for the purposes of "is the matchmaking system working correctly," we generally care about how close teammates are to one another, and how close teams are to one another by MMR.

For post-match analyses dependent on the result, we also do that - getting matches with close MMR differences only works if the rating is correlating properly with actual skill/wins. Loosely speaking, for any given match, we can assume that the team with the higher MMR is predicted to win - and the difference gives us a level of confidence in our predictions. We're aiming to keep matches as close to 50/50 as possible, so when a match is close, the prediction is less confident because both teams could very easily win. We then check that prediction against the actual results to make sure the system is predicting correctly, and especially so in cases where it has strong confidence.

Definitely not seeing the same stuff on EUNE, and this would only be affecting the very top of the ladder.

The change we made would not impact ARAM, I believe.

Flex ratings differ from solo queue ratings - they're informed by some of the same stuff, but not the same. The algorithms are almost exactly the same, but the base info is different. It's something we're looking into changing in the future!