Original Post — Direct link

The fundamental issue to MM is how internal MMR distributes over a range of ranks.

At the core of this problem is the overlap in MMR distribution, you can imagine this as a range of values over each given rank.

This chart should look something like the following:

[Iron       ] <---->
[Bronze     ]   <----->
[Silver     ]     <------>
[Gold       ]        <--------->
[Platinum   ]              <------>
[Emerald    ]                 <----->
[Diamond    ]                     <---->
[Master     ]                        <--->
[Grandmaster]                          <-->
[Challenger ]                            <->

In actuality however, the overlap looks vastly different:

[Iron       ] <------------->
[Bronze     ]   <------------------->
[Silver     ]     <------------------------>
[Gold       ]        <--------------------->
[Platinum   ]              <--------------->
[Emerald    ]                 <------------>
[Diamond    ]                     <-------->
[Master     ]                        <----->
[Grandmaster]                          <--->
[Challenger ]                            <->

Yes, there will be overlap at all ranks, but the core issue to how match making becomes busted, comes down to a few different factors:

  • First, the overlap shown in this second chart is first and foremost caused by initial placement matches seeding players into SI / GIV at best. You simply can't place higher than this even if you go 10/10. This means you can have a challenger tier player with 100% win rate sitting at a Silver rank despite winning all placements. This is the core fundamental issue with the rank system, you can have very high elo players sitting in very low ranks without those players purposefully screwing anything up. Will those players stay in those ranks? No of course not, they will climb very quickly relative to others. But by forcing players to start in Silver, that means there will be Challenger players in Silver matches at some point.
  • The next issue that leads to the chart above is elo boosting. This can happen in a number of forms, for example: duo / trio groups getting carried by a high elo friend, people buying high elo accounts or boosting services, players grinding tons of matches all season long and eventually ranking up when the high competition players have made it out of their elo, since all you need is a win streak, this becomes more and more likely with high games played.
  • Not all champions are created equally. Climbing on all tanks vrs all enchanters vrs one tricking an assassin will yield different results and has little to do with player skill. High skilled players will climb on anything, they might climb slower on bad champs vrs good champs though and this climb might have variations on what is better to climb with at different elos and most importantly, depending on where your elo lies in the current ranked spectrum. I believe it is easier to climb on assassins, because if your skill is high, you will get matched with the worst players in the match, and their carry performance will not justify taking a supporting role, but as you climb in rank this problem might actually reverse to the point where supporting roles can outperform carry roles, once things stabilize.

Now you have to think about what this causes in matches as well...

Remember, the game is trying to get even matches within a rank, this means they want both sides to have a 50% chance to win the match and they want all players to be the same rank. So what happens when you have a Gold player that should be much higher ranked and that player has a 90-100% win rate? Look at the Gold MMR spectrum above and notice that you even have Iron/Bronze MMR players that are in Gold? Well, this is exactly the player that the high win rate / high elo player will end up getting stacked on their team. Now, because that player is so strong, the likelihood of a win is still incredibly high for them, because they will still be the best player in the match by a wide margin, but their team will also be constantly feeding, overextending, and attempting the throw the game at every turn throughout the match. Eventually this will reach a breaking point where they must lose.

The worst thing about this, is it will continue to define their match making experience through all ranks until they get to Master / GM / Challenger and end up in matches with only Master / GM / Challenger players, where the problem will still be visible to them, but will be less impactful because the worst Master player the system can find is still much better than the worst Gold player the system can find. The system can easily find an Emerald player who should be in Gold to balance matches with. And this is basically where 90% of the terrible match making in game comes from.

Possible fixes?

  • Solo queue only ranked mode (I know it sucks but it is the only way to weed out most boosted players).
  • Base ranks strictly off internal elo.
  • Place players at a more appropriate starting rank.

Another fun thing to think about is how it works in chess:

[Novice (~1200)               ] <->
[Class D (1200-1400)          ]    <-->
[Class C (1400-1600)          ]        <--->
[Class B (1600-1800)          ]             <--->
[Class A (1800-2000)          ]                  <---> 
[Candidate masters (2000-2200)]                       <->
[Masters (2200-2300)          ]                          <>
[IM (2300-2500)               ]                            <>
[Grandmaster (2500-2700)      ]                              <>
[Super GM (2700+)             ]                                +

Notice that in this system, there are "ranks" but there is absolutely no overlap between any of the ranks, unless you are attempting to cross compare different elo systems like USCF vrs FIDE.

In this system if you get matches against a Master as a Class A player, you know this is going to be a difficult match, you know you are expected to lose, and you know that if you fail to perform, the result is completely on you to identify and correct mistakes. If you win this match you will gain a ton of MMR because the system now believes you are at an inaccurate rank. A Super GM would have less than a 1% chance of losing to a Master tier player, so it would only take a handful of such matches to accelerate through the ranks or M / IM. In GM they would start to meet some resistance but would still plow through quite rapidly.

Now in a team game there are always going to be matches out of your control, but you can see how elo systems with ranks are supposed to function by going back to the original system they were designed for. Now also remember that this problem is immensely complex to solve, and essentially ranks and mmr can never be 100% accurate in a team based setting due to the complexities involved here, the only thing that is really possible is to slowly improve it to it's least bad state. But without examining and understanding the core issues that arise based on the current construction, there's no way to solve them or attempt to move in a better direction. And also, at the end of the day, tradeoffs must be made. I don't expect then to make ranked solo queue a true solo queue because there are trade offs here in terms of MM queue times for solo vrs grouped players. But it is a possible solution to some of the ranked inconsistencies.

The other reason it is good to visualize this is honestly so players can understand what exactly is happening in their matches and why you run into players sitting at all elos that have honest to god no clue how to play the game. When you match for 50% wins over a rank spectrum and have all these outside influencing variables, weird stuff happens...

External link →
over 3 years ago - /u/NextdoorMMR - Direct link

These are actually really good insights into how the system works and the experiences that come out of it, and while it can't be 100% accurate without internal sources, it does a great job at highlighting our problem spaces.

Like the OP has identified, any system that starts at the bottom will result in the type of MMR overlap across Ranks displayed in the post (I actually speak a bit about this myself in my 2016 GDC talk). So it's possible to have fair matches despite a large rank spread because the MMR spread can still be small. In general, MMR is more accurate than Rank at predicting match outcomes.

We are aware of where we can make significant improvements and have several on the way. I list some of them here: https://twitter.com/joshua_menke/status/1417597287849725952?s=20

Ideally, we would give fair matches (50/50 odds) while making sure great players that are in lower ranks still move up appropriately to their correct Rank. The upcoming improvements aim to bring us closer to those goals.

over 3 years ago - /u/NextdoorMMR - Direct link

Originally posted by TheNorthernRaider

If you don't mind me asking a question, if someone has diamond level gameplay but is in goldIV how does the mmr match the game? Does the opposing team also get a diamond level player or will the system give the other team an overall average higher mmr that the diamond level players teammates?

I want to clarify I don't mean someone who insists they should be diamond but someone who is actually on track to settle in diamond rank.

In an ideal situation, the matchmaker and team balancer would only need to look at MMR to both match and team balance.

So if a Gold IV player belongs in Diamond, they will have a Diamond MMR and ideally be matchmade with Diamond MMR teammates against Diamond MMR opponents, all regardless of what their actual visible Ranks are.

If there's a mix of players with different MMRs, it will ideally organize players such that the sum of the MMR of both teams is as close as possible, since that is the best predictor of who will win the match.

over 3 years ago - /u/NextdoorMMR - Direct link

Originally posted by TheNorthernRaider

Thanks for the feedback, I appreciate your response!

Edit: Does this make it harder for them to climb? If actually playing against diamond mmr opponents wouldn't this balance them back to a 50/50 wr regardless of rank? I know that before promos were removed I skipped a couple of ranks when I did exceptionally well in lower ranks, I haven't since but I assume that's cause I'm where I belong, is this still the case?

Great question! That's exactly the type of problem designers of ranking systems need to think about. The interaction between your skill system, ranking system, and matchmaker / team balancer is critical.

In Wild Rift there are pieces of the ranking system in place to help the climb, like Ranked Fortitude and Placement to a lesser extent.

But in general, yes, when you gave good matchmaking, you need good systems in place to facilitate the climb. In Diamond and up this translates into getting higher VP gains etc.

over 3 years ago - /u/NextdoorMMR - Direct link

Originally posted by TheNorthernRaider

I appreciate your communication with us, it's really refreshing!

One last question, does mmr play a role at all in ARAM :)? I'm just having a laugh, I got matched with two grandmasters and my team was three unranked and two emeralds, we did not win that game.

Keep up the amazing work, we love your game!

It does, though we haven't focused as much on polishing it there yet. We've leaned a bit into it being a bit of a random fest, though we we do peak in now and then and it may likely inherit improvements applied to other modes by default.