Introduction:

Edit: There seems to be an issue where some users cannot see the text. I am removing the royaleapi link to see if this resolves anything

TL;DR at the bottom

On this subreddit, there are countless posts about overleveled cards and ladder matchmaking. In this post, I will detail how I built and used a computational model of the ladder to determine which model parameters minimized card level differences

The guiding principles that I looked for in a ladder model were this:

• The ladder should minimize card level differences for as many players as possible
• The players should not be clumped around a certain trophy level
• The ladder should work on a long term
• Level 14 players with fully maxed decks (modeled as 112 card levels) should reach the top
• The top of the ladder should be between 8000 and 8500

After testing 28 different models of ladder, the design that most accurately fit the aforementioned principles was a system where card levels (notably not king level) was capped by league and inflation remains unchanged.

How this investigation was done:

Simplifying Assumptions

Since this is a computational model, there had to be some assumptions. For the testing different ladders phase they were:

• There is no 2v2. All players play ladder exclusively
• Players are *always* matched with someone that is within 40 trophies of them, regardless on position on ladder
• All players hit cancel within 5s when king tower matchmaking active
• A season is about 4 million battles for 25000 players
• There is an even distribution of players at king tower 8,9,10,11,12,13 amd 14.
• The card level on a player of each king tower level is evenly distributed*

When 2 players face each other, the chance of the overleveled player winning can be approximated (R^2 for the regression was 0.9933) by the equation 0.0186*lvlDiff + 0.521. With equal levels, I made this a 50/50 coin toss for testing different models. The card level approximation is from 90k real matches.

Data Collection Process

First I created a set of "baseline data" where I ran 25000 players starting at 5000 through a simulation of 10 million battles. I used the inflation that is currently in the game (80% loss from 5000-5300, 90% loss from 5600-6000), turned king tower matchmaking off and reset it to create the starting point for all of the tests conducted. This data is stored in the file 'baselineData.csv'

This is 2 of the graphs I generated from the data:

https://imgur.com/a/zmBKIgZ

Processing img p4ylvuc4j5a81...

Processing img 22ecj8bcj5a81...

Now with the baseline data, I could do a season reset (with reset numbers from royaleapi)

The first "test" I did was test the current ladder. I used +-1 king tower matchmaking, the same inflation from the current ladder and king tower matchmaking ending at 6000 trophies. Any model that is "better" than the current ladder needs to have lower card level difference and more separation between the players when I run it through the same condition, which is 4 million battles.

King level distribution by Trophies currently

Processing img d6xb9nfxl5a81...

Looking at the upper graph, its clear that the current ladder system with king tower mm and inflation running to 6000 trophies brings players to 6k very easily, and many get stuck there. This results in a spike of card level differences and general "unfairness" as a player reaches the 6k mark.

To try to fix the ladder, the 6k spike needs to be lowered. To do this, I tested these conditions: (All the data is in one of the csv files here)

• Trophy gates at 5000, 5300, 5600 with no inflation
• Trophy gates at 5000, 5600, 6300 with no inflation
• Matches based of card levels with max difference of 8
• Levels capped by league, starting at lvl 8 at 5k (This worked the best)
• Levels are capped by league, starting at level 11 at 5k. Normal Inflation
• Levels are capped by a player's king tower
• Levels capped by league starting @ level 8, inflation by card level difference
• Opponents have an equal king tower, no +- 1
• Constant 90% trophy loss from 5-6k, no King tower matchmaking (KT mm)
• Constant 90% trophy loss from 5k-6k, king tower mm to 5600
• Constant 90% trophy loss from 5-6k, king tower mm to 6600
• 90% trophy loss from 5000-5300 and 5600-6000, 100% trophy loss elsewhere
  • Tested with no KT mm, KT mm cutoff at 5600, 6000, 6600
• Inflation expanded to up to 7k
  • Done in 3 ways, with and without KT mm to 6k
    • Constant 90% loss from 5k-7k, (Not tested with KTMM)
    • 5000-5600: 70% loss, 5600-6300: 80% loss, 6300-7k: 90% loss
    • 5000-5600: 80% loss, 5600-6300: 85% loss, 6300-7k: 90% loss
• Inflation by card level difference
• Inflation by Card and King level difference (no max trophy cap for inflation)
  • This was tested across 5 seasons since it too worked remarkably well
• Loss percent based off a sine function
• Ladder with no rules, just trophy mm
• Trophy caps every half league (lvl 8 below 5k, lvl 9 at 5150, lvl 10 at 5300 etc)

Notes on Data Collection

In many of these tests, I address inflation, or losing less than you would normally lose. Loss percent means the percent of trophies you lose from 30. (90% loss percent means +30, -27). From the december 2020 change that was reverted, its clear that inflation is a necessary evil. The puzzle is managing it so that it doesn't harm players.

Also note that many of these ideas did not end up very well and had did worse in terms of level disparity, low level players getting to the top or a large clumping of players.

I did not combine king tower mm and card level matchmaking. As u/notkasa said, "Each time you add a new rule, it takes more and more time to the algorithm." (from this post). King level matchmaking required 7 separate mm queues** and card level matchmaking required 53 queues. To combine them in my model, I would need 371 separate queues, which takes significant memory and inhibits performance. The amount of searching to find a match doubled for KT mm and went up even further for card level matchmaking, and combining them makes it even slower.

Technical side of this project

None of this segment is necessary for understanding the idea of fixing the ladder. This segment is for those interested in understanding the back end of what I did here and how it can be used for future projects.

The code for this project is broken up into 2 files: A player class and a simulation file.

The player class includes the player object with attributes ID, trophies, wins, losses,king tower, card level, total level difference, skill and party percent. The class methods allow me to update the attributes after a trophy reset or after a game. The number won and lost is from this post. The non class methods are for tweaking inflation and other parameters, along with creating player objects with specified card levels for their decks.

The simulation file contains 4 simulations plus a final simulation, which I will discuss later on.

Each simulation goes through these steps.

1. Initializes the queue or dictionary of queues.
2. Picks a random player.
3. Looks through the queue(s) per the matchmaking rules.
4. If an opponent is found, play a match and remove the opponent from the queue. Repeat back to step 2 until the correct number of matches have been played
5. If an opponent is not found, then add it to the appropriate queue in the correct spot. (Since mm is always based on trophies, queues are sorted by trophies to take advantage of binary search). Repeat back to step 2.
6. Sort the player array by trophies and return it.

Further Optimizations: I used a numpy array for my matchmaking queue and appended it when adding a new player. Appending these arrays is slow. The fix would be to use a binary search tree to retain logarithmic opponent finding time (per queue) and make appending to the queue faster.

Analysis:

For each ladder change I tested, I created 6 graphs:

• A bar graph showing Average level difference per match (a measure of ladder fairness) vs king tower
• A bar graph of avg lvl difference/match vs card level.
• A histogram showing the number of players of each king tower over their trophies
• A histogram showing the number of players of each card level (range of 4 levels) over their trophies
• A scatterplot of card levels vs trophies
• A scatterplot of level difference/match vs trophies.

All the graphs are in this document. I will not detail of them since some are quite similar to each other and many are worse than the current ladder.

The first change I tested was merely removing the KT mm rule and not touching inflation.

Level Difference/match vs trophies

The players are nicely spread out across the trophy ranges

From these two graphs alone, the distribution of players is nice, but at 6k, the card level differences are about 3x as bad as before, meaning this is definitely not the way to fix the ladder.

Another popular suggestion on this sub is card level based matchmaking. Here is what happened on the moderate 4 million battle test where the max card level difference is 4.

Trophy distribution shown with card levels.

A significant number of players get stuck far lower than they should be

Although the distribution of players is quite good, there are many players that are stuck far too low because of their high card levels. In other words, players are penalized for upgrading cards.

It has also been proposed that players card levels are capped by their king tower.

Card level difference vs trophies

The spike around 6k trophies is pretty close to as big as the ladder is now, and it doesn't curb off as sharply past 6000 trophies. This change seems like it would help, but it does very little and takes away the usefulness of upgrades for players. It also means that in scenarios where levels were capped, having a 1 king tower disadvantage is an 8 level difference, which puts a player at a 2/3 - 1/3 chance of winning against them. That is hugely unfair.

Lastly, there has been some discussion, especially on r/ClashStats about capping levels by league. I didn't use the exact level caps from their discussion, I tried capping at lvl 8 until Challenger 2, lvl 9 until challenger 3, lvl 10 until Master 1 and so on.

Level difference/match vs Trophies

King tower distribution vs Trophies

There is still the spike at 6k, but it is lowered from the current ladder. This would work in practice moderately well. There would be no level 14 cards at 5k. However, the major drawback is that players may lose the incentive to level up cards since there is no reason to get beyond level 8 if you can get to the highest arena. This also means the only players that can use lvl 14 cards are 7k players and that makes up a very small percent of players. (Top 10k was ~7070 last season).

This model also allows for a better distribution of players on the ladder. Before, there was huge number of players stuck at ~5800. This model alleviates the log jam by letting more players continue on. There are more players near 6500 but it still dwindles as we approack 7k.

Larger Scale Tests

Next, lets look at models that will not work sustainably. For these few tests, I ran them on a much larger scale of 100000 players. I also added 2 parameters: Skill and Party Percent. The reason the other models weren't tested on this is because I used 4x the players and ran it through 5 seasons at the minimum, which takes 80 million battles

Skill represents the skill of a player. When 2 players have : equal levels, then I use this eqn to find out who wins a match: Chance of more skilled player winning = 0.5 * skill difference/2.5

Party percent represents the chance that a player plays party mode instead of ladder. If a player's party pct is 0.3, there is a 30% chance that the player isn't added to the queue when picked randomly.

Strict card level matchmaking with a difference of 8 levels. 1v1 showdown uses card level matchmaking, so if SC wanted to implement this quickly, they would use the same mm algorithm as party mode. In my matches, I almost always match with someone between 8 card levels of me, so I decided to test it with an 8 level difference.

Trophy distribution with card levels shown by color.

I capped card level matchmaking at 6k here. I was quite surprised to not see the spike like we see in the current ladder. As expected, there are player with a total card level of 60 reaching 6k, which is far lower than should be required to reach 6k.

Level difference vs Trophies

The matches are more unfair at the 5000 trophy mark, but are closer around 6k.

Histogram of trophies with King Tower illustrated

This is what I see as the drawback to this model . There are a significant about of level 11 accounts that are passing 6500 trophies, which isn't not necessarily a good thing. There are also level 13 accounts that are stuck below 6k, which isn't necessarily a good thing.

The next one was a suggestion I heard from CWA in march 2021 and ocassionally on the sub: King tower matchmaking but restrict it to equal king towers.

Trophy distribution with equal king tower opponents

There are low level players at the top of the leaderboards. The #1 player is at 7877 trophies, which is below the normal top of the leaderboard number (Typically ~8200). The player is a level 11 with a skill attribute 3.49 standard deviations above average. A skilled level 11 should not be able to reach the top of the leaderboard.

Conclusion

The most equitable and fair ladder is the one with level caps by league:

League	Maximum Card Level
Challenger 1	64
Challenger 2	72
Challenger 3	80
Master 1	88
Master 2	96
Master 3	104
Champion and Above	112

For this model, I ran it through 13 seasons. What I found was remarkable: The card level difference was low across most trophy ranges with a few outliers.

Level Difference/match vs Trophies

Another thng that this model is meant to fix is that players who are highly skilled are penalized with massive card level disadvantages.

Level Difference/Match vs Skill

This model eliminates much of it. There are some level 8's that experience some larger level differences, but these are significantly in the minority. The level difference does go up as players are more skilled, but a difference of 4 card levels isn't the end of the world, especially since one card can't be overlevelled by 4 levels.

We can also look at the skill level at each trophy range.

Skill Level vs Trophies

The skill level goes up as the torphies increase, as expected. There are a couple outliers of high skill and low trophies and it ends up being due to levels. As players go further in trophies, the matches get more fair and the spike at 6k goes away, which eliminates the log jam where players get stuck.

Of all the models I tried, this one was the best, although capping lvls by league starting at level 11 at Challenger 1 also works, but not quite as well. If this idea was to be implemented by SC, I suspect that they would have it capped at level 11 so that more players can use their level 14 cards.

Testing these models with KT mm revealed a couple interesting things:

• In a lot of cases, the average card level difference did decrease
• When the KT mm is cut out (6000 currently), there is a spike in players around the cutoff
• Whenever KT is involved in MM, there are always players with high king towers left behind.
• Players with low king towers could climb higher than players with higher kings would with equal card levels.

Because of the spike in player count and players with low kings pushing up higher, I decided to not have king tower mm in my finalized model. It would make queue times faster and incentivize players to level up their king tower. Since levels are capped, the avg card level difference per match is already low, and is mostly attributed to king tower differences, which are necessary to keep level 14 players at the top of the leaderboards.

Acknowledgements:

Thank you u/Milo-the-great for some little things in the write up of this investigation. Thanks to OJ for a couple ladder fixing ideas too.

TL;DR

• Here are the graphs for all the different conditions I tested.
• Here is the github repository with the source code and all the data in CSVs
• Trophy based level caps are the best way to ensure fair matchmaking throughout the leagues.
  • Card levels are capped at 8 for Challenger 1, Lvl 9 for Challenger 2 and lvl 10 for challenger 3.
  • 11 for Master 1, 12 for Master 2, 13 for Master 3
  • Anything beyond 7k is uncapped with no special matchmaking rules.
• No, I was not paid by SuperCell for this post

*The exact numbers are listed in the Player class in the method 'createPlayer'

**These are not FIFO queues. They represent the mm pool when you join the queue for a match

External link →