EvrMoar

EvrMoar



09 Jun

Comment

Originally posted by ohbump

Hey man, super insightful post I appreciate the time you took to make it. I wasn't knocking you guys or shitting on the current system, just more of a discussion kind of thing. I was under the impression that Valorant used TS2 or another system that came from TS2. And yes, it was Menke I was referencing, he was always transparent about the state of H5 and was known for tweeting data showing why the thing that happened, happened. That's part of the reason I assumed you guys implemented TS2 with one of the updates that dropped a couple months after he was hired. We love him and miss him dearly over in Halo

I do think Valorant specifically is a fairly accurate judge of players' skills overall. Thank you for breaking down the bell curve graph, I didn't really put a lot of thought into the inflated numbers or even consider them at all in the bigger picture when I was typing all that out. I should also say that the Lebron analogy was made assuming TS2 was the primary match maker....

Read more

Hidden MMR is in every PVP game, for the most part. You will only find mobile games using "rank" as the main way to match make people. If you think about it, using rank to match-make is just making rank your "MMR".

Matchmaking rating(MMR) is just a way to rate a player's skill and put them in a match with others around the same skill. So it doesn't matter what you use, at some point something is going to act as MMR and put you in a match. The only games, that exist today, that don't have some sort of skill-based matchmaking are server browsers, mobile games, or indie titles(usually due to smaller communities).

So hidden MMR is the only thing that determines matchmaking, and this is true of Valorant. I think there is a heavy belief that MMR is somehow holding people back, or doing weird things to manipulate a player's experience in VALORANT(or any game really). But, honestly, MMR is just a ladder with complex math to determine how fast you move up and down it.

...

Read more

08 Jun

Comment

Originally posted by IBlubbi

I would love to see a rank added between Immortal and Radiant or Immortal rank to be capped to a certain number of people (coming from an EU player that has been stuck in Immo 3 300+rr for a while now). Especially coming from League of Legends where even Diamond rank ment you were in the top 1.6% the current state of Immortal feels atrocious. Psychologicaly Immortal should be Valorant's equivalent to Leagues Master rank, seeing that it is above diamond, whereas in fact Immortal in Valorant has a higher percentage of players (1.8%) than Diamond in league (1.6%). The actual equivalent to Master in League (0.18%) is the current immortal 3+ (0.2%) but it doesn't feel rewarding in any way because you will still get the exact same reward the 50000 immortal 1 and 2 players will get at the end of the Episode.

Another problem is the 450 rr gap between Immortal 3 (200rr) and Radiant (currently around 650rr). That is essentially enough for 1.5 entire new rank brackets and feels very...

Read more

Great feedback!

I think we could probably do a better job of rewarding Immortal 1-3, or exploring the leaderboard space more. No plans right now, but I have it on the list of improvements I'd like to explore at some point. (Maybe after our tournament system, which our team is STILL pushing along and working on! Have to always have a list of things we can do, so we don't run out of stuff to improve!)

Comment

Originally posted by Serito

If you're up to answering, do agent roles and map choice impact player consistency enough to cause issues with accurately guessing skill distribution? Can these be accounted for?

Overwatch implements role queue, so I've always wondered where that needle is at for Valorant.

I don't know enough of the space, but I do know your familiarity with an Agent and how many times you've played them can affect your winrate. It's so small tho, and you are in the same closed environment as everyone else, that it's not a big thing we need to worry about for matchmaking (yet). I have wanted to explore this space, there may be a world someday where we have to take into account role or Agent to get a better idea of skill but we aren't there yet.

Comment

Originally posted by JD_352

I went back and read all of the replies you did and they came together well. It can be hard for players to sometimes understand - when we see a post that does go into quite some depth; it’s almost like “ah ha!” moment where it all makes sense. But, just because it feels relatable doesn’t necessarily mean it’s fully true.

Thank you!

It's hard right, the matchmaker is a black box and you just have to take my word for it. I don't expect to make everyone believe me, or for people not to have their theories. If anything the feedback, and issues people face, all make us want to improve the system or explore new spaces! Healthy skepticism is always good.


07 Jun

Comment

Originally posted by crazyjake60

Honestly, silver, gold, or plat should have an extra rank. A little bit of extra padding between diamond and the ranks below would be better for both ranks. I think that would also let you push the distribution up if you guys decide to.

Interesting take, adding ranks is definitely something I've seen people mention. Where would you put the rank and what would you call it? :P

Comment

Originally posted by JD_352

Thanks for the reply!

no problem! Sorry it was short, I went indepth in my post in this thread.

It's hard, matchmaking will always be super complicated and mostly a black box with lots of beliefs in how it works. I would be worried if we stop having posts complaining about ranked and matchmaking, because I've never seen a game not have complaints. Hopefully future updates will help alleviate some of the issues people feel with their ranked experience.

Comment

Originally posted by JD_352

I’m assuming this enforces the importance of queuing with others if the game has designated you the golden child of the match? It also explains why people who solo queue can often feel stuck unless they are super consistent and actually do manage to get good teammates?

You often hear solo queue players (especially in the “elo hell” of the positive skewed bell curve) say they almost always team MVP but they get bad teammates. If you’re consistently MVP’ing, the algorithms have designated you as that from the start. So you need 1) to perform to that standard (no bad games for you) and 2) get some luck and some teammates that can actually carry some weight if their own.

But, you often hear the community tell them they are losing and top fragging then it’s their fault as they are not playing for or with their team. However, it could actually be the matchmaking algorithm that is pairing them with subpar players. Even in gold lobbies, I’ve experienced a random teammate...

Read more

There is no "golden child" queue. In fact, no player is held higher over another and the entire match is based on each individuals MMR. Our Match maker grabs groups of players of similar skill, then tries to make two teams for those players. It does not choose one player, says "this is supposed to be the best player" then purposefully tries to choose lower players or singles anyone out. If you get into a match it's because all the players around you are the same MMR. The only time this isn't the case is in 5 stacks which can break grouping restriction rules. Top 100 players can also be an edge case, but there is a restriction in the match maker which limits the distance of MMR players can be put in a match with.

Comment

Originally posted by EvrMoar

Super interesting take, and a great read!

I just want to be 100% clear, VALORANT does not use true skill 2 - we made our matchmaker and MMR system.(which by the way is technically two different things/systems).

I want to start out that your post seems like it's coming from a high level player, which technically is has the worst impact when it comes to matchmaking issues. I don't doubt the issues you faced, and in fact experienced the same match-making woes when the industry was trying to figure out how to do this(I was in early esports). I also want to talk about "the employee" you referenced, and what you are eluding too. Lastly, I feel like there is a "I feel this happened" rather then data or arguments to really support it. I understand you don't have access to that data, and at the end of the day feedback of "This feels bad" is super important and is the main source of player to dev feedback; I just wanted to point this out because while your thoughts on how the...

Read more

Lastly, you brought up that True Skill 2 makes matches off of trying to create a 50:50 match. This is actually true of almost all match makers, they all try to make a 50:50 match and that's the goal when implementing a match maker in your game. Also, while you can have a Lebron vs. High School team, there are actually settings to prevent this(again most match makers should have this). How it usually works is there is a skill gap that tries to be enforced, but usually the skill gap will widen the longer a player is in queue(again this is to solve players never finding a match). This isn't unique to True Skill 2, the longer you are in queue the wider rank disparity you may see in your match. So match makers actually try to prevent the thing you were experiencing, but this kind of ties back to higher rank players will experience this the most.

The other thing to think about is that players are often on their own "journey" in terms of their ranked experience. It doesn't matter ...

Read more
Comment

Super interesting take, and a great read!

I just want to be 100% clear, VALORANT does not use true skill 2 - we made our matchmaker and MMR system.(which by the way is technically two different things/systems).

I want to start out that your post seems like it's coming from a high level player, which technically is has the worst impact when it comes to matchmaking issues. I don't doubt the issues you faced, and in fact experienced the same match-making woes when the industry was trying to figure out how to do this(I was in early esports). I also want to talk about "the employee" you referenced, and what you are eluding too. Lastly, I feel like there is a "I feel this happened" rather then data or arguments to really support it. I understand you don't have access to that data, and at the end of the day feedback of "This feels bad" is super important and is the main source of player to dev feedback; I just wanted to point this out because while your thoughts on how the...

Read more

17 May

Comment

Originally posted by ajikeshi85

we suddenly reduce or remove our ability to find any players actual skill level.

that point is basically moot since the game is free to play and cmpetetive unlocked accounts can be bought for pennies

at least for gold and below... around diamond+ the actual skill level might be closer to the real one though

Even if all players above plat smurfed that's less than 15% of ranked players. We know that not all of those players are smurfing, because if they were you wouldn't be able to find matches over plat :P

We are getting better and better at reducing smurfs, as well as getting them out of low ranks. Unless you are purposefully manipulating an account, by throwing games, most smurfs get out of low ranks pretty quick due to encounter MMR(most under 5-10 games, if it's an account that has been purposefully manipulated it can take longer tho which we are looking into actively).

That being said, we know smurfing is a pain point; in fact almost every ranked design change has made sure that it does not promote smurfing and even tries to reduce it. A great example is the 5 stack restrictions removal, one of the biggest reasons we saw players smurf is because they want to play with friends so we tried to find ways to make it easier to do so. Or the account level requirement to p...

Read more

10 May

Comment

Originally posted by DoolioArt

No, it's not confusing, it's actually a pretty interesting point.

I do wonder if there are ways to offset that, though. For example, by making a very complex system that tracks a lot of stats? So, my skill level could be assessed from those and compared to the rest of the player base? With stats that aren't affected by the opponents "that much" being the more important ones than my win/loss or kda. I am just thinking aloud here. Like, "if evrmoar doesn't run often in Apex, he's probably not that good" - repeat this a lot for various stats.

Lastly, I'd like to say I very much enjoyed this exchange, thank you for that - and if you don't dislike more walls of text, both making and reading them, you're welcome to continue it, even though I might have overloaded it a bit. But, I do find it interesting, especially since I don't have much insight into the industry today - my insights are left in 2010's, I am in a completely different ecosystem now work-wise.

Probably the best matchmaker in the industry, true skill 2, actually looks at players stats using machine learning. It can do fascinating things like understanding a players rank in like 1-2 games - it can look at how that player is performing against an opponent and because of all the machine learning stats it has predicted where that opponent should be(aka this player is performing like a diamond player would perform against a bronze player).

I think it would be interesting to see if you can feed stats that are not PvP stats, but like you said more individual movement/gameplay stats with no impact on other players. Also no problem! I probably wall of text way too often, and I love talking about these things. It's nice to work at a studio that enables me to actually have these discussions(some places I've worked don't like these kinds of interactions).

Comment

Originally posted by jonathancyu

I’m trying to get into the game dev industry, where did you start?

My advice is to make mods, or download unreal and start doing some tutorials! Most designers need engine experience and have an idea of how to implement game features. If you don't/can't get a degree I would go the QA route and try to get into an entry level position(be warned QA can be a pretty rough experience and I'm not a huge fan of how they are treated in the industry as a whole).

I would then look at universities that offer Computer Science or Game Dev degrees. Personally, I'm not a fan of Full Sail, but they've gotten better since I entered the industry(I actually really like digipen; I would probably avoid smaller no-name private colleges). Honestly if you are driven, actually apply yourself to learn a game engine, you can make it; it just takes a lot of work, possibly some failures, but that's all part of the journey and learning!

If you are more art driven obviously you can look for 3D art style degrees, but I'm super unfamiliar with that space.

Comment

AWWWW YEA LOOK AT THAT MATCH FAIRNESS BABY LETS GOOOOOOOOOO

It's pointed out in this thread but 5 stacks play against 5 stacks and it will always try to find the fairest match it can for these groups. Because we allow all ranks to queue together in 5 stacks that means your matchmaking can get kinda wild in terms of ranked spread in these matches.

(I have to say, before I get the comments that people have issues with the matchmaker, we are always trying to improve it and get better. We really want people to feel like they can play ranked with their friends, and 5 stacks having no ranked restrictions is such an awesome space to do it. Our data shows our 5 stack matchmaking is extremely accurate, and that's awesome! I love to see people queueing together at different ranks, playing with their friends; one of the biggest reasons people smurf is just to play with friends. Again this is not an excuse or invalidating anyone's feedback/issues with matchmaking. I love this c...

Read more
Comment

Originally posted by Alky314

The Elo system used behind the ranks is the reasoning. Basically everyone has a elo score (linked within Valorant to a rank). If you have the elo of 2 players (in this case 2 teams), you can compute the probability of victory based on the difference of Elo. So immo vs Gold have the same probability of victory that Gold vs iron (very high of course)

The community learning about MMR makes me so proud /wipesawaytear

Comment

Originally posted by muthgh

I've a question, I'm assuming you are probably familiar with things link CCRL, if for example you take an engine rated 2800 on the list and a human with 2800 FIDE rating (assuming they both use the same Elo model for the sake of simplicity, haven't checked if they do), it'd still be wrong to claim that they are of the same rating, since the Elo is relative to the player pool they faced, my question is, is it possible that the hidden mmr is creating something similar, different pools of players who are being frequently matched within such pools

I'm d2 on my main, different ranks on different alts, on which I play with different friends, which you previously explained is probably the reason, so far so good, but for a while my internet has been horrid, I made an alt, that's in s3 rn, both my brother & I play valorant as our first fps, but I started ep1 act 1, he started much later, the difference between our aim and game sense is night and day, he's almost s3 now, and when ...

Read more

In theory you could argue this, but unfortunately, there aren't really separate pools. I mean silver is big, I think it's like 20+% of the player base, so even being one rank up or down in silver can create a vastly different experience to the skill levels of those your are playing against.

I think the other thing to think about is that MMR is just a ladder, and you are just playing against players that are close to you on that ladder. So there aren't really "pools of players" just players that are around you on the ladder. The only thing you need to do to climb is to beat players and climb up the ladder and push them down. We just have some fancy math to move you faster up and down the ladder if you clearly are performing far off from your current position.

There is an inherent flaw in matchmakers when they are applied to team games; this is why MMR systems when applied to teams actually bring your rank to the average of the team and not your individual MMR. This i...

Read more
Comment

Originally posted by EvrMoar

I just wonder if it would be able to be kept a secret LOL. I think people give too much credit to a lot of companies who's backgrounds or degrees are not at all based in psychology. I mean, I came from one of the other biggest games in the world before Riot and I didn't even really have access to stats to make design decisions(and this is for simple things like weapon balance). Lots of game studios are not as complex or organized as some players realize, but games can often get away with being an extremely creative process and learning from what games are finding success.

So my first point is keeping it secret would be near impossible. It may not seem like it, but every day tons of people on the dev team are raising issues with things they don't like about the game. There are plenty of people, especially in the game industry, who feel burned by their employers or their voices are not being heard. With how many people work at Riot, or any of the major game studios, all it wo...

Read more

Oh and in terms of match making manipulation being in-effective, I think my point about that being a match making paradox is the best example of why you can't manipulate matchmaking in that way.

In order to be able to "manipluate" match making we need to know a players actual skill. I need to say "You are better or worse then X/Y/Z player" so that when I want to manipulate your match or experience I know what skill level you are to do so. So first I have to find your actual skill, then I can manipulate it. Well the problem is that once I find out your skill, it's going to change; and not only that I need to do this for every single player. The moment we start manipulating your matches, we suddenly reduce or remove our ability to find any players actual skill level. This creates a paradox where we can't manipulate your games because by manipulating games we can't actually know any players actual skill - thus making it impossible to manipulate games. Hopefully that makes sens...

Read more
Comment

Originally posted by DoolioArt

I don't see a reason why we would keep something secret if it was beneficial to the game, and the players. It would be very risky, and make no sense, to hide something like this from the players. Retention comes from having a fun game, increasing player satisfaction and having an enjoyable gameplay experience; this is how you keep and grow your gaming community. We now live in a connected age where we aren't dumb, people see through marketing, corporate tricks, etc. This is my personal belief, but often mechanics like gatcha games or gambling games are not about retention but often targeting big spenders to keep spending. Those games are less about retaining a ton of players and only retaining the big spenders(which you could argue are also addicted to the game). Our goal is to have an enjoyable game that players want to play, especially because we are a PvP game and need a healthy community to keep queues rolling! This leads me to my next point.

...

Read more

I just wonder if it would be able to be kept a secret LOL. I think people give too much credit to a lot of companies who's backgrounds or degrees are not at all based in psychology. I mean, I came from one of the other biggest games in the world before Riot and I didn't even really have access to stats to make design decisions(and this is for simple things like weapon balance). Lots of game studios are not as complex or organized as some players realize, but games can often get away with being an extremely creative process and learning from what games are finding success.

So my first point is keeping it secret would be near impossible. It may not seem like it, but every day tons of people on the dev team are raising issues with things they don't like about the game. There are plenty of people, especially in the game industry, who feel burned by their employers or their voices are not being heard. With how many people work at Riot, or any of the major game studios, all it wo...

Read more
Comment

I talk about losers queue here!

https://www.reddit.com/r/VALORANT/comments/tvb772/this_problem_is_now_getting_soo_common_dude/i393gp7/

It's crazy going on a 15-game lose streak, sorry you ran into that. I can't ever prove losers queue does not exist, but I think my biggest argument(which I cover in the link above), is that lose streaks would not be beneficial to keep people in the game. If anything, the correct answer would be to catch players that were on a lose streak and put them into an easy match.(which...

Read more

07 May

Comment

Originally posted by Lopsided-Perception2

Thank you for taking the time to come up and write down this extensive answer.

You are right, on a significant part of it, I'll have to take your word for it.

However, after reading your answer a couple of things occured to me, but there's this specific one I want to mention.

Whenever you implement a new feature, remove or adjust a current one, you have to be tracking the results of those actions in one way or another. I'm not sure if it's some parameters/metrics that you automatically see without needing to reach out to players directly (eg. you make a change to a gun, and its buy rate drops to 0, pretty sure that's a sign of an unsuccessful change from the perspective of having fun). or if you run surveys for a sample of your playerbase, but you need some type of data to make those decisions. I'm certain it's not blind implementation of random features that game designers have dreamt up the previous night.

What I'm trying to say...

Read more

It's good to point out, to let you know what we do when we make changes and how we measure them we definitely use more than just data.

During all stages of design we can leverage data, social media feedback, player surveys, and player labs! Sometimes it's just a survey asking general health of features, other times we have an idea and what to see how players would feel about it. The early stages often are identifying, or confirming, issues that we may want to fix. Then we discuss the results, figure out next steps, and ultimately decide if we implement.

Then after it passes "Implementation" we play the changes and everyone on the team provides feedback. We can follow up with more surveys, maybe a player lab and bring players in to play the feature, etc.

Then after launch we do all of those things again to make sure that players like the change and it improved things. So I focused on data in the above answer but we definitely make sure that player perception...

Read more
Comment

Originally posted by Lopsided-Perception2

I'll quote a part of your answer you've linked to:

Retention comes from having a fun game, increasing player satisfaction and having an enjoyable gameplay experience; this is how you keep and grow your gaming community.

Can you share what is the operational definition of the concept FUN within Riot Games? And is player retention based on creating addictive mechanisms that are based on causing a feeling of frustration and expecting players to keep coming back to the game just to end it on a good note?

I see comments often that are along the lines of "Riot makes this mechanic to be addictive to make money, drive engagement, etc."

While I'm speaking as a Rioter, I also just want to share my overall game industry experiences. If I were to be honest there is no studio, or designer, that I've worked with that's ever said "I'm making this because it's addictive" because it doesn't matter if you try and leverage some psychology on addiction if you don't have players being remotely interested in your game or playing it.

I would also argue 99% of designers don't know what mechanics are addictive, because most designers don't research psychology or have it covered in school. The design process usually comes from two things:

  1. You design to solve a problem facing players.
  2. You design because of a blue sky/gameplay experience idea you have.

Your original goals, or designs, often aren't rooted in psychology. Again, this isn't always the case...

Read more