Original Post — Direct link
over 2 years ago - /u/phroxz0n - Direct link

Originally posted by UNOvven

I kinda wish they asked Riot if the balance team had any statisticians/data scientists and how they informed their balance decision, because lately it seems like there isnt even a single one.

We have a insights analyst who builds out all our reports (we have many reports that tell us a lot of different things; items, game pacing, winrate reports, an automated version of the champion balance framework, role power, champion learning curves, etc.)

Data scientists work on creating the pipelines to get the data to the analysts so that they can then write queries to get the data and put it in a usable format for designers, game analysts and producers, etc. So to answer your question, we do have both analysts and data scientists. Researchers like Blau are a different form of insights. They collect more qualitative assessments from players through surveys and player labs, rather than analysts or data scientists who are more quantitative.

over 2 years ago - /u/phroxz0n - Direct link

Originally posted by UNOvven

Interesting. Do any of them work directly on the balance team, or are they more focused on obtaining the data and making it accessible?

We have an analyst that works on the summoner's rift team 100%. But what reddit knows as the balance team is actually only 1 of the 3 pods of the summoner's rift team. That analyst services the 3 different pods.

You wouldn't really assign a data scientist to service a particular team, that's too low scope for the type of stuff that they're trying to do. Their job is to make different types of data available for the analysts to have access to all across LoL, whether that's for Skins, Champions, Balance, etc.

over 2 years ago - /u/phroxz0n - Direct link

Originally posted by UNOvven

Gotcha. Do any of the analysts have a background in statistics, to ensure that the team doesn't end up relying on data that is misleading or not terribly indicative of what youre looking for?

Every analyst has a background in statistics, usually a math degree or something similar. They are very proactive about making sure anybody looking at the data understands all the biases present.

over 2 years ago - /u/phroxz0n - Direct link

Originally posted by Capek95

I have one big concern though, would you happen to know if any of the analysts or balance team play the game? I can't seem to find any info on that.

I'll take the bait. Yes we play the game.

We have 7 designers on Summoners Rift Team split amongst the 3 pods. Of those, 1 is Master, 4 are Diamond, 1 is Plat and 1 is Silver.

We also have 2 QA analysts on the team, who are Master and GM respectively.

In addition to that, we have a separate Game Analysis Team who are typically minimum D3 and 7 of them are Masters+. 3 of them hover around GM/Chall.

We generally try to have as many different opinions from each different skill band as possible. It would not be a good thing to have every game designer be Challenger for example, because they would have such a myopic view of the player perspective that it would be actively damaging.

over 2 years ago - /u/phroxz0n - Direct link

Originally posted by UNOvven

I see. In that case, what has motived you to completely exclude mastey curves from the balancing process, and what is your perspective of the false positives that have been popping up lately (most recently Annie)?

Simply put, it's a complicated problem.

It is certainly a limitation of the current model and something we discuss a lot internally. One of the values of having a transparent balancing process is that the community can understand why we are making certain balance decisions.

Including mastery curves is potentially valid, though while it has its benefits, it also has drawbacks and we'd need to weigh the pros and cons of including that information in the balance framework.

We'd also need to figure out the correct way to implement this information.

Example Considerations:

  • Do we discard only the first game that a player plays?
  • The first 3, 5, 10?
  • Do we discard 10 games for an easy champion, but 30 games for a very hard champion?
  • How do you weigh mastery for a person that played Lee Sin a lot 4 years ago, but has only played 5 games recently?
  • Should a game played by a main matter equally to a person who has played them 5 times? If not, what weighting do we apply?
  • What defines whether a player is a main?
  • etc.
over 2 years ago - /u/phroxz0n - Direct link

Originally posted by Spideraxe30

Hey Phroxzon would those pods be the patch to patch balance pod, the mid scoped pod (that did the Sona and Tahm changes), and the preseason pod.

Hmmm, I can't really say, but I will say that one of the pods is the balance pod.