Original Post — Direct link

Vod | Meet the Ninja Team PAG


Player Advocacy Groups (PAG)


Creation


The Idea Behind It

  • Create a system where someone experienced can explain nuanced issues in a way where developers can make changes.
    • This would be done in a way that will be trusted so they can have clear expectations of the outcome.
    • The community can get an insight into the conversations that are happening.
  • Not meant to replace getting general community feedback, surveys, or newsposts.

Designing the 1st PAG - The RS Guy

  • The starting point was scattered and everyone had a different idea but it came together as it progressed.
  • Needed to figure out:
    • How to collect information?
    • How will we display it?
    • How are we going to present it?
  • First thing was to form a team then have calls and meetings with them and Jagex.

The Team


Finding a Leader

  • This was a paid position since it would require a lot of work.
  • This person needed to be:
    • A good leader, a player expert, and could handle the scrutiny of the wider community.

Selecting a Leader (The RS Guy) - Jagex

  • We needed someone who was conscientious about the selection process.
    • If we see someone is forming a biased team we would scrap it and try again.
    • All nominees were in a position to claim expertise and advocate for the playerbase.
  • Great organization skills needed to be an interfaces between the players and Jagex.
    • Insight into the development process and strong leadership skills.
  • The RS Guy was also partly chosen partly due to the great discussions at and following Runefest.

Forming the team - The RS Guy

  • Important Criteria:
    • Knowledgeable players who were representative of a community.
    • Someone who could understand a topic clearly and get involved and fight for the things they cared about.
    • Knowledgeable about 1 subject but willing to learn more about the game.
  • Process:
    • Reached out to a number of discord communities for nominations.
    • Interviewed and spoke with 20-30 people and read applications of 50-60 more just in 1 week.

Meetings

Themes: PvM Bossing, PvM Slayer, Clue Scroll, Skilling, Ironman, Other

  • 1 Initial Meeting, 4 Team Meetings, 1 Retrospective Meeting.
  • Meetings occurred weekly, where different categories were tackled on different days.
  • 10+ hours a week just on calls going through every item we wanted to bring up for all aspects of the game.
    • Everyone participated even if the discussion wasn't about their expertise.

Objectives


Objective

  • PAG effectiveness:
    • Are we on the right track?
    • How much investment is required on both sides?
    • What sort of results do we get?
  • Find all the little changes that are so spread-out with the level of knowledge required to make a strong change.

Determining to do a Ninja PAG

  • Due to the existence of the Dojo we could see how effective the PAG was compared to general community feedback:

    • Does it provided a different value, perspective, deeper information or ideas on how things could be different?
    • Is the amount of time invested to get to those conclusions less or more?
  • By allowing a broad PAG focus it allows us to get a deeper focus on future ones without players wanting to bring up additional issues alongside the primary focus.


Results


Process wise that was the rundown of the project. It was different from but it was a good challenge. It was a lot of work but I'm really proud of what we got done.

Results

  • We were hitting on all those metrics/goals
    • Doing things the Dojo wasn't doing but was still in agreement with it.
  • Obtained 2 years worth of work for the Ninja team as well as passing some off the other teams.
  • Despite this PAG being over, we have documentation we can reference back to at any point.

Future Updates

  • PAG strike coming up in July and another later in the year.
  • They may also be sprinkled into other strikes depending on their themes just like we do with the Dojo.

Determining to do Future PAGs

  • Future PAGs will require an NDA which may result in some players not being able to participate.
  • PAGs are done at the developer's request and they must have a clear goal and timetable.
    • There will likely be more as the model grows.
  • The charter process will be made more visible and will be live updated for players to refer to.
    • It will contain a working agreement with the PAG and their criteria.
  • Make plans and procedures in advance so those involved know what to expect going into it.

Current Plans

  • There are no current plans of a specific PAG however there are discussions.
    • We will notify the community when a future PAG is planned.
  • Current plans include: Making a logo, making sure the presence is easier to find.

Other


Sharing with the Community

  • In the future there will always be a charter and a retrospective provided to the community.
  • We aren't sharing the actual submission list as there can be a feedback loop that where it can create biases on player needs.
    • We also want to leave people open to surprises.
  • Ninja PAG Meeting minutes will be pushed out in a retroactive post around the July strike.
    • For future PAGs that may show results much later it would be posted immediately.

Updates in General

  • Updates that came from PAG will NOT be marked as such.
    • The goal of PAG is to bring up problems not implement them. They don't do development/balancing.
    • The PAG team is also unaware of any of what they brought up would or when it would be implemented.
  • There are checks put in place to make sure no one takes advantage of the economy based on internal knowledge.
External link →
over 4 years ago - /u/JagexHooli - Direct link

Originally posted by Kadem2

So the player advocacy group is under an NDA and can't discuss anything they met about, the submission list is not being released, and all updates related to the PAG won't be marked as such. So basically there is no way for the community to have any idea how the PAG went or what it accomplished, which seems directly contradictory to the initial claims that this would be as transparent as possible to the wider community.

Don't get me wrong, I hope it was helpful for the devs and for the health of the game, but it's weird that they literally will not discuss any details of the project.

Thanks for the thoughts here - definitely something for us to go away and think about.

There is an upcoming 'PAG' branded Ninja Strike which will give you a clear look at some PAG-inspired changes. PAG Team members are also permitted to discuss anything they were involved in after it releases too, so extra details on the process around those items will trickle out beyond just what we talk about ourselves.

To provide some context, the current approach is in part to be respectful of the process of development. As Kalaya mentioned, we have over 2 years worth of Ninja Fixes via a combination of the Dojo and our first PAG alone.

The Ninja team works super hard to release as much as they do on a 2 week cadence, but game development is a constantly shifting beast - especially for a living game like Runescape. Something weeks away today may be much further down that 2 year Ninja hit list next week. We also think Ninja being able to be nimble and reactive is a big strength of their setup (personal note: so is their passion to do as much as humanly possible every update, it's kind of insane!).

The other factor we have to consider is expectations. The more we expose specifics of what came out of the PAG, the more expectations are built up that these things will happen at some point.

In the same way we can't always say we're going to change something based on online feedback until it's very close to or actually released - just in case anything goes wrong or the scope of the issue is actually much bigger than expected - the same applies here.

With two years worth of suggestions directly from our players to balance against, the desire is there to address all the PAG feedback in some way, but the reality is that the nimble nature of Ninja means we can't be sure when or if every single thing raised will be acted on. It's a valuable part of a bigger picture, as is the Dojo and as is the conversation we read online every day.

PAGs are new and we'll always be looking at how we can improve our communication around it - please keep suggestions like this coming. Hopefully this post just offers a little insight into why things are the way they are today.

over 4 years ago - /u/JagexHooli - Direct link

Originally posted by etueryre

So instead of communicating with their playerbase -all of it- went behind closed doors to communicatie with a select group of players, meaning whoever has the loudest voice? No disrespect to them, but there are a lot of players out there and Jagex will just pick out big streamer man as leader because he is easily visible to them? Then have no transparency between what goes on with PAG as a whole?

There are check put in place to make sure no one takes advantage of the economy based on internal knowledge.

lol good joke

PAG is very much part of a bigger picture rather than something that dictates direction or overrides the entire community. The intention is to add even more depth to the voice of our players rather than altering or detracting from it.

PAG provides the chance for the development team to have immediate conversations in a way that's difficult to have effectively at scale - and do so directly inside the development environment with a group of eloquent players that represent relevant player groups.

(On that latter note, TheRSGuy talked a lot on stream about the lengths he went to in order to recruit passionate representatives for as many types of players as he could).

Due to the process being under NDA, it also means the developers can focus on getting the most out of the PAG without having to worry about the reality of communicating in a public environment - ie. something being misconstrued or accidentally setting expectations. In the future, having the NDA process in place may also mean we can use PAGS to bring a level of player feedback into the development process earlier than we maybe could otherwise. Lots of potential here.

As I mentioned in my other post, this is very much balanced against all your amazing Dojo requests, what we see you talking about online every day and internal feedback from the team. This is just an added layer to all of that.

over 4 years ago - /u/JagexHooli - Direct link

Originally posted by ShadowBall1215

When's the roadmap coming? We're being left in the dark for half a year out here and it's not fun in the slightest.

Right now, the team is primarily working on the next quest, with Ninja also working really hard on their beefy player-inspired Strikes every 2 weeks.

Unveiling our new quest is the next focal point of what we'll be talking about - and you won't have long to wait to hear about it (read: SOON!).

over 4 years ago - /u/JagexHooli - Direct link

Originally posted by ImRubic

My biggest concern with PAG is controversial topics.

The point of PAG is to bring notice of various problems or provide insight on niche topics. This is fine for most things but if something is controversial or has split opinions, there are a number of different takes/suggested approaches. I do not trust players to be able to provide insight on all viewpoints equally, and keeping that discussion behind close doors will lead to some of those biases being implemented into the game without people being able to have a say.

For example, Group DG in Ironman is one of those topics that is split among the community. Many want it but there's also a vocal bunch who do not. Internally, Jagex may feel confident they can filter those out, but as a player I'm not confident in Jagex to do that. I would rather that discussion be started/shared externally before it is even considered to be implemented in game.

Thanks for your thoughts Rubic.

With any element of feedback, designers will always want the broadest spectrum of perspectives so they can use their design expertise to solve or embrace as many of them as possible (especially given the many different ways people play Runescape!).

From my experience so far, many JMods here (especially Ninja) are really plugged in to the community already on top of being very active players themselves. Even when you don't see a post, I can guarantee you that multiple JMods have read it as well as the CM team. There's a lot more internal understanding of the nuances of our community throughout the team than you might expect.

What this means is, when we approach a PAG, the expertise is there to be cognitive of these divisions in our community - if not already with the JMod Developers then through us as a Community Team.

I touched on this a little in my other comment, but PAG feedback is not gospel, nor does it dictate what we should or shouldn't do. The purpose of PAG is to help us to bring an added layer of player feedback in a format that we couldn't do at scale. It forms part of that bigger picture of community voice rather than defining it entirely.

I understand if you don't necessarily trust the process right now, but the end result should hopefully speak for itself - as the saying goes, the proof is in the pudding.

over 4 years ago - /u/JagexHooli - Direct link

Originally posted by joe32176

I know you’re kinda newish so it can slide. But in the future don’t say something is coming soon, ever. Say a week, writhing the next month, but not soon.

The official mobile release has been coming ‘soon’ since Winter 2017.

Thanks for the feedback on this. Fixing elements of our past communication is a focus for the team all up, so hopefully we can win back the trust around the word 'soon'... uhh... soon.

This Quest announce will be a good demonstration of that, unless something suddenly goes horrifically wrong because I jinxed us in a Reddit comment.