[2020] The State of Heroes of the Storm A.I

Hello, and good day to you all!

My name is Kevin Gu, and I am a Senior Designer on the Heroes team. I have been working alongside Jeff Beaudoin, our Lead Gameplay Engineer, on our game’s A.I. We wanted to take a moment to update you all on where we are and how our three major A.I. initiatives are progressing.

Reconciliation of the Goal and Tactical Systems

There are two systems under the hood of Heroes that work together to drive agent behavior: the Goal system and the Tactical system. The Goal system is responsible for determining where A.I. agents should move, while the Tactical system provides agents with awareness of their surroundings so that they use and target their abilities intelligently. Last year, in an effort to improve agent in-combat positioning, we made it possible for the Tactical system to override the Goal system. Although we believe the changes were ultimately a good iteration, they did expose several underlying issues that we are addressing.

Moving forward we will be reverting some of changes made in the last major overhaul and restructuring the Tactical A.I. behavior tree. We are reinforcing the intended separation of the Goal and Tactical systems, and will be paying closer attention to make sure that any future changes do not disturb this delicate balance of power. This will reduce the frequency of agents being pulled in two directions at once, most commonly observed as agents rapidly bouncing back and forth in place. This work will also allow us to more quickly diagnose the root cause of misbehaving agents.

Laning and Defense

Our second priority is to improve A.I. agents’ understanding of “laning” and how they approach pushing and defending those lanes. In the most recent patch, we added logic that allows the Goal system to understand that lanes pushed closer to a core are usually more valuable to defend. Our next step is to include units within each lane to further modify how valuable lanes are to defending agents. Both pieces will ensure that the A.I. will better understand how to defend their lanes.

Goal Staging

Our third priority is to improve the ability for agents to prepare or “stage” for their assigned Goals. Our internal Tactical A.I. tree can now better determine if all assigned agents to a goal have arrived. This allows agents to intuit if they should wait in shrubs because their team members are not yet near their collective objective, and subsequently helps agents work together.

Thank You

We want to extend a big thank you to those who have shared feedback on A.I. We hear you! We are hard at work improving the A.I., and plan to bring the items mentioned above and more into the next major Heroes patch. Again, thank you for your continued patience, and we’ll see you in the Nexus!

Yes, this is something that I have fixed for the next release.

Hey, mynd.

I agree with the frustration of watching disconnect replacement A.I. try to help, but often end up becoming a liability to your team. I want to share with you more details of additional changes related to your feedback that we are hoping to ship in the next release, details that we did not have time to include in my original post or the blog.

When your ally disconnects from the game, their replacement A.I. is put into a precarious position. It’s like suddenly dropping a Silver player into a match with 9 Master players. It’s very difficult for the AI to keep up with humans. In the past we have tried to make A.I. more sophisticated by attempting to capture more and more game context into their decision making, hoping that it would make them react more intelligently. Our collective observations have lead us to accept that disconnect replacement A.I. will never be strong enough in its decision making to play well with human players on their own. Instead, we want to focus on addressing the frustrating aspects of disconnect replacement A.I.

Our plan moving forward is that we will be simplifying disconnected A.I. behavior by decreasing their metagame decision making. We will be paring down disconnect replacement A.I. behavior to two choices: follow the human who pinged you, or return to base if no human has pinged you. This will prevent moments where the lone disconnected AI trying his hardest to win the game by soaking top lane and getting picked off while the rest of your team is fighting at an objective. More importantly, this change will make replacement A.I. much less unpredictable. We feel like this is the best starting point in addressing a lot of the A.I. community feedback on disconnect replacement A.I.

Hope you have a great day!

Hey AnaBanana, this is something I will look into.

I think this is reasonable, thank you. I will take a look at the health thresholds that govern the healing fountain goal and see if it’s safe to adjust.

Hey, Stavros. This is reasonable, I can look into Nova (love her) decoy A.I.

Hi Teo!

Thanks for your notes! I can give you some insight into some of your points.

The reason why AI ignore Deathwing fire breath is because we made a conscious decision for them to ignore it. Otherwise, AI agents would be constantly running away from Deathwing. This is due to our pathing system. It isn’t sophisticated enough to navigate around dangerous areas like how a human player would. Unfortunately, “fixing” pathfinding is not a trivial task as it would require substantial engineering efforts.

The AI think tree can check target states like unstoppable. It might be the case that Stitches isn’t checking that his intended hook target cannot actually be hooked. I will note this for investigation.

This was also a conscious decision we made so that agents prioritize their activities near objectives instead of engaging enemies while traveling to those objectives. It’s to prevent them from becoming too distracted. But maybe it’s too passive?

Yes, this is definitely something I am watching closely. It’s a tricky issue to solve because chasing can very easily get the agents killed. I think it’s best to be on the side of caution for now, though I am not opposed to pushing agents to be a little bit more aggressive in the future.

Hey, thanks for the comment.

Good question, I can provide some insight into how an agent would approach each situation.

The way AI travels to a Goal (like a map objective) is the same as a move command. It’s the same as if you right clicked a tribute and just let the game take you there. This is our pathfinding at work and it’s what our agents use for traveling to Goals. While it will get you where you clicked by taking the shortest route, it does not consider other possibilities like flanks or shortcuts. Those types of decisions are simple for human minds to make, but would require a much more sophisticated pathfinding system for AI to replicate.

Once agents get within a certain range of their goal they will then begin using Tactical fight logic (using abilities). Imagine a bubble around each objective where they say “OK, I’m nearby, lets fight!”. All targeted hero abilities require a point to use. The point is found by nodes in the AI tree if the appropriate conditions are met (things like target must be below this % of health, or find the centroid of a group 3 or more of units who are X distance from each other). It’s entirely possible for a Johanna to Falling Sword onto the enemy across terrain, or ETC to slide across walls to stun someone, if the right conditions are met.

“Escaping” really just means that the Goal system has determined that there is another Healing fountain/Healing altar/objective/lane/camp on the map that is more valuable to be at than where we are right now. Because this is determined by the Goal system it will use the pathfinding system, which as I have said earlier is not sophisticated enough to find alternate paths such as sliding across a wall. Now, is it possible to “program” this behavior into the Tactical tree? Yes, it’s possible, but it isn’t straightforward or easy, and can bloat the Tactical tree and make it harder to navigate. It would also undermine our separation between the Goal system (agent movement) and Tactical system (ability usage).

TL;DR - we would need a more sophisticated pathfinding system in order for AI to attempt alternate paths to their Goals.

It’s an interesting idea, I think I’ve seen it suggested before. I can imagine maybe putting in a new ping to direct agents who are following a human to only use their heroic abilities on the pinged target. I’ll add as possible future improvement, thanks!

Hello SigylVanz, thanks for the questions.

Yes, it is intended they remain in the Hall of Storms until a player pings them to follow. They will attempt to return to the Hall of Storms if they are no longer following a player.

Yes, this is intended. I will explain. During testing there was feedback that AI agents who are following humans were too difficult to pull back for retreat because their leash ranges was too large. You had to run really far away in order for AI to begin leashing back to the player. We dialed the leash range down pretty significantly so that the AI will follow their player much more closely. Consequently, this means that having a melee AI follow a melee player hero makes them perform better than mismatching a melee AI with a ranged player hero. I’ve seen feedback similar to yours which leads me to believe that we may have reduced the leash range too much. I am aware of this and will be testing some adjustments. Cheers!