Friday, October 24, 2014

[IndieDev] I Choose You! Threat and Other Artificial Intelligence Criteria

Making enemies look somewhat intelligent has always been a difficult problem in video games. When you as a player decide what you want to do, you're taking into account a massive amount of data, distilling it down (often subconsciously), then making the decision from there

Things you might take into account are how much ammo you have, which enemy is the most dangerous, which enemy is healing others, which enemies have the biggest impact of health to damage so you can take out smaller enemies first, dodging attacks, buffing allies, using special abilities, hitting levers and other level-specific objects, and so on.

It's no wonder that creating an AI is difficult, and not only that, but different enemies may have different priorities and abilities! The slow-but-sturdy enemy isn't going to try and sneak by you to flank you, whereas the rocket launcher mob isn't going to run into melee.

When you look at a game like WoW, the grand majority of AI is relatively simple. Based on a stat called aggro, you decide who you're going to attack. Then, based on your abilities, you choose an ability. Melee? Run up and hit them. Caster? Stay back and cast (some enemies will even run away a bit and cast). Use your most powerful ability that isn't on cooldown. Bosses often have complex choreography which are basically scripted patterns rather than any sort of intelligent decision making, which makes many bosses in MMOs more of a puzzle rather than a dynamic fight.

The Heigan Dance is the ultimate example of an extremely scripted battle.
I enjoyed it, but intelligent behaviour it was not.
But when players can blatantly manipulate enemies--which the aggro/threat concept generally allows them to do--enemies feel less like they're smart and you're not fighting for your life, you're actually composing a battlefield. Playing Combat Tetris, essentially. I'm not saying that's bad--I love the boss fights in most MMOs; I also love Tetris. Puzzle games are one of my favourite genres, and we're basically solving and executing a group puzzle when we're raiding.

However, the beauty in a game like D&D or in PvP is that there is actual intelligence on the other side of the playing field. A DM who's coordinating the enemy party, or the other arena team who have their own goals, strategies, ideas, and human nature.

So, how do you make enemies smarter? By making them take more criteria into account, and by giving them more options. This isn't easy, or cheap in developer time, though.

Threat is a popular mechanic because it's easy to implement, and (relatively) clear to communicate. Players perform actions, they generate threat. Whoever has the most threat, or passes some threshold, now has the enemy's attention. Some players can generate threat at an accelerated rate (tanks, usually) to hold the monsters' attention. Maybe in one game, healing generates four times as much threat as damage, so the healer is constantly in danger of getting attacked. It's basically a calculated heuristic to emulate in D&D when your Ranger crits the boss, the boss is going to turn around and deal with them because they're currently the biggest threat.

But what are some other criteria we could use?

Proximity is one that could be used. Some enemies are just fodder, and used to hold the front line, so they'll go engage the closest player. Maybe you want to have a rogue NPC skulk around the outside of the battlefield, so they'll choose the furthest player. Or if a player starts casting spells, perhaps you want your archer enemies to focus on them first. Or you have a dual-wielder who goes after whichever player has the weakest armor so they can do the most damage.

In WoW, some of my favourite fights were ones that eschewed the traditional threat mechanic. Take the Faction Champions in Trial of the Crusader, who usually targeted whoever was at the lowest percentage health; or Garalon in Heart of Fear, who chased whoever was affected by the Pheremones debuff.

Garalon, also known as the PUG breaker. Hell, he was a raid breaker before he got nerfed.
And if you're really clever, you can find a way to weight the criteria together. Closest spellcaster? Or choose someone at a weighted random based on defense and threat. Once you start putting these things together in a way that makes sense mathematically, you can build pretty specific behaviours quite easily that look smart; like your enemies are actually thinking about who they should attack. Because they're doing exactly that!

It's not just targeting who, though, it's also about what powers you want to use. Given a list of powers, it's easy to say, well, pick the most powerful one that's not on cooldown. Basically, a priority queue not unlike most DPS classes in WoW today. But maybe they also have a power that's a stun, and they want to turn and stun the person on their back before turning around and beating on their target. You could build a feedback loop between target selection and power availability to help make something that really has a good idea about the best thing it can do.

Which all leads back to the player, who needs some way to counter that thought. Aggro and threat in most MMOs are simple, because combat is simple. Players and enemies can walk through each other; throwing down impediments to movement exist, but outright barriers often do not. Thinking back to the Warhammer MMO, where tanks were tanks because they literally blocked enemies from passing. 

This sort of thing allowed for more complex behaviour, because if the enemy can just choose the one in the dress and walk through everyone to get there unimpeded, you either need to be able to kill the enemy before they arrive, block the enemy, or grab its attention (or, I suppose, make yourself untargetable either via invulnerability or invisibility or the like). Otherwise the player will feel that the enemy is being unfair. Taunting enemies gives players the ability to have some control over the battlefield when other game mechanics cannot.

When you can't influence the battlefield as a party and the enemy is coming straight at the healer.
AI design is inherently a loop between game mechanic design and ability design, on top of actually building the criteria weighting. But while you can make a super smart AI, you also need to ensure the game mechanics allow the AI to be "fair" with respect to the players. It's fun to have smart opponents. It's not fun to have smart opponents who don't take into account your own actions. NPCs should be intelligent, but players still need to be able to influence the battlefield somehow, or you'll end up with bored and frustrated players.

Arguably--and most MMO games would likely agree with me given their own designs--player agency is more important than super smart AI. But if you can grant players agency, having the enemies also able to use those mechanics makes for really interesting combat.
#IndieDev, #GameDesign


  1. So what's your plan to push your AI forward while preserving player agency?

    1. That's an excellent question.

      Given our game is turn-based, every action has a lot of weight to it. Actions often include movement and something (sometimes more than one something). There's a focus on positioning, line of sight, and engagement within combat itself. Giving players the tools to play with those as part of their powers (in a way that makes it easy for players to understand what those effects are)--similar to how many aggro-based games give players tools to drop or force aggro--should allow players to influence the battlefield, albeit less directly than just forcing enemy A to attack you (though I believe our designers are planning for that as well).

      Also important is to be able to visualize or communicate player agency. If I use an ability, it should be obvious as to the effect. If I blind someone, it should be easy to see that perhaps their ranged attacks are no longer effective or available, or if I toss down a wall it should block line of sight and movement. That communication is as important to the feeling of agency as the actual abilities themselves. Enemies need to be able to react to those things as well.

      Of course, teaching the AI how to use those abilities in a way that makes sense as well is definitely important. It's "easy" to program basic patterns (ie: spellcaster at the back is pinging you, drop a Wall of Earth in between melee and casters to block their attacks) to work in concert with encounter design, and that might be enough (or have to be enough given our extremely limited resources), but I admit I haven't had much time to think about a more general system for our AI's action decisions yet.

      Bringing it down to Earth for a second, it's a little bit interesting from my perspective, as I'm a developer, not a designer on our current project, and my time is split between implementing combat (and AI) and other more mundane subsystems. I have the capability to provide some feedback, but unfortunately I'm not necessarily involved in all the discussions. Small indie company means we do wear a lot of hats, so I get to have some participation, but we'll see if we have the resources/time/capabilities to turn vision into reality. As your own recent Netherspite blog post shows, sometimes you just have to ship it.

    2. A realistic answer.

      There's a few other options though.

  2. Garalon *still* wipes LFR groups. It's amazing at first, and then horrifying when you realize you'll have to try it again.

    Strangely, despite our decision making capabilities, people never seem to realize that standing in the big purple circle under him is bad.

    1. When you have 25 people, especially random people, chances are high that you'll get somebody who's super new, or super unobservant, or just plain bad at the game (and don't care to improve). Garalon it only takes one...

  3. Would you say that there is a difference in difficulty between improving AI in turn-based games like yours, and real-time games like MMOs, RTS and FPS?

    Back in cataclysm my guild had a bit of a discussion about boss AI and how to make those fights feel more dynamic and reactive. One idea we came up with was simply adding triggers that player actions could set off, that would produce reactions in the boss.
    So, for example, let's say a melee dps did a critical hit above X damage, that would trigger a % chance for the (let's say humanoid) boss to turn around and kick them away, doing serious damage - 2/3rds of a full health bar, maybe. Or maybe if there is a burst of damage on the boss, say it took X% of it's health in Y seconds, it would try to run away from the group for a bit. Or perhaps this boss really hates fire, so any hits it takes from fire attacks have a chance to make it go after the source for a little while.

    All those are purely reactive behaviours, so with my limited understanding of coding would seem to only require adding "if X happens, roll to see if boss does Y" type clauses to the encounter, rather than be preference-based AI behaviour. Does that make sense? Is this a viable path, or would more complex AI be an easier solution?

    1. That's a good question. Honestly I'm not sure of the answer offhand. Real-time often has the benefit of not having everything scrutinized too closely in the heat of the moment. In an FPS you don't usually have time to sit and watch the enemies too closely--it ends with you getting shot. However, with a turn-based game, people are watching every action.

      But really, I think the bigger factor here is your game's ruleset. The more complex the rule set, the harder it's going to be to make your AI look smart. An RTS will likely have much more complex AI than most individual enemies/encounters in an MMO or turn-based game. The AI has to deal with so much more in an RTS (basically the entire ruleset), whereas in an encounter-based game like WoW, you can get away with a lot of scripting and encounter design to fake it. You're also dealing with a much smaller list of possible actions in most RPGs (where enemies may have 3 attacks at most quite often, excepting bosses) compared to an FPS or RTS, where you need to deal with much more complex geometry as well as unit/weapon balance, etc.

      Purely reactive behaviours are easy. As you imply, if you're just scripting specific behaviours to occur in reaction to some trigger, that's simple. And probably pretty good bang for your buck in terms of cost:effect ratio. It gets harder when you want to create a more generic base system (ie: go after healers or the highest DPS, etc.), which returns us to the original point at hand. Threat systems are basically enemies reacting to player actions in a generic way.