You can see the bug in action in the animated gif above this paragraph. Simply put, sometimes, when an enemy died, they would get up partially and then get stuck between laying on the ground and standing straight. Sometimes the enemy would get all the way back up off the ground.
As you can imagine, this was confusing in gameplay as it made it rather difficult at times to tell if an enemy was dead or alive. It also looked pretty amateurish, though I've been told by a number of our Early Access players that they've seen similar bugs in shipped games so I guess it wasn't the worst bug in the world. Still, I really wanted to fix it.
For the longest time we thought this was an animation bug of sorts. Our death animations were set to loop, and changing them to not loop did sort of almost solve the issue in some cases. We also had issues with the animation system we were using not having animation events, so we couldn't fire events based on the animation frame, meaning we had to guess as to when the animation was complete based on the running time and elapsed time. But even setting that to cut it off at 98% or even 95% of total time still didn't fix the issue.
The other problem was that the bug didn't seem to be deterministic. That is, we hadn't found a repro case where we get get the bug to show up 100% of the time. Anybody who's had to debug something that doesn't repro consistently is probably shuddering right now. Non-deterministic bugs are terrible to figure out. Sometimes the death animation would work, sometimes it wouldn't and they'd get back up. The only clue we had on that was the issue affected human NPCs almost exclusively.
Getting More Clues
Over the course of the project, I'd probably spent a good 5 programmer days hunting this damn thing down, investigating a bunch of systems that I knew and some I didn't know offhand to try and figure this out. In the end, it turned out to be a confluence of a number of different design/engineering decisions coming together and creating a vexing environment where it became very difficult to troubleshoot the issue.
The decisions in question were:
- The "stance" animation system (what determined the default idle state of the animation) was a data-driven state machine. When certain data points on the actor changed (down, concealed, combat, exploration), the stance animation system would switch the default idle animation based on a priority system, Down/Dead->Concealed->Combat->Exploration. The highest priority active state would dictate what the idle animation was.
- Down/Dead was data driven, rather than state driven. It was purely based on whether the actor had more damage than they had maximum health.
- To create a "corpse", once an NPC actor is dead, a set of scripts run
and delete everything off the actor, except for the actor's visual. This
would leave a model with no AI, no animator, and so on.
- Status effects would clear on down/death, so HoTs, DoTs, buffs, traits, etc.
- Design created an enemy spawning system where, to create variety in enemies, it started with a template and randomly added buffs/debuffs for differentiation. For example, a Wounded Sellsword would have -5 Fortitude compared to a Sellsword Initiate (the base template). Design currently only uses this differentiation on a subset of NPCs.
If you guessed that a differentiation status effect with a health reduction getting cleared on death--"healing" the actor--was causing the stance animation system to think the actor was alive and made the actor start standing until the concurrent death script deleted the animator thus stopping the enemy mid-stand, then congrats, you nailed it :) If not, don't feel too badly, since it took me a while to piece everything together.
The "easy" solution from there was to allow certain buffs/debuffs to persist through down/death. For the human and dog NPCs, their randomized differentiation buffs/debuffs need to use this option so that the actor's maximum health doesn't change when they die.
Another possible solution might have been to modify down/dead to be state based rather than data driven (ie: regardless of your health pool, you can still be dead or alive), but frankly that comes with a whole different set of issues, not to mention the amount of risk changing something so fundamental would create in the project at this juncture.
I hope that was an interesting look at how a bunch of different systems can act together in ways one might not expect. Even when you try to keep your systems isolated, at the end of the day they still need to interact somehow, and those interactions are where bugs tend to crop up.
And bonus, in our next patch, actors should now stay dead when they die, rather than trying to re-enact Thriller. #IndieDev, #EonAltar, #GameDesign