Dev Diary: AI Deep Dive
MCOC Team
Administrator, Admin [en] Administrator › Posts: 651
AI is a hot topic in our community, and there are a lot of misconceptions around what it is and how it works, so we thought it would be a good time for us to pull back the curtain on how it functions, and talk about where it’s going in the future.
Before we begin, it’s important to separate what exactly the AI controls from how it makes its decisions, as those are two different conversations. We plan to have both conversations, but for this particular post, we want to focus on the latter. Then we’ll dedicate a future post to clarifying what things do and don’t fall under the umbrella of AI in the first place.
So to start with some important questions:
What is the AI?
You can see here each frame has an event chain based on what rolls and decisions are happening. Here’s a closer look at an example of what that chain could look like at any given moment.
Obviously, it’s impossible for all of these things to actually occur, even the successful roles, but this creates a lot of noise when one is trying to figure out exactly why a certain outcome occurred.
What about the things that modify AI behavior?
With all the above being said, the AI doesn’t really know why it’s hitting the buttons it’s hitting, but when it makes a decision, it does so instantly and with perfect information (Frame Perfect Button Masher). So if there’s any window the AI can capitalize on, it will eventually find it. Think of how you sometimes tap a light randomly and find that you’ve just intercepted your opponent, and now imagine you have the perfect reaction timing to turn that into a full combo every single time, and to block if you don’t and see yourself being attacked in turn. That kind of in-the-moment randomness is what everything the AI does is built from. So while behavior patterns do emerge based on the probability spread of the different Action rolls, the most universal truth of the AI is that unpredictability. Now, some of that is the point in a skill-based fighting game, where an element of your skill is reacting to new information, but the current system is more random than that.
It’s important to reiterate that the AI has no concept of techniques like parrying or intercepting. That’s why it’s so difficult to address certain kinds of issues, like the AI light intercepting the player too often, because there isn’t a system specifically for that behavior in place, and so there’s nothing we can cleanly tune up or down. We can change global aggression values or tell the AI to use light attacks less often or to block more often, but all of those cause significant collateral damage to the overall play experience. At the end of the day, when the button masher at the heart of the system sees your character vulnerable in front of it and in range for a light attack, it is going to see that as the only valid action some real percentage of the time.
For these reasons, our long-standing policy was that we simply wouldn’t change the AI at all. Why risk making gameplay worse overall when you can’t even be confident the change will have the intended effect? As we’ve said before, we decided last year that this approach was untenable. Too many small changes were happening whether we wanted them to or not - we’d optimize the way fights are loaded, or clean up some background variables, or find and fix a seemingly unrelated code bug, or do something crazy like upgrade the entire game to play and function at 60 frames per second instead of 30… And something somewhere would change without us being able to roll it back or even meaningfully make a change in the opposite direction.
For a while, we tried making those targeted fixes anyway, being as careful as possible, knowing we couldn’t just watch the system decay in front of us. But even as careful as we were (putting fixes through rigorous QA, internal playtesting, CCP playtesting, etc.), we still sometimes discovered unintended knock-on bugs after we pushed a fix live. (Or in the case of the Medium Intercept fix, fixed one problem only to uncover another that had been hiding beneath it). And so, despite some clear wins (defender reparry, special baiting for larger characters), we know that this slow, piecemeal approach isn’t working either.
One bright spot in this part of the story is that this process has brought huge improvements to our internal toolsets. Some of you may remember that we announced a refactor to some internal logic shortly before the release of Necropolis, which was instrumental in helping us more quickly diagnose bugs and test fixes. That change allowed us to build out fight recording and scripting tools, which have already been a huge help in finding and fixing bugs both with the AI and with other parts of the Contest, and has done a lot to set the stage for what comes next. Speaking of…
The Moment You’ve Been Waiting For?
We don’t usually share this sort of information this early, but we want to continue this train of transparency and show you how seriously we are taking this. Our dearest and faithful Summoners, we’re actively in development on building a BRAND NEW AI system.
AI 2.0, to be exact!
Our goal with this rebuild is to gain the ability to actually do things like “just tune Intercept down.” Or to build a feedback loop where the AI actually understands that it has been throwing heavies against the far wall for 20 seconds and should maybe try something else. But first, we still need to build out new systems for perception and memory, and then for making decisions, and get those decisions as close to parity with the current system as possible. After all, that weird, random button masher is a core part of the Contest as we know it, and we don’t plan to start things off with a wild departure from what you’re used to. We know we won’t perfectly replicate the current experience, but we intend to get very close, only with access to many more tools and levers for further improvements. Additionally, we are building this system so that we have the ability to roll it out carefully and deliberately. We don’t plan to surprise you one day with an announcement that the whole Contest has swapped over, but instead plan to test this first in only specific quests or game modes.
Now, this is still a long way out, and there will certainly be a number of challenges in delivering this, but we are committed to improving the player experience here, and even further, to keeping you in the loop as we build out this massive new addition to The Contest.
Until then, and on behalf of the team, we’ll see you… In The Battlerealm!
Before we begin, it’s important to separate what exactly the AI controls from how it makes its decisions, as those are two different conversations. We plan to have both conversations, but for this particular post, we want to focus on the latter. Then we’ll dedicate a future post to clarifying what things do and don’t fall under the umbrella of AI in the first place.
So to start with some important questions:
What is the AI?
- Not an actual artificial intelligence
- A frame perfect button masher with random outputs ( More on this later )
- Too random, but also doesn’t respect randomness (It’s too random for designers to control, but also has old, hard rules that make it harder to understand than true randomness at times) making this very difficult to debug
- Actions - What “buttons” are available to press (Light, Medium, Heavy, Special Attacks, Block, etc.)
- Distance - How far apart are the Attacker & Defender
- State - What animation state are the characters in (Idle, Block, Dash, Special Attack, Hit Stun, etc.)
- Effects - The AI doesn’t know about the existence of Buffs, Debuffs, Passives, Timers, Nodes, Traps, etc. There are effects that modify how it makes its random decisions, but it doesn’t know if they’re active or not.
- Masteries - The AI doesn’t know about the existence of Masteries, even ones that now live at the center of MCoC gameplay. As an example, the AI is never attempting to Parry, because it doesn’t know what that is or does. It simply tries to block incoming hits, and this can cause it to Parry. The same goes for Dexterity, no matter how impressive the performance.
- Memory - The AI only knows about the present game state; it has no concept of things that happened earlier in the fight. This also means that it has no idea if it’s attempting an action for the first time in the fight or for the twentieth time in a row.
- Each Action the AI can take is given a weighted chance of being rolled.
- Every frame the AI rolls to see what Action it attempts, even if it is already performing another one.
- There are some rules that make an Action invalid (e.g. you can’t do this Action if you’re in this state, you can’t do this Action if your distance to the opponent is too great, and so on.)
- If the AI rolls an invalid Action, it will keep re-rolling until it finds a valid one, as the animation below shows.
You can see here each frame has an event chain based on what rolls and decisions are happening. Here’s a closer look at an example of what that chain could look like at any given moment.
Obviously, it’s impossible for all of these things to actually occur, even the successful roles, but this creates a lot of noise when one is trying to figure out exactly why a certain outcome occurred.
What about the things that modify AI behavior?
- AI Profiles - Each fight has an AI profile assigned that determines its default weights for rolling each Action. We primarily use two… “easy” for early game content, and “hard” for everything else.
- Champion Specific Tuning - In rare cases, if a Champion relies on a specific Action as a core part of their gameplay loop, we can add additional weighting to that Action for that character (e.g. Heavy Attack on Doctor Doom). We do mean rare here - there are far fewer of these cases than players usually suspect.
- Aggression - During the fight, hitting the Defender builds up a hidden Aggression stat, which adds more weight to certain attack Actions. This decays over time if you stop hitting the Defender. Both the growth and decay of this system vary by profile.
- Mistakes - Another hidden system is “Mistakes.” The AI has a chance in the roll to make a bad decision, with the original intent of giving the player an opening to capitalize on.
- Effects - There are effects that modify AI behavior (e.g. Taunt, Intimidate, Infuriate). They do this by adding weight to the targeted Action, such as a Special Attack in the case of Taunt, or by modifying the overall aggression of the system.
With all the above being said, the AI doesn’t really know why it’s hitting the buttons it’s hitting, but when it makes a decision, it does so instantly and with perfect information (Frame Perfect Button Masher). So if there’s any window the AI can capitalize on, it will eventually find it. Think of how you sometimes tap a light randomly and find that you’ve just intercepted your opponent, and now imagine you have the perfect reaction timing to turn that into a full combo every single time, and to block if you don’t and see yourself being attacked in turn. That kind of in-the-moment randomness is what everything the AI does is built from. So while behavior patterns do emerge based on the probability spread of the different Action rolls, the most universal truth of the AI is that unpredictability. Now, some of that is the point in a skill-based fighting game, where an element of your skill is reacting to new information, but the current system is more random than that.
It’s important to reiterate that the AI has no concept of techniques like parrying or intercepting. That’s why it’s so difficult to address certain kinds of issues, like the AI light intercepting the player too often, because there isn’t a system specifically for that behavior in place, and so there’s nothing we can cleanly tune up or down. We can change global aggression values or tell the AI to use light attacks less often or to block more often, but all of those cause significant collateral damage to the overall play experience. At the end of the day, when the button masher at the heart of the system sees your character vulnerable in front of it and in range for a light attack, it is going to see that as the only valid action some real percentage of the time.
For these reasons, our long-standing policy was that we simply wouldn’t change the AI at all. Why risk making gameplay worse overall when you can’t even be confident the change will have the intended effect? As we’ve said before, we decided last year that this approach was untenable. Too many small changes were happening whether we wanted them to or not - we’d optimize the way fights are loaded, or clean up some background variables, or find and fix a seemingly unrelated code bug, or do something crazy like upgrade the entire game to play and function at 60 frames per second instead of 30… And something somewhere would change without us being able to roll it back or even meaningfully make a change in the opposite direction.
For a while, we tried making those targeted fixes anyway, being as careful as possible, knowing we couldn’t just watch the system decay in front of us. But even as careful as we were (putting fixes through rigorous QA, internal playtesting, CCP playtesting, etc.), we still sometimes discovered unintended knock-on bugs after we pushed a fix live. (Or in the case of the Medium Intercept fix, fixed one problem only to uncover another that had been hiding beneath it). And so, despite some clear wins (defender reparry, special baiting for larger characters), we know that this slow, piecemeal approach isn’t working either.
One bright spot in this part of the story is that this process has brought huge improvements to our internal toolsets. Some of you may remember that we announced a refactor to some internal logic shortly before the release of Necropolis, which was instrumental in helping us more quickly diagnose bugs and test fixes. That change allowed us to build out fight recording and scripting tools, which have already been a huge help in finding and fixing bugs both with the AI and with other parts of the Contest, and has done a lot to set the stage for what comes next. Speaking of…
The Moment You’ve Been Waiting For?
We don’t usually share this sort of information this early, but we want to continue this train of transparency and show you how seriously we are taking this. Our dearest and faithful Summoners, we’re actively in development on building a BRAND NEW AI system.
AI 2.0, to be exact!
Our goal with this rebuild is to gain the ability to actually do things like “just tune Intercept down.” Or to build a feedback loop where the AI actually understands that it has been throwing heavies against the far wall for 20 seconds and should maybe try something else. But first, we still need to build out new systems for perception and memory, and then for making decisions, and get those decisions as close to parity with the current system as possible. After all, that weird, random button masher is a core part of the Contest as we know it, and we don’t plan to start things off with a wild departure from what you’re used to. We know we won’t perfectly replicate the current experience, but we intend to get very close, only with access to many more tools and levers for further improvements. Additionally, we are building this system so that we have the ability to roll it out carefully and deliberately. We don’t plan to surprise you one day with an announcement that the whole Contest has swapped over, but instead plan to test this first in only specific quests or game modes.
Now, this is still a long way out, and there will certainly be a number of challenges in delivering this, but we are committed to improving the player experience here, and even further, to keeping you in the loop as we build out this massive new addition to The Contest.
Until then, and on behalf of the team, we’ll see you… In The Battlerealm!
102