Intro

Back to the Backrooms is getting an exciting update with a brand new AI system for the enemies. Our team has been hard at work to improve the responsiveness of the AI, fix bugs, and futureproof the code for easy expansion. In this devlog, I'll take you through our journey from the old AI system to the new, highlighting the problems we faced along the way and explaining how the new system will make gameplay even more engaging.

The Old AI System

First things first, our game uses Unity, and as some of you may know, Unity has an asset store with some great tools. One of these tools is Node Canvas, a visual node-based tool for creating scripts, which we used in the old version of the AI system. We started using Node Canvas because we thought it would be a faster way to create the artificial intelligence, and that it would be easier to translate our planning diagrams into it.

Here's an example of the first planning diagram we created for the AI:

Using Node Canvas was a bit of a learning curve for us since we had never used it before, but we managed to figure it out and it worked fine at the time. 

Node Canvas had some logic built-in that we could use, such as the wander routine which made enemies walk randomly around the map. However, there were some things that we had to create ourselves, such as how the enemy would investigate sounds in the level or how it would patrol the level.

The First Problems

Although the old AI system worked fine for most situations, we encountered some bugs that were hard to fix, such as enemies getting stuck to corners and attacking at nothing or not moving properly while patrolling. Debugging was also a challenge because we were limited in how we could change the code and we weren't familiar enough with Node Canvas's debugging tools.

Another problem we faced was that we wanted to keep every AI variation under the same node tree to avoid constantly switching between them to program each enemy and to try to keep some parts of the behavior identical. However, this made things more complex and harder to track as we added more behaviors to the AI system. For instance, we had three "regular" states an enemy could have: static (stand still until the player gets close), wander, and patrol (walking around the same room). These states were already making the old node tree look messy, and we knew it would become unmanageable as we added more of them.

Keeping everything in the same node tree also meant we had to choose one of the behaviors the enemy would use, and the way we did that was by using enums. While enums are fine for selecting things that won't change over time, they're not suitable for an AI system that needs to evolve. We knew we had to change how the system worked to address this issue.

The Solution

Although we could have continued using Node Canvas and tried to learn it more, it would have slowed us down. Instead, we decided to create a new system from scratch. The way we did it is similar to the node tree, but it involves the concept of state machines.

A state machine is a way of representing the different states that an object or system can be in and how it transitions from one state to another based on certain conditions. It's like a flowchart that shows the possible states and the actions that can be taken to transition between them. In a game, a state machine can be used to represent the different states that a character can be in, such as walking, running, jumping, or attacking. The state machine can also handle transitions between these states based on game events.

To create the state machine, we used Unity's Scriptable Objects. Scriptable Objects allow us to create and store data objects that can be reused across multiple game objects and scenes. The first step was to create a way to represent the states, transitions, and conditions.(For those who don't want to read code specific stuff skip to the next topic).

The basics of an state is that it will run a script when enter, updating and when exiting. When an enemy is spawned all their states duplicated so that it doesn't affect other enemies that use the same state, and for this reason, we need a reference of the enemy that is using this state so we can call it from within the states if we want.

Next, we created the code for the condition testing and transitions. The conditions are checked if they are truthy or not, and the transitions are stored on the EnemyState itself. Each transition is a simple structure that contains what the next state is, the condition, and a way to invert the condition if needed.

With all the base code of the enemy states defined, we only needed to insert it into the main Enemy code. for that I'll spare the details and simply say that every game update, the enemy checks the conditions of the state its currently in, and if one of the conditions is valid. If one of the conditions is valid, the state changes to the one that was present in the transition.

Implementing New Enemy State Routines

Now that the base of the state machine is complete, we have begun creating new states and conditions for our enemies. Some are based on existing states, while others are entirely new. We've also created icons for the scriptable object assets to make it easier to identify what each enemy is doing and which files need to be configured.

We currently have 7 states and 8 conditions ready, and we plan to have 11 states and 9 conditions for the next update. With scriptable objects, we create a file that functions as a saved instance of the original scriptable object. After creating this instance, we can configure it however we like. For example, here is the configuration of a Patrol state:

Once we've created scriptable objects for all the states and conditions we need, we add them to the enemy state machine.

The great thing about using scriptable objects is their modularity and ease to configure each state uniquely for each enemy, if we want to . If we need to change any part of an enemy's behavior, we can simply remove one state and replace it with another without much trouble.

As an additional feature of our system, we can check for any reference errors in the enemy states or display the gizmos of each condition. Gizmos are a visual tool for seeing what the condition is testing.

There is still much to do, such as improving the visualization of what's happening between states and better error detection. However, the system seems to work well for now, and we hope to continue improving it further.

Thanks for reading. -Diogo V.