Command Pattern - Encapsulation, Undo and Redo

Programming folks (otherwise known as “programmers”) often talk about encapsulation - which can be a very power concept and tool that can save plenty frustration and prevent bugs as a project grows large.

The idea of encapsulation is that a class or a system has everything it needs to function INSIDE of itself - it doesn’t need to reach out for references and is not dependent on other classes or systems to function. Encapsulation decouples classes and systems. The coupling of systems or classes should generally be avoided to keep your project stable and robust as it grows and has features added.

The command pattern is all about sending information, or a command, to an object. In the case of the command pattern the command itself is encapsulated! The sender and the receiver of the command are not necessarily encapsulated, but rather the command itself is. 

This means that all the information needed to execute that command is wrapped up inside a command class. What this means is the object executing the command doesn’t need any external references - all it needs to do is tell the command to do its thing and the command is fully independent and decoupled from other systems! 

While this requires some framework and intentionally to create - it can be very useful. It means that commands can be added to a list or queue, run immediately, or run at any later time. This adds or allows significant functionality that would be difficult to achieve with just direct function calls.

This makes it great for asynchronous applications - one example might be a turn-based strategy game where a sequence of commands is created one at a time and then all the commands, or a select number of commands, can be run when it is that object’s turn. 

The command pattern can also be used to “easily” create of an undo system - which depending on your project might be helpful or even crucial.

For this post (video), we’re going to start simple. Just using the command pattern to collect commands in a list and execute them over a short time interval. We’ll then add in the ability to undo and redo commands. This will result in some messy code, but we’ll finish up the post by wrapping all that messiness into a “command handler” class - which ends up pretty clean and tidy due to the encapsulation of the commands. 

The Code Begins!

The Command Interface

Again, the core idea of the command pattern is to encapsulate the command in an object. To do that we first need to create an interface that will have two functions. The first function is the “execute” or “do” function. This will contain all the code for the command to complete its action.

The second function is an “undo” function - which will need to have code that reverses or undoes whatever happens in the execute function. 

Exactly what the undo function looks like is 100% dependent on what the command is doing.

With the interface complete, we then need to create classes for the actual commands. 

The Move Command - Which Implements the Command Interface

For the purposes of this video, I’m just going to create one command which is a “move command.” This class will of course need to implement the command interface.

In the move class, we need three variables. The first is the direction to move, the second is the distance to move and the third is the transform that will be moving. All three of these variables are private and will have their values set in a constructor.

By doing this, the command instance will have all the information and references that it will need to execute the move command. 

This is pretty clever in its simplicity. It’s also pretty clean!

The Execute and Undo functions are fairly straight forward in that they change the position of the transform. The execute moves the transform in one direction and the undo moves it in the opposite direction.

Simple and tidy. Just the way I like it.

I’ve also included a “get move” function that is used in order to draw the path taken by the object - this certainly isn’t the only way to do it. It’s not particularly clean, but it gets the job done and it’s not the focus of this video. For the most part I will be ignoring the path drawing functionality as it’s tangential at best to the command pattern.

turn Based Input Manager

To control the object and issue the commands, I’ve created a few UI buttons. These are connected into a basic “input manager.” 

In the start function a listener is added to each button. The direction buttons all call a “send move command” function. While the “do turn” button will call a function on the player object that will execute the list of commands.

How exactly the input is gathered, of course doesn’t matter, but the “send move command” is crucial and does a few important things.

This function takes in the transform to be moved, in this case the player object’s transform, the direction to move as well as the distance to move. The function then creates a new instance of the move command and send the command to the character controller. 

For my example it also adds the command to a UI element that displays the commands on the screen. 

A better approach to sending out these commands would probably be to use the Observer pattern or in other words use events… but I didn’t want to go too far astray from the command pattern and complicate the main idea. If you aren’t familiar with the observer pattern or events - definitely go check out that video it’s easily one of my favorite and most used patterns.

Turn Based Character Controller

In our character move class, there is a list of move commands that each incoming command will be stored in. You can see this happening in the “add command” function. 

In general, the list could hold types of ICommand, but in order for my path drawing to work I needed to constrain the list to the more specific type.

For simplicity I’m not showing the code that does the path drawing, but if you want to see that you can check out the full code.

There is then a “Do Moves” function that will be called by pressing the “do turn” button. This function calls a coroutine that iterates through the command list and calls “execute” on a command and then waits a short time before looping through to the next command.

And that’s pretty much it for a basic implementation of the command pattern. You just wrap up the commands which contain everything that needs to happen and ship it off to a list for storage so it can be executed when needed. 

Again, pretty simple, nice and tidy.

Adding Undo

So let’s look at how an “undo” system might work. The command interface and the move command class remain the exact same. No changes needed at all.

The input manager is functionally the same, but now has an option for an undo button. This new button will call a public  “undo” function on the character controller - again not the cleanest implementation and an event would likely be better.

The big difference comes in the character controller script. Here I’ll modify the behavior by now having the player object move in real time as the buttons are pressed. This isn’t necessary for the undo function to work, but I think it makes it easier to demonstrate the undo functionality.

In order for the player object to move in real time, we simply need to execute an incoming command whenever that new command is added. 

Real time Character Controller with Undo

New to the class is the “undo command” function. This function first checks that there is a command to “undo” by checking the number of commands in the command list, then calls “undo” on the last command in the list and then removes that command from the list. 

Tis is super simple! All thanks to the command interface! 

When I first saw this I was surprised and even a bit shocked how easy this could be. Now of course if the command is more complex than the move command the writing and testing of the undo function will be more difficult and more time consuming.

BUT! 

The interface makes it easy to call that undo function and the inclusion or encapsulation of all the values and references provides a solid framework to create undo functionality.

Redo…

For many games this may be enough, but if you want to also implement a “redo” functionality things are going to get a bit messier - or at least the way I did it. All the changes are made on the character controller. The commands stay the same and the only change to the input manager is to add the redo button and connect it to a redo function on the character controller.

Character with Undo and Redo

For a redo system, we could simply not remove commands from the command list and then execute them again when a redo button is pressed. Essentially just working our way up and down the list of commands...

But! And there is a but.

If some commands are undone and then new commands are added this will cause problems as those new commands would just get added to the end of the command list…

SO! 

We need to modify our lists and keep track of which command on the list we currently executing or undoing. So yeah. Sounds simple, and it’s not crazy, but it does get a bit messy.

So the first thing we need to do is add an index variable that can track which command on the list we currently working with. 

Each additional command added to the list, the index will increment up one AND with each command undone the index will increment down one. This means we also need to change which command is being undone to correlate with the index and not just the length of the command list. 

With that all done we now need to add a few lines to our “add command” function. We need to check and see if our index is at the end of the list - meaning we’ve undone some commands and are now adding a new command. 

If it’s not then we need to remove all the commands between our current index and the end of the list by using the “remove range” function.

With this complete, we should be able to add commands, undo some of them and add more commands without screwing anything up. If we were to test this, on the surface we haven’t gained new functionality, but it does mean we can now add a “Redo Command” function. 

This function is actually fairly simple, we do some checks to make sure we won’t get a null command or get an out of range error. And then we simply execute the command that correlates to the value of the index. 

Once again, the interface with the execute and undo functions makes this surprisingly simple  - minus the tracking of the index.

But!

It’s So Ugly!

Command Handler Class

There is still some overall ugliness to this solution. I don’t like that this code is in my player controller… The AddCommand, UndoCommand and RedoCommand have NOTHING to do with the player controller they could and should be general and usable with ANY command. This code is generic and should be reusable. 

Remember the whole point of the command pattern was to encapsulate the command… so these functions and the list of commands could be just about anywhere.

So is no need for all this code to be in the player controller AND if you were making a game with multiple types of objects that could all receive commands it would be a pain, and more importantly error prone, to repeat this code in multiple classes. 

So let’s extract the command code and stick it into a command handler class. 

Super Clean!!!

This class contains the list of commands, the index as well as the functions to add, undo and redo commands. Then in the player controller script, we simply need an instance of the command handler class.  

Final Input Manager

The final tweak is to reroute our buttons in the input listener to call functions on the command handler. 

If we take a step back, what we have is really clean and surprisingly generic. 

Yes, we do have a lot of classes, but that is often a trade off with our programming patterns and I think cleanliness of the implementation more than makes up for the extra classes.

Due note that this implementation does lose our ability to easily draw the path of the player, but with some cleverness, especially if commands are sent with events, this can be gotten around without too much work.

A few last thoughts…

Like many patterns it’s not the exact implementation that is important, but rather the large idea and framework. While I used direct function calls, I think using events i.e. the observer pattern could be more generic and more powerful. 

The command pattern may result in commands being created and destroyed with some frequency. Which can create unneeded garbage collection. While not a problem with my simple example, it would be possible to implement an object pooling solution to use with the commands if performance is crucial or if you just want to squeeze out a few more FPS out of your game. 

Code

GitHub Link: https://github.com/onewheelstudio/Programming-Patterns

Strategy Pattern - Composition over Inheritance

The strategy pattern is a subtle pattern. It’s all about minimizing the duplication of code and decoupling classes. As an added bonus, the strategy pattern can also allow behaviors or algorithms to swapped at runtime without any messy switch statements or long chains of if statements. In the end it doesn’t have a super flashy or exciting outcome. It’s just good coding practice.

With some aspects of the pattern it’s easy to think, “Yeah, but this other way works…” But! The pattern is solid and avoids lots of little issues that can pop up later down the road when your project gets bigger.

Before we jump into the pattern lets first look at the problem it solves.

Inheritance Is Not Always Awesome

Let’s imagine you’re making a game based around weapons, or at least has a lot of them in your game. It would seem reasonable to create a “Weapon Base” class that other weapons can inherit from. This base class can contain basic stats and functions that can be used or overridden by sub-classes.

For example, we may have a DoDamage function that gets called every time a weapon is used. The function might simply reduce the health of the player’s target.

This is all reasonable.

Going a step further, let's imagine that we want to create 3 fire-based weapons that will all inherit from the WeaponBase and on top of reducing the targets health will also do some actions specifically for fire damage.

I now have 3 new classes that all have duplicated code. The DoDamage function has the same code from the base class, plus fire damage specific parts.  Updating the fire behavior means opening and changing the code in all the every fire weapon class in your project. This isn’t horrible with 3 weapons, but imagine having 20 or 50 or 100 weapons. Yeah, that’s not going to work.

I could also call base.DoDamage, but then all my weapons would be dependent on the base class DoDamage function, which is definitely NOT AWESOME.  If the base class function changes, all the inheriting classes change too and that’s not good. That’s not a solid foundation to build on. That’s a way to break your game in a hurry. This coupling between classes is what we want to reduce!

Now you might now argue that you could create a “Fire Weapon” class that inherits from the weapon base class and that all fire weapons inherit from… Which may work, but it is starting to get messy. Imagine now that you want to add ice or poision damage? You’d have to create Ice Weapon and Poison Weapon classes that those new weapons have to inherit from.

Okay, push comes to shove this might still be okay… Ugly, but okay, if the project stays small.

What if you now have a weapon that will do both fire and poison damage? Which class does it inherit from? Fire or Poison? Or do you make a combo class to inherit from? NO! Please don’t.

The strategy pattern can help solve these problems…

Strategy Pattern

The strategy pattern is all about encapsulating or wrapping up a behavior or algorithm in it’s own class. It’s also very closely related to the concept or belief that composition is better than inheritance! The exact details of how we do this are less important than the overall pattern so let’s start with a simple and common way to implement this pattern.

First, we create an interface called “IDoDamage” (you can argue all you want about using “I” to name an interface - I don’t care). This interface will have one function called “DoDamage.”

At this point, you might be thinking, “Okay, we’ll just implement the interface in all our weapons.” And that would be understandable, but it would be a mistake to do that as that would cause lots of duplicate code and not really buy us much in return from just good old inheritance.

Instead, we are going to create an instance variable of this interface in the Weapon_Base class. This class will also have a function that calls the DoDamage function on the IDoDamage variable.

Why? Good question. This is the crux of the whole pattern.

We can create classes that implement the IDoDamage interface. Each of these classes will have a different damage behavior. This will encapsulate the damage behavior AND make it so that we can change behavior at runtime by a simple assignment - no ugly switch statement or crazy chain of if statements needed.

For example, we can create a “FireDamage” class. This can do all the basic damage bits and most importantly it can then do any fire specific bits - maybe there are events that play sound effects or trigger specific lighting effects.

Then!

We create a new class for each weapon that inherits from Weapon_Base. Rather than hiding variables or overriding functions we use a constructor to set basic variable values AND to set the damageType variable.

While we now have a poop ton of classes, which could be a criticism of the pattern, we have very little duplicated code, and if we need to change the fire damage behavior, it only needs to be changed in one place in our project.

There is a neatness, a tidiness, a cleanliness that just feels good with this implementation. All we are doing is using a constructor to set up the weapon. The entirety of the damage algorithm or behavior is fully encapsulated in another class. While we are still using inheritance, we have decoupled much of our code, and much of the messiness of inheritance isn’t present in our solution.

Adding More Behaviors

The strategy pattern also works if you want to create other types of damage, such as IceDamage. To implement this style of attack, we need to create new IceDamage and IceSword classes.

Going Abstract

You could go either further and create generic weapons that have their damage and damage type set by a constructor. This could allow generic classes for each weapon type with all the data PLUS the behaviors injected into it.

Changing Behaviors

And I think the real cherry on top is that with the strategy pattern is that it allows easy changing of behaviors at runtime. Sure, you could do that with some if or switch statements. But those tend to be ugly. They break. They’re generally a brittle approach to programming and we can do better.

We can add a function to Weapon_Base to allow the damageType variable to be set. This would have the effect of changing behaviors. Something the code on the right.

Yes, I realize I made the variables public, but I don’t like changing values in classes from outside the class without using a function. If this was my project, I’d probably use private variables or maybe a public getter. 

With this functionality, a click of a button or the invoking of an event can change the weapon's damage type and thus much of it’s behavior.

If that’s not useful. I’m not sure what is.

Combining Behaviors

What if you really want that fire poison sword? Maybe your game is based around combining behaviors or abilities? Then what?

Make a list of IDoDamage values. The code can then iterate through the list and called DoDamage on each item in the list?

I’ll be honest I haven’t tried this but it seems solid and pretty useful.

Other Thoughts

The choice to use an interface in the strategy pattern is not the only choice. You may want to use an abstract class instead so that you can define variables. Personally I like the cleanliness of the interface and then simply injecting any needed data.

I also thought to use scriptable objects. And while I think that would work, I think it’s stretching SOs to a place they don’t fit particularly well. Writing the classes and then creating assets seemed like too many steps and I was struggling to find a situation where that would truly be better. But maybe I’m wrong?

I also wrestled with making the base class a MonoBehaviour or not. For simplicity I kept it as a MonoBehaviour so I could easily attach it to a button (for the video). I think that choice really depends on the use, but my gut say most the time I’d want it to NOT be a MonoBehaviour.

Observer Pattern - C# Events

The observer pattern is one of my all time favorites. I use it ALL the time. I’ve shared it with students and they almost immediately see the usefulness of the pattern. It’s not always the easiest to wrap your head around, but it is one of the simpler programming patterns to implement in Unity.

The Big Idea

The observer pattern is all about communication between objects, sharing information and doing so in a way that decouples the object that is sharing the information from the objects that might need or make use of that information.

Score and Achievement System

Using the same example project as for the last 2 programming patterns, let’s imagine that when the NPC kills a critter the NPC’s score goes up and gets displayed on the screen. One way to do this would be to have the critter (or the NPC) call a function or change a value on the UI element. This requires the critter to have a reference to the UI element - some form of “Find Object” or assigning it in the inspector. This works and it’s how many of us did when we first started.

But, what if there is an achievement system too. That system wants to display a message for the first kill, after 5 kills and then every 10 after that? Do you link the critter to the achievement system? Does the achievement system connect to the UI element? What if you have an audio system that plays a SFX each time the score goes up?

You can probably start to see the problem.

And it gets even worse! If the UI element changes or the designer forgets to put the achievement system in the scene then errors will get thrown and the game will start to break.

The result of all of this is a mess of spaghetti code that is highly coupled or inter-connected. If any one piece is missing from a scene the game will likely break. Plus if you change how one element works that could break all the connected pieces. This is a brittle project and will not be easy to finish. And yes, we’ve all been there.

This is where the observer pattern comes in and changes how objects communicate. Instead of all the objects being connected or having references to each other. The critter can broadcast a message that it was killed. Any objects that might be interested can choose to listen for that message (or not) and then do something based on what they’ve heard. The critter no longer cares or is aware of the UI element or the achievement system. If those systems change or aren’t in the scene - nothing breaks. If new systems want to be aware of when a critter is called all they have to do is listen for a critter to broadcast a message.

This is huge! This turns the UI and the achievement system into observers of the critters!

How Does It Work?

The observer pattern is so useful that C# has essentially baked it into the language. That makes the implementation of the pattern quick, but not always super clear or intuitive. So before we get to the implementation we’re going to talk about delegates, events, actions and a tiny bit about funcs. All of these bits are related, connected and useful. If you want to skip the explanation of these bits, you can jump down to the implementation section.

Delegates

I’ve seen few things in C# that seem to confuse folks more than delegates. There’s just something odd or mysterious about them. And admittedly there is a lot going on in the syntax of a delegate. So let’s try to clear some of that up,

And to give credit where it’s due - check out the two part video series by Sebastian Lague on delegates and events. When doing my research, I couldn’t find a better explanation than these two videos. He also goes over a few more or at least different examples that I will.

Delegates can be thought of as a variable that can be assigned a function as a value. A delegate can hold a reference to a function and then the delegate can be “invoked” and the function will be called.

Now that may seem strange. Why not just call the function itself. But you can imagine a scenario where you may want to change what a particular key does when it’s pressed. One way to do that would be to invoke a delegate each time that key is pressed. Then to reassign what the key does you simply have to change the function that is subscribed to the delegate. Easy. And hugely flexible!

But for me the real benefit comes from the fact that delegates in C# are “multicast” delegates - meaning that they can have multiple functions subscribed to the delegate. So invoking one delegate can call as many functions as needed. Add to this the fact that delegates can be made public and even static and that allows classes to subscribe and unsubscribe from the delegate.

And that right there is the basis for the observer pattern!

To keep things grounded, let’s think about what this means for our example project. If our critter has a “CritterKilled” delegate that gets invoked when the critter dies, then our scorekeeping UI element and our achievement system can both subscribe to that delegate. Whenever a critter dies it invokes the delegate which in turn calls a function on the UI element and a function in the achievement system! Each class is in full control of which delegates it listens to. The UI element and the achievement system have become “observers” of the critters!

Basic Implementation

A very basic even silly implementation of Delegates

To use delegates we must first define the delegate itself. You can see this in the first line of code (after the class definition) on the right. We then need an instance of the delegate - these can be defined locally inside a function or in this case they are defined with a class wide scope. It is this instance of the delegate that will be subscribed to and invoked!

We then need to subscribe a function to the delegate. This is done with an assignment statement. Notice that we have not included the parathesis after the name of the function! We are assigning the function NOT calling the function.

It’s weird. I know.

The last step is to invoke the delegate. This line also checks if there are any subscribers (actually it’s a null check) - this is done by the question mark. If a delegate is invoked and there are no subscribers an error will get thrown - which is why we need to check before invoking.

Now to be clear. This is a simple implementation. Not necessarily how it should be done, but I want to walk through delegates step by step and not jump straight into the shortest but most abstract syntax.

More On Delegates

Examples of return values and input parameters

Delegates can have multiple input parameters and can even have a return value - or both. It’s important to note that any function that is subscribed to a delegate must have the same input parameters and return value in order to not throw an error.

The input parameters are a great way to send information to other objects. For example when a critter dies it might want to send a reference to itself so that systems know which critter died. That’s not needed in this example, but can be useful in plenty of other cases.

Subscribing and Unsusbscribing of two funtions

In general return values are not used. This comes from the fact that delegates are multicast and can call multiple functions which could mean multiple return values. However, only the return value of the last called function will be returned, which can cause all kinds of confusion as it’s not easy or even always possible to control the order that functions are subscribed.

One than one function can be added to a delegate by using the += operator with each function. This operator adds a particular function to the delegate and likewise the -= operator will remove a particular function from the delegate. In general, it’s a good practice to do this in the OnEnable and the OnDisable functions. This is particularly important when a delegate is public and functions from other class are subscribing. If a function from a class instance doesn’t unsubscribe and the class instance is destroyed an error will be thrown when the delegate is invoked.

Example of a static instance of a Delegate

Also if the assignment operator = is used all other functions will be removed from the delegate, so in general += and -= are the best practice for subscribing and unsubscribing.

As mentioned above delegates can be made public and even static. In general, I have found that public static delegates are the most useful for the observer pattern. If delegates are made public and static they are accessed (and thus subscribed) to just like any other public static property or field.

Events

Great… So what about events?

Example of a Public Static Event - with an error!

Glad you asked. Events are just delegates. The difference is that when we are creating an “event” we are actually going to create a delegate but with the keyword “event” in front of the delegate. This does is a few very important things.

With a generic public delegate the list of subscribed functions could be overwritten by any class OR that delegate could be invoked by any class. Neither of these are good things - at least in general. Using the “event” keyword prevents these two actions from happening. All that can be done publicly to an event is to add or remove a subscriber - which is much safer!

Beyond that the implementation of an event is identical to a standard delegate! Notice that when the assignment operator is used we get an error.

Actions and Funcs

Okay, so delegates are awesome. What’s the deal with actions and funcs?

Both of these are objects inherit from delegates. And in reality they are just shortcuts to create a delegate. Actions are delegates that can have input parameters, but do not have a return value. Whereas funcs are delegates that can have input parameters and have a return value - funcs handle “return values” as an out value that is always the last parameter.

So what does this do for us with the observer pattern? Not a ton, but what it does do is reduce the number of lines needed to create an event.

Notice that each event is now defined on a single line. The use of the action has already defined the delegate for us. Notice too that the second event will handle an integer input parameter. This is put in as a generic argument to the action. This input parameter is assigned or determined when the event is invoked. Using actions this way is just a short hand for what we’ve already done above.

Back to the Project

Define and Invoke the event

If you’re still with me and your brain hasn’t melted let’s apply the Observer pattern to the example project.

For our example all the action happens when the critters die and this of course could and should be expanded to other game mechanics as well. So to keep things simple and keep going with the theme of “de-coulping our code” I’ve created a new script that will invoke an event when the object it’s attached to (a critter) is disabled.

Nothing too fancy. This lack of “fanciness” is no small part of the appeal of the observer pattern.

Then we have code on the UI element that is displaying the score. Here, the code subscribes in the OnEnable function and unsubscribes in the OnDisable function. When the event is invoked the “Update Score” method is called.

Then finally we have the code that displays the achievement message. This is very similar in that we subscribe and unsubscribe to the event and call a function when the event is invoked.

The observer pattern in C# is basically built-in, but it is super useful all the same. This is one of those patterns that if you aren’t using it you really should be. If it doesn’t make sense, then keep working until it does, because it will make your projects so much easier to maintain, easier to add new features and best of all far less error prone.

And that’s really it.

The State Pattern

Whoa. It’s been nearly a year since I’ve made a post. I’ve been busy. A lot has happened, but that’s a topic for another post.

So I’m going to try something new, well a few things. But the first thing I’m going to try is writing a blog post for my next video. I figure taking the time to explain the video in writing will help me cement the ideas and provide one more resource for you all. There are many people who’d prefer to read than to watch a video. So maybe it’s a win-win? We’ll see.

But enough of that. Let’s get onto the second new thing.

That’s is exploring programming patterns. It’s something I’ve wanted to do because I want to learn more about programming patterns. Plus Bolt 2 will eventually (!!) get released and when it does many of the basic programming patterns will be far easier to implement AND folks developing with Bolt can benefit from implementing them.

Since Bolt 2 is now fully in hiding under the protection of Unity (who knows when the alpha will make it out to the public again) the upcoming video will be entirely C# based.

I should also give credit where it’s due. I’m no genius - that should be clear if you’ve read more than a few paragraphs of my writing. Much of what I know or think I know about programming patters comes from the book “Game Programming Patterns” by Robert Nystrom. It’s well worth a look if you are unfamiliar with the topic of programming patterns..

The State Pattern

“Allow an object to alter its behavior when its internal state changes. The object will appear to change its state.”

Like any other pattern the State pattern has its limitations, but also like any other pattern when used correctly it can help to detangle code which can make debugging code or adding features easier and much less frustrating. And that’s a win no matter how you look at it.

You may have heard the term “Finite State Machine” or “FSM” thrown around. Both of these terms refer to the same thing which is an implementation of the state pattern. The Unity asset Playmaker is based on the uses of FSMs and Bolt has the ability to create them with “State Macros.”

To help illustrate the state pattern, I’ve created a simple Unity project. There is a NPC played by a capsule and it’s goal is to wander around the maze and collect the yellow cubes. When it sees a red sphere (critter) it will chase it and attack it. Nothing too fancy.

So with that in mind, how would you program this behavior? Take a minute and think it through. No cheating!

Just about ready to go up on steam! ;)

This actually doesn’t look too bade with the use of functions… but will get out of control in a hurry.

I know when I first got started with C# and Unity, I would have done it with an if statement or two (or seven). And that works. It really does. Until you need to add another behavior. Or another game mechanic gets layered in. And we all know it won’t end there. The layers will continue to get added until the code breaks, bugs appear or the project gets forgotten because it was getting too hard to work with.

Simple FSM = Enum + Switch

So let’s look at the simple FSM. I still remember learning about this and was utterly confused by how to implement it but at the same time amazed at what it did to my code once I finally got it working. At that time I hadn’t really learned to use functions well so I had 100’s of lines in my update function. Needless to say it was nearly impossible to debug and when I discovered FSMs it was a major break through!

Simple FSM using an ENum and a Switch

In the case I described above the NPC behavior can be broken down into 3 different states or 3 different smaller behaviors. I thought of these as wander, collect and attack. A custom enum can then be created with these three values.

With the enum created a switch statement can be placed in the update function. For each case we can call the relevant function for that behavior. But we still need a way to change or switch (get it!) the state. For the most part this can still be done with basic if statements.

And now this may sound like we’ve back pedaled to our original solution and just made things more complex.

But!

There is a major advantage here!

When we started, inside the update function we had to have a way to determine which state we were in - picking from ALL the possible states. Imagine how complicated that would be if there were 5, 6 or maybe even 10 different states. Yuck! Nothing clean about that.

With this method we only have to decide IF we need to change states GIVEN that we are already in a particular state. This greatly simplifies the decision making albeit at the cost of a few extra lines of code.

In a FSM it’s common that a given state can only transition to a limited number of other states. For example in my case maybe I don’t want the NPC to stop collecting if it sees a red sphere and it should only attack a sphere if it’s wandering around with no cubes in sight. If that’s the case the “state pattern” may actually reduce the amount of code needed as well as serving to untangle the code.

This implementation of the state pattern can be used for all kinds of different behaviors. It could be used to control animations, background music, or which UI elements are being displayed. 

As an aside, in my personal project, I use a vaguely similar approach that makes use of a static enum and events for game modes so that various managers and UI elements can act or react based on the state of the game. The clarity of being in exactly one state has greatly simplified my code and made it far less error prone.

So this is all well and good, but you may be able to see the end result of this approach is still messy if the number of states starts to get large.

A Step Further => More Abstract + More Manageable

Having all the states switching inside the Update function can quickly get unmanagable and if nothing else is pretty hard to look at.

So let’s go one step further. This step adds another layer or two of abstraction to the system, still has some downsides but does clean up our code and will make larger FSMs easier to manage.

This step will place each state in its own (mostly) self contained class. The state will be in control of itself and will determine when to make a transition as well as determine which state to transition to.

Step 1 is to create a new interface. In my case I’ll call the interface INPCState. The interface will have just one function that will take in all the NPC data and will return a value of the type INPCState. This returned state will be the state for the next frame as the “Do Action” function will be called by the base class of the NPC which in turn will run the behavior for a given state.

The Wander State with all the functions included

Step 2 is to create a new class for each state of the FSM and that state needs to implement the interface INPCState. Each of these classes should also contain any functions or logic needed for that state. In my case I have choosen to leave all the variables on the NPC base class. This keeps each state class tidy, but does mean that the state class will be changing values on the NPC base class which is usually something I like to avoid - you win some and you lose some.

Inside each state’s “Do Action” function we need to call the relevant functions AND we need to decide which state to use for the next frame. You can see the full code to the left, but for now let’s jump back to the NPC base class which will help make sense of the return values.

Step 3 is to refactor the NPC base class. Here we need a new variable of the type INPCState that will hold a reference to the current state. We also need instances of each state - it would be possible to make this work with static classes and using an abstract class rather than an interface, but if you know what that means you can probably implement it yourself.

The simplified NPC Base Class

In the OnEnable function I’ve set the current state to the wander state to ensure that the current state variable is never null.

Then the real magic happens in the Update function. Here we have just one line! So awesome. Here we call the “do action” function on the current state and then set the current state to the value returned from that state.

That makes total sense, I swear it does, but it’s abstract.

Give a mintue to let it sink in.

This is so clean.

Step 4 is to add conditionals to the state classes to determine which state to be in for the next frame. Notice that we have passed in a reference to the NPC base class into the “Do Action” function and from there we can get a reference to the instances of the state classes.

Pro’s

The base class is exceptionally clean. You can now see how the class appears to change it’s state (behavior). Effectively the entire Update function changes depending on the current state and what in that state’s “Do Action” function. That’s cool.

If we want to add an additional state we simply create a new class, add an instance to the NPC base class and finally add conditions to allow a transition to the new state from other states.

As long as the number of states stays fairly small 5 or maybe 10 this is a pretty manageable solution.

Con’s

It’s hard to maintain if the system truly gets complex. In this case most sophicated structures such as behavior trees might be needed.

All the variables in the NPC base class are public. Which is a bit messy and not super awesome. This could be partially solved by creating a data container that itself is private in the NPC base class but has public variables… This passes the buck a bit, but would prevent those variable from being changed by classes without access to the data container. A generic data container would also potentially open the doors a bit wider for reuse of the states by other NPC types - so worth thinking about.

As mentioned before, the states are changing values of variables on the NPC base class. Not ideal and something I usually go out of my way to avoid if possible.