These articles were written whilst employed with NetworthPick, the website and social media have been deleted, so I have preserved my articles here.
From the birth of gaming, playing against computer-controlled opposition has been a core part of most gameplay loops. Over the years we have seen the technology behind it get more and more complex, offering different types of AI (Artificial Intelligence) within their games. With the exploding growth of ML (Machine Learning), can some of these work alongside other technologies.
When looking towards the future of these applications, it’s important to understand the synergy between these technologies. In the world of academia and in commercial development, AI and gaming have been in a symbiotic relationship driving forward each other’s development further than I’m sure anyone initially believed possible.
Past & Present Examples
Static Actions & Donkey Kong
The history of AI in Video Games has been vast, with many different implementations being used to suit the time. One of the earliest examples of a game using AI to improve the player experience is with Donkey Kong. This early arcade game adopted a system using static actions and dynamic breaks based on those actions. Whilst seeming quite basic behaviour in comparison to some of the AI today, in its time it was key in building the responsive gameplay which spurred on the development of games as we know it.
The way this technology works is through having predetermined actions (called static actions) which contain simple sets of instructions for the behaviour of the computer. The way the game decides which action to perform is based on the actions taken by the player. This sense of feedback from your actions is a driving factor behind the enjoyment of such games.
Finite State Machines & Pacman
Another famous arcade game which left a cultural mark on our society was Pacman, this beloved arcade game is one of the best early examples of a type of AI technology called Finite State Machines (FSM). In short, an FSM is where the game gets organised into a series of sets of instructions called ‘States’. The AI is driven into these different states based on the player’s action or a trigger.
In Pacman, this was implemented through the use of two main states, Neutral or Escape. In the neutral state all ghosts in the game will chase Pacman through the level trying to catch him. One of the ghosts chases him by pathing directly to Pacman’s location, whilst another does it’s pathing to the tile ahead of where Pacman is. The third ghost combines the pathing actions of the first two. And finally, the fourth ghost has a slightly different behaviour in the fact that it will path normally but actually retreat if you get too close. (Don’t take my word for it, load up a game of Pacman and see for yourself!).
In the game, the ghosts will continue with this neutral state chasing Pacman until they are triggered into another state. In this game the trigger is the Power Pill, once Pacman eats this, suddenly all the ghosts get moved into the Escape state where they all follow the same set of instructions to try to escape and avoid the player. This is a great example of showing how the implementation of simple systems can lead to complex behaviour.
The use of FSM’s were not limited to the early arcade games however. Stealth games from the last 10 years like the Arkham Asylum series or Splinter Cell are an example. In these games the enemies follow their default pathfinding up until the point they are triggered by a player’s action or presence to force a change in behaviour, where they instead start attacking you.
The Machine Learning Boom
My own personal area of interest when it comes to AI is in the area of Machine Learning (ML). My interest began about 6-7 years ago when sites like YouTube and LinkedIn were sharing loads of examples of ML being put to work on older and classic games to show how they could be beaten. This method of using an actual game to implement these types of AI technologies allows you to be able to conceptualize what is going on when you can see it in game.
In recent years we have witnessed monumental growth in this sector as many people start sharing the different applications in which Machine Learning can be applied. This has led to quite a stark mainstreaming of this technology now, with much of the industry believing ML solutions are the future to the way we build AI in games.
What is Machine Learning and how does it apply to Games?
In its simplest form, Machine Learning is about using algorithms and statistical models to make the entity or machine act without needing to program it in full. The technology of machine learning works alongside the technology of Neural Networks.
The new gold rush when it comes to new IT technologies is that of Data. For any ML application to work well, a vast, accurate dataset is needed. This data is then used to train the statistical model and to test it to make sure the desired behaviors are produced. In the next section we will be going over the different implementations of Machine Learning and how they have been used within games.
Whilst earlier developments in the AI space was to create the experience for the player, a large majority of ML applications are being used to play the game themselves, in a way trying to ‘Solve’ the game using the power of Machine Learning.
Q-Learning – Super Mario
Q-Learning is a fascinating sub-section of Machine Learning which specialises in creating solutions without the need to build the model around it. This allows this algorithm to be applied to many different scenarios and it will perform. This area is under the banner of what’s called Reinforcement Learning. These types of ML share the benefit of not being reliant on a large dataset, making the bar to entry in learning and processing power accessible for most users.
Reinforcement Learning consists of three key components: an environment, a positive reward, and a negative reward. If you are able to define these components for your task, then this type of solution could be for you!
The most common example of this type of technology being used in games is through the game Super Mario. By giving the model the ability to press any input in the game; assigning a positive reward for progressing further along the level; and a negative reward for taking damage you can quite quickly develop an AI of learning how to play the game and complete levels. The Youtuber SethBling popularised this method in a video of his.
From this, many classic games have had Q-learning or other Reinforcement Learning algorithms applied in an attempt to play and beat them, to much success. The reason for choosing these old games was due to the limited amount of inputs so complexity remains quite low.
Deep Learning – The Future?
So apart from being in the game and being able to play the games, what else can we expect to see AI do in the future of gaming?
Image processing has always been a large part of the use of ML models in the past, recently Nvidia released their AI powered upscaling onto their RTX cards through a driver update. This technology is called Deep Learning Dynamic Super Resolution, or DLDSR. This technology works by rendering your game at a higher resolution than your monitor/card can support, then downscaling it to your own resolution for a clearer image. Whilst this method was possible in the past through other means, through utilising Deep Learning technology it is able to train itself and adapt based on the data gathered through users using it, so performance will always be increasing as improvements are made.
Another interesting avenue in the future of AI in gaming will be in the generation of content. One of the more intuitive areas of AI is through the use of Generative Adversarial Networks (GAN’s).
In short these models produce new, unique content based on characteristics of the training dataset. You may have seen examples of this on social media with the ‘I made an AI read the entire script to X and got it to write a scene’. When developing content for games, especially in the current market of procedurally generated content within the games we play, a GAN can be used to aid the developers by being able to produce large amounts of visual or text based content to adapt or work with in development.
Conclusion
When you are playing games it’s easy to forget the amount of research and development which has been undertaken to bring the systems you enjoy to fruition. This underappreciation however can be seen as a testament to how well these AI systems worked within the games, to seamlessly engage the player by giving them the dynamic experience we now all associate with games. As we’ve seen in this article all the different ways in which AI technology has been used to interact with games and to even help develop games.
If you are curious about a more technical look into how these technologies work, keep a look out for future articles on the topic!