The familiar Game Over screen pops up, but then, just below, something different. “Please report the level of difficulty.” You only lasted a few seconds in this hair-raising chase, so, with the adrenaline still pumping, hit ‘hard’. Next time, the thing chasing your every step will be a little less aggressive.
Artificial intelligence joins the battle!
Romain Trachel and Alexander Peyrotmachine learning specialists at Eidos-Sherbrookedemonstrated the game I just described at Unreal Fest 2022. Combine machine learning with a feature of Unreal Engine called Environment Query System (EQS), which allows developers to use spatial data to inform AI decisions.
Typically, this is managed via behavior trees that layer variables and branching possibilities. But in this demo, the AI’s behavior is driven by a machine learning model. Unreal EQS acts as the AI’s eyes and ears, providing insight into its environment, while the machine learning model becomes its brain and decides how it should respond.
The game isn’t as scary as I made it sound, mostly due to its top-down presentation and placeholder graphics, but its gameplay is a classic cat-and-mouse chase that tasks players with collecting orbs scattered across a map. It is Pac-Manbasically, but the ghost’s behaviors are no longer scripted.
“So, for example, if a developer decides to turn on a stronger tracking mode, the only thing to do is raise a reference value in EQS tests,” Trachel and Peyrot say in an email. “It really has the potential to streamline the development workflow, because in real game productions, it would be up to a game designer to decide which game variables need to be adjusted to change the difficulty.”
The key phrase in this explanation is “up to a game designer”. A traditional behavior tree can become unwieldy, requiring back-and-forth between designers, programmers, and other developers to fine-tune the behavior. Modifying a machine learning model might be an easier option, giving designers a way to model difficulty without diving into the branches of a behavior tree. Able to put that aside, designers might be better able to focus on what’s important: whether AI makes the game more challenging and more fun.
A better boss isn’t always a smarter boss
Machine learning candies be used to create a brutal enemy. Deep Blue from IBM and DeepMind Alpha Star from Google they proved it. However, this is not always desirable, not only because it increases the difficulty, but also because specific AI tactics can be counter to enjoyable gameplay.
Trachel and Peyrot tried using AI for several game modes, including a “multi-output model” that learned to predict the player’s score (earned by collecting orbs) and interrupt it. “But in this game mode, the enemy tended to camp on the orb locations. It wasn’t fun and engaging to play against them, so we didn’t show these results.”
Camping at the orbs’ location is a solid strategy – the player must collect orbs to win (imagine if Pac-Man’s ghosts just lingered near the entrances to each corner of the map). It also makes the game less fun. Players no longer experience a thrilling chase. Instead, the AI could launch an unpredictable ambush. Trachel and Peyrot say their goal isn’t “to create superhuman robots, which wouldn’t be fun and engaging for a novice player, but rather to find ways to incorporate machine learning into the game’s AI tools already used in production.” .
It might seem boring for gamers who want better AI. Yet the machine learning techniques demonstrated by Trachel and Peyrot remain useful for adjusting difficulty even when the enemies players face in the finished game don’t use them. Julian Togeliusco-founder and research director at Model ai, spent nearly five years using AI to test games. Modl.ai uses bots to hunt down graphical glitches, find flaws in the geometry of the world, and sniff out situations that make it impossible to win.
“Can you tell us what kind of error status you are interested in. And then it actually works. You submit a job and it runs depending on how much you want to explore,” Togelius says.
Modl.ai’s test bots use machine learning to adapt to each game tested, although its current implementation limits those adaptations to each specific title. Togelius says the company is prototyping the addition of deep learning that will train bot behavior across more games. Once in use, Modl.ai’s bots will learn to emulate the behavior of real players, which should more efficiently uncover problems players may find.
For true machine learning, game engines need a revolution
When it comes to difficulty, then, machine learning can be both a problem and a solution. But creating a fair and fun challenge isn’t the only obstacle facing developers who want to use machine learning in games. The problems run deeper—so profound, in fact, that they may force a rethink of how games are built.
Performance is a barrier. Machine learning requires a lot of training data for useful results, and that data can only be acquired by playing a game thousands or tens of thousands of times (although robots can lighten the load, a tactic used by Trachel and Peyrot in building their demo). And once the training data is collected, the resulting model can become cumbersome to run in real time.
“Yes, performance is clearly an issue, particularly with large ML models that process frames for every beat of the game clock,” Trachel and Peyrot said in an email. “In our case, to avoid performance issues, we used a small neural network that only deduced at specific points in the game.” Scaling up to the massive open world environments modern gamers have come to expect is another matter altogether.
Togelius says the way modern game engines work exacerbates the problem. Machine learning, he says, “will necessarily be slow because game engines aren’t designed for it. One of the many reasons we don’t see more interesting modern AI in games is because Unreal and Unity and all their ilk are fundamentally terrible – anti-AI in so many ways.
Animation is another issue. Most modern game engines expect animations to be strictly frame-by-frame. This works well when the animators know for sure how the characters in the game will behave, but an AI controlled by machine learning could behave in ways the animators didn’t expect. Designers can work around this problem with a physics-based approach to animationbut this strains the performance of a game console or computer hardware and comes with its own development challenges.
In short, the developers are up against a monster of their own making. Game engines are built to use behavior trees and prescribed actions to create worlds of AI-controlled NPCs that run well even on poor hardware. But as machine learning takes hold, these classic solutions will need to be reconsidered.
“If you go and talk to a machine learning researcher who doesn’t know game design, they’ll say, ‘Why don’t you use new things and get NPCs that are more realistic and fit how you play?’ so on,” says Togelius. “But you can’t just plug it into an existing game.
Leave a Comment