AI Agent Schooled by Ubisoft to Navigate Vehicles in Racing Simulation
Ubisoft, the renowned video game developer, has made significant strides in the field of AI applications in video games. The French company's research team, Ubisoft La Forge, has published a groundbreaking paper detailing an AI algorithm capable of carrying out predictable actions within a commercial video game.
The paper, which focuses on an AI algorithm designed to work under the specific constraints faced by game developers, adapts the Soft Actor-Critic model for both discrete and continuous actions. This model, originally developed by researchers at the University of California, Berkeley, is known for its ability to create a model that can effectively generalize to new conditions and is much more sample-efficient than most models.
Ubisoft's use of this advanced reinforcement learning technique helps train AI agents that balance adaptability and robustness in dynamic game scenarios, leading to more effective simulation of player actions and interactions with non-player characters (NPCs). This hybrid approach allows AI bots to better learn game mechanics and player strategies without exhaustive manual scripting, improving the realism and reliability of automated testing.
The team tested the AI algorithms on three different games, including two soccer games, a simple platformer-style game, and a driving video game. While the algorithms performed slightly worse than the state-of-the-art industry results on the tested games, they achieved much better results in a test where the AI agent had to follow a given path and negotiate obstacles in an environment it hadn't witnessed during training.
The practical usefulness of Ubisoft's algorithm for the video game industry was demonstrated by these results. The average game developer typically does not have access to the computational resources required for creating AI systems like those by OpenAI and DeepMind. Ubisoft's training approach can potentially work for a wide variety of possible interaction approaches, including instances where the AI agent has the exact same input options as the player.
This research reflects a broader industry trend where AI and reinforcement learning are transforming game production workflows by automating complex, context-rich tasks while maintaining high-quality player experiences. Ubisoft’s success showcases how hybrid reinforcement learning methods like Soft Actor-Critic can be practically applied in commercial game development to meet the growing demands of increasingly complex and immersive gaming worlds.
References:
[1] Ubisoft La Forge. (2021). Automated Game Testing with Reinforcement Learning. arXiv preprint arXiv:2102.09723.
[4] Ubisoft La Forge. (2021). Practical Reinforcement Learning for Game Development. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI).
Artificial intelligence, particularly the Soft Actor-Critic model, has been adapted by Ubisoft for use in sports games like soccer, as well as racing games, demonstrating the versatility of this technology in various video game genres. This application could potentially streamline the automated testing process for game developers, who often lack the computational resources necessary for advanced AI systems.
