Unlocking New Gaming Experiences with AI Agents

In this article we’re talking about AI Agents and how they’re changing the gaming landscape. 

Recent advancements in AI are starting to offer more immersive, adaptive and engaging experiences that we’ve already seen demos of from the likes of InWorld [1] and nVidia [2] (among others). The exciting part of the advancements is AI Agents, entities that can replicate human behavior and are capable of perceiving their environment, reasoning, learning and taking actions without human / player interaction. 

We’ll be covering what we’ve seen already with the advancements of AI Agents, the different experiences and genres that could benefit the most, things to consider and overcome in the space, how agents could work together with players on a massive scale and how on-chain Agents could crank things up a few more levels. Let’s dig in! 

Natural Language Processing (NLP) - AI can now understand more complex player commands and conversations in a natural manner enabling more dynamic and engaging dialogues enriching the story and lore of the games’ universe. In games that have come before this usually has been dialogue trees with a few set outcomes, sometimes even the same outcome with different dialogue giving the appearance of choice.

Emotional Intelligence - Understanding, tailoring and responding to player emotions is now also possible. By learning from players' responses and behaviors in game, AI Agents can be more empathetic and responsive towards players. If a player is being merciful or taking a pacifist path in their story missions, Agents can recognise this and respond more warmly and sympathetically to the player.

Machine Learning and Adaption - AI Agents can learn from game data of the recent sessions, player data and global game stats across all players and matches. Agents can take all this data and create a far more personalised response and experience to players.

Real-Time Decision Making - Tied to the above, AI Agents could analyse data on a more granular, session-based level. More on this in a future post.

Greater branching narrative choices - More complex, branching narrative choices can be created leading to deeper story arcs and increased replayability, especially with different characters and progression tree choices.

Emergent, Dynamic Storytelling - Based on player actions and choices, AI can generate different outcomes that can create vastly different outcomes for every player. If a player were to save an NPC or show mercy to them, they could become a mainstay character going forward and or take on a more meaningful role within the game. 

Infinite Planets - AI could generate thousands of planets or worlds that feature diverse, rich and unique ecosystems providing players with vast universes and worlds to discover and explore. These could be different for every player or be generated on a seasonal content basis.

Adaptive Difficulty - We’ve had dynamic difficulty in games for decades, AI can analyse many more data types and variables to truly create a tailored experience that can be a consistent challenge, allowing the player to feel a greater sense of accomplishment once they complete an encounter, story arc, mission or the entire game. 

Competitive Play - Taking some of the same variables and data points mentioned above in Real Time Decision Making, AI Agents can be adaptive and challenging opponents. They can take player behavior and weapon / item usage to come up with different attacks to push players into different tactics and play styles. Players can get more enjoyment out of their experience with challenging AI opponents, with the use of adaptive difficulty to really feel a sense of mastery and accomplishment at the end of a session.

Cooperative Play - We’ve seen some skillful use of AI as cooperative partners for players in games from God of War to Last of Us. The upcoming GTA 6 looks to be a co-op duo experience, how much overlap the two story arcs has is something I’m looking forward to seeing. With AI being able to use the models mentioned in Human Behavior section, future AI cooperative experiences can get closer to teaming up with friends.

This area is still very much a work in progress with many different, strong opinions of how much AI can genuinely help game development. General AI help and AI Agents differ with Agents being a more powerful version of general AI that could take on specific roles.

I believe AI can help and assist in production but will still require game development experts to craft the overall experience. Areas where AI can help:

Generated Level Design - Initial whitebox or procedural generation of environments based on specific criteria.

Art and Animation - Using existing models and assets to generate more and varied content such e.g. building textures or character animations.

Music Generation - AI can assist in audio creation by generating music based on play and action happening on screen in conjunction with developer direction. There are a few early versions of this technology such as Suno [3].

Automated Testing - Various levels of Automated testing has existed for a number of years, AI can make these tests far more thorough and cater for different play types, styles and more scenarios than possible today.

Simulation Games - In games such as city builders and life simulators, AI Agents can simulate across many complex systems creating a much more realistic, believable and responsive experience. 

RPGs (Role Playing Games) - NPCs can become much more dynamic with branching, emotionally responsive, emergent narratives based on player actions and behaviors creating a richer, more immersive experience.

FPS (First Person Shooters) - AI Agents will be able to behave far more like real players making enemies and teammates far more strategic and variable. Enemy AI could coordinate attacks against the player in real time whereas AI teammates can provide strategic backup based on player objectives. I still remember being really impressed with the original Halo AI [4], these were well scripted encounters in conjunction with AI behaviors. Can’t wait to being wowed again by AI Agent enemies.

Open World and Sandbox Games - These genre of games become immersive when we see lifelike behaviors from other characters, wildlife or world simulation. AI Agents can increase this immersion by behaving more naturally and organically mimicking real world behavior rather than following set patterns programmed by the developers.

Strategy Games - Players can experience more challenging and tailored experiences with AI Agents adapting to player behaviors and actions, either increasing the variety and types of conflict or strategies deployed against the player. 

Multiplayer Online Games - AI Agents can take on a more administrative, moderator role dynamically managing resources or economy balancing and rewards. As discussed above, they can also become challenging enemies or teammates, especially useful at times of lower player counts.

Sports Simulation Games - AI Agents can take the role of teammates and control players on the pitch, managers, coaches or trainers providing additional, tailored challenges to the player based on performance or their chosen objectives. For example, if a player is competing for the world cup, a manager AI Agent could prioritise and recommend players from the home country and those that would promote the best team synergy. 

Adventure and Puzzle Games - Players could get much more help and hints based on their play performance. I thoroughly enjoyed playing through Super Mario Bros Wonder which featured Talking Flowers [5] that provided tips and tricks at certain sections of the game. Imagine them being much more dynamic based on number of lives lost in a given section, what friends and other players have done to complete a certain section or which badges result in the most success.

Imagine a game where thousands of intelligent AI Agents were playing alongside each other and other players. We could deliver experiences not seen before in games:

Emergent Gameplay - AI Agents taking on a variety of roles in an MMO / Sandbox game where they populate the world working in given professions, living their lives, interacting with each other, players, creating conflict or alliances etc. All these behaviors would create some routine and many emergent interactions that could provide near endless replayability even with the same characters or missions. It would create unique experiences with players' experience and outcomes differing from each other as the game / world states would rarely be the same.

Increasingly Complex Battle Simulations - In strategy games troops, units, or squadrons could strategise and make decisions independently with minimal or no input from the player. They could assess the battlefield in real time and make strategic choices based on their personalities, skills and risk tolerance. Seeing these battles play out in real time or replaying streams of matches after the fact would create for some fascinating viewing and commentary. 

Increased Immersive Storytelling - Already discussed above under Richer Narrative Experience, and tied to Emergent Gameplay, thousands of AI Agents would increase the scope of what’s possible 10x. Each character or faction can have their own history and lore that directly ties to their alliances, beliefs, personality and actions in the game; these can be shaped based on player actions and how they interact with them. 

Having tens of thousands of AI Agents in a game world is going to unlock new gameplay experiences but it’s not without its challenges. Some considerations:

Performance and Scalability - The game and engine have to support the number of AI Agents running simultaneously, especially in online scenarios where players could interact with them across the entire map, world or galaxy. Players want increased immersion and real-time experiences, they don’t want to wait for AI Agents responses or have them be non-responsive. This would break the experiences with players opting for games without AI Agents, turning them off if the option exists, or avoiding them altogether.

Syncronisation and Consistency - AI Agents need to be aware of one another with states and resolutions shared in real time. The magic of AI Agents will be when they are all aware of states across the game and can react and act accordingly. For example, if the player attacks a faction or saves someone from a near-death experience, the ripples should be felt by those who saw it firsthand, those within earshot, Agents telling faction members or people hearing it on the news. Each of these needs to be syncronised with varying levels of believability and trust based on how an Agent heard the news and how much they trust the source.

Data Management and Privacy - Player privacy and data management is a large area that will need to be addressed and may see legal intervention if not handled correctly. What type of data is OK to be stored? Who has access to it? How much is exposed back into the game? How do the AI Agents use the data available to them? What type of data history is stored, can we see AI Agent interaction history? Where is the data secured and can it be viewed or deleted at a player's request? All important topics that need to be addressed at an industry level.

Multimodality - Today's AI Agents [6, 7, 8] are limited by their inability to visually or acoustically perceive their environments, relying instead on text-based semantic inputs to interpret world events. While this design simplifies implementation, it severely restricts the agents' capacity for spatial reasoning and richer contextual understanding. It is inevitable that AI Agents will eventually evolve to "see" and "hear" their surroundings, mimicking human sensory capabilities. 

Transparent and verifiable - With every decision and action recorded on-chain a transparent and immutable record is maintained. This can be verified and checked for increased security and protection against cheating, fraud or players being able to confirm that opponents are real.

Trust and Reliability - Agents and their code living on-chain allows anyone to verify it capabilities and trust factor. Agents could have ratings or trust scores based on past performance providing players valuable data for how much to trust said Agent and with what resources or decision making power.

Autonomous - Complex rules and actions can live in smart contracts as part of AI Agents where they can act on them at will without player input, even when the player is offline or when developers have sunset the game. To quote ludens [9] 'Autonomous Worlds have hard diegetic boundaries, formalised introduction rules, and no need for privileged individuals to keep the World alive.’ [10].

Economic Incentives - With agents able to act autonomously, economic incentives and rewards can be built into their smart contracts to encourage them to achieve their objectives which could be player-led. AI Agents that belong to players could also be traded on a marketplace, this could be especially lucrative with verifiable trust scores as detailed above.

The future for AI in games has never advanced at such a rate as the last couple of years and we’re still in the early days of AI Agents. 

There are many, many developers and companies leading the technological advancement of AI in games with plenty of obstacles to overcome. The most rewarding things often come with the toughest challenges, and I can’t wait to see these challenges addressed and where AI Agents progress.

[1] https://inworld.ai/blog/nvidia-inworld-ai-demo-on-device-capabilities 

[2] https://www.youtube.com/watch?v=psrXGPh80UM 

[3] https://suno.com/

[4] https://www.youtube.com/watch?v=kda7rz5qFtI 

[5] https://www.gamedeveloper.com/design/the-talking-flowers-in-super-mario-bros-wonder-became-nintendo-s-very-own-mr-motivator 

[6] https://github.com/joonspk-research/generative_agents

[7] https://voyager.minedojo.org/

[8] https://github.com/altera-al/project-sid

[9] https://x.com/l_udens 

[10] https://0xparc.org/blog/autonomous-worlds

__________________

About Sparsity - Sparsity is a multi-chain acceleration layer for real-time transactions. By 10x transaction speeds, Sparsity enables highly interactive decentralized applications and on-chain games to deliver a frictionless and secure user experience. Games built with Sparsity will be live on major L1&L2s.

Founded by technical and product experts from MIT and top-tier gaming companies, Sparsity brings decades of experience in network research and game design to push the boundaries of blockchain performance and bridge the experience gap for mass adoption.

About the author - Charnjit Bansi, a design and product leader who served at some of the worlds most prestigious web2 and web3 studios across the globe from Polygon, Yuga Labs, Mythical Games, Activision and EA. All my releases have either been critically or commercially acclaimed with my work having sold over 180 million units (and counting!) generating over $5b in revenue. 

Cover image courtesy of Justin Zhang.