Artificial Intelligence has been applied tomany facets of the game design and development process. Though this has led to many advances in games and AI research, there remain few examples of games in which play centers around engagement with AI processes: the design space of AI-based games remains underexplored. By examining a breadth of playful experiences through different lenses, it is determined that games which forefront AI are beneficial for players, designers, and for the field of game scholarship itself. Moreover, there is evidence that symbolic approaches (rather than statistical) lend themselves to experiences with more agency, greater human interpretability, and more controlled authorability.
Artificial I ntelligence i s u sed i n m any a spects o f t he game design and development process and is also a central part of gameplay. While there is much research in AI and games, there are aspects of game AI that are still not well understood and are critical to our understanding of games, gameplay, and design. Recently, the range of AI techniques used for game AI research clusters around machine learning and statistical approaches, and while this lead to considerable advancements in areas such as player modeling, procedural content generation, natural language interfaces, and other areas, the use of AI as a core part of gameplay is not frequently addressed in depth.
Games that make AI available for the player to
interact with and also understand were have referred to as
AIbased games (Eladhari et al. 2011)
This paper aims to promote AI-based game design research by laying out a roadmap to the techniques and challenges within the space. We do so by first identifying where exactly AI-based game design exists within the space of AI research and application in game development. We make this step not to discourage work in the other areas of the space but to identify and emphasize that the space of AI-based game design is under-explored. In the following section, we describe several aspects of games—such as a game’s agency and intepretability—in which AI influences the experience of both authors and players and how different approaches to AI influence them. The two general approaches we focus on are the broad areas of statistical and symbolic AI. We make these distinctions because we believe that symbolic approaches will promote good design, and we intend to push gently against the current focus on statistical methods. In the last section, we soften the dichotomy and present a comprehensive view of hybrid approaches that acknowledges the strengths and weaknesses of both approaches. When taken as a whole, these contributions present a vision for future research into AI-based game design.
As much as the authors might like to refer to “AI in Games”
and leave it at that, that phrase inadequately captures the
diverse stages of game development and the varied ways
artificial intelligence is applied to them. To focus the conversation
of this paper, the authors first present a novel view on
artificial intelligence in games: two axes that form a graph
featuring the placement of different aspects of game design and
game development. This is an extension of previous writing
on AI-based game design
To understand this model, we must first understand the axes. One axis is the designer-centric Surface / Background axis. The other axis is the player-centric Mental Model axis. Upon this model, we place elements of the game development process that have had (or could have) artificial intelligence techniques applied to them. Note that any given individual game could occupy multiple places on this graph if AI was used in multiple aspects of its development.
The Surface / Background Axis captures the notion of where in the artificial intelligence is applied to the game. If we think of game development as a pipeline, there are many stages to the process. Some of these stages include the initial brainstorming and ideation of pre-production, development of the game, marketing the game, and post-production upkeep and content drops (e.g., new DLC and other expansions in the currently popular “games as a service” model, maintaining servers for multiplayer components, etc.). Techniques that are considered “high-surface, low-background” are incorporated into the actual act of play itself and directly affect or influence the player experience while playing. Crucially, “high surface” techniques are also capable of being influenced by a player’s play as well–thus, they can influence and be influenced in turn. Application areas considered “high-background, low-surface” are peripheral to the act of play but are indeed no less critical to the process of game development.
To illustrate this, let us look at the game Left 4 Dead
The first axis roughly corresponds to applications of artificial intelligence on different parts of the game development production process. The Mental Model Axis shows how different forms of artificial intelligence might be present in the player’s mind. We refer to this as the mental model axis to speak to how much the player’s experience of play is affected—and, more specifically, rewarded—by devoting the energy required to construct a mental model of how the underlying AI systems of the gameplay experience work.
The presence of an AI system in a playable experience is
insufficient to guarantee that a player will be incentivized to
develop this mental model. Wardrip-Fruin’s Sim City Effect
A visualization of these two axes is featured in Figure 1.
The upper-left corner describes applications of artificial
intelligence that directly affect the gameplay experience but
require minimal mental modeling by the player. Many
procedural content generation systems, in which the generator
is run once and then left to explore, fall under this
heading. Examples include Minecraft and other mining games
(Kreminski and Wardrip-Fruin 2018), the map generation,
procedurally generated sound, and many other applications
The lower-left corner describes applications of artificial
intelligence that do not affect the player, nor do they directly
influence gameplay. Uses of AI in this quadrant are
concerned mainly with the business of game development. As
previously referenced, gathering user data for player
retention purposes and monetization would fall into this quadrant
The lower-right quadrant contains application areas of
artificial intelligence that could potentially reward the astute
player but are designed not to be interactive. One
example of this is the Matchmaking system that exists in many
games. Systems of this type are a means to an end; they
ferry the player to the next match. However, even an
innocuous system can lead players to intentionally throw matches
to artificially lower their matchmaking ranking, thus
enabling them to face the strings of easier opponents.
Similarly, MOBAs and other team-based games have had to find
solutions to draft dodging. Telemetry and analysis of user
play traces have been conducted to help inform the
development of matchmaking services
The upper-right quadrant is the quadrant that the authors
call for more people to explore; the type of game previously
discussed as AI-based games. These are aspects of
gameplay informed or enabled by artificial intelligence that
players interact with directly, and for which the experience
depends upon the player developing a mental model of these
systems through play. Game experiences such as The Sims
(Maxis 2008), Dwarf Fortress
The following section presents a comparative approach to different AI techniques and how the same ’underlying’ experience might be radically different based on the AI technique employed to enable it.
The overall goal of this paper is to advocate for and facilitate the creation of games that use AI within the surfacelevel/requires a mental model quadrant described above (i.e., AI-based games). Such games involve players directly engaging with the AI as part of their aesthetic experience, and this section lays out several ways that the use of AI influences that experience. Throughout the section, we compare two broad categories of AI: statistical and symbolic. We realize that these categories can be blurred (the next section goes into more detail), but we found it useful to consider each approach at this level. Overall, this section is meant to push against the current in AI research and explain why we believe there is ample opportunity to create novel and satisfying games by employing symbolic AI in AI-based games.
While both creating and playing games with targeted
representations or experiences, the ability to interpret a game’s
behavior is an important factor. Game designers have framed
this as the ability to learn the rules of the game system
(Koster 2004), creating Meaningful Play
In games that use AI that does not require mental models in order to intentionally interact with, interpretability is less important. For example, while understanding how the landscape of a planet was generated in No Man’s Sky (Games 2016) is arguably not particularly relevant, understanding the connection between a Sim’s (Maxis 2008) ”needs meter” and their action is relevant.
Statistical Approaches Interpretability is a wellrecognized issue in machine learning (Gunning 2017). Limited interpretability is inherent in most statistical methods as the content of a learned model is not represented in human-interpretable ways. However, statistical methods often still produce results that adhere to expectations and are very powerful for many applications in which the output is more important than process (e.g., texture generation, speech to text, and computer vision).
However, this obliqueness can be an issue for AI-based games that prioritize the player’s ability to build a mental model of the AI’s operation. From a player’s perspective, the interaction loop (listen, think, speak (Crawford 2003)) is a process of learning. Every output of a system gives the player material to build an understanding of how it operates. When a player concludes the operation of a system that primarily relies on statistical methods, those conclusions necessarily cannot map to the actual operation of the system, as the operation of the system is not internally represented in human-understandable terms.
That is not to say that players of these games can not build mental models that are useful for attaining desired ends. However, such mental models are limited in that their primary ability is to anticipate how a system will behave, but not why. Though from a player-centric perspective, this will only be a problem if the output of the system significantly violates the player’s model without providing enough evidence to update the model or descriptions of why different behavior occurred (which, typically referred to as explainable AI (Gunning 2017), is one of the core challenges of statistical methods).
Symbolic Approaches Since symbols are the words used to refer to representational entities, symbolic approaches are highly interpretable as humans can ascribe meaning to them. When a theorem prover makes a proof, the step-by-step process is available for humans to inspect. Likewise, when a system enacts a plan for an agent, the system’s rationale is available. While the “reasoning” of symbolic systems is not always the most obvious to a human, they are valid and justified. This mechanical or “alien” manipulation of symbols might be considered aesthetically good or bad depending on the game, though it can negatively affect the believability of agent behavior.
Often, symbolic systems make use of sophisticated architectures to select content or choose agent behavior. Symbolic AI-based games involve the player acquiring a rough model of these architectures to achieve their desired ends. For example, in The Sims 4 (Maxis 2008), the player can associate a Sim’s need for hygiene meters with the degree of hygiene a Sim has because to a human, a low amount of “hygiene” is symbolically associated with not being clean. The system architecture and the authoring support this with its behavior in other game areas. Furthermore, there can be a strong relationship between authored content and what is presented to the player.
However, it should be emphasized that a human’s
interpretation of a game (and its symbols) and even their
perception of what symbols are available for interpretation can, and
likely will, be different than what the system uses or the
author intends. In order for a game to symbolically represent
an author’s intent, a game needs to be carefully designed
and delivered to a receptive audience
Most often, game designers strive to give the players of their games the experience of high agency. Generally, having a sense of agency involves the feeling of one having the power to take specific action in a situation toward a desirable end. In games, it is theorized that agency is achieved when the formal and material affordances of a system are in balance (Mateas 2001). In other words, when the game leads a player to consider doing something (the formal), it can also respond to it (through the material). In many games, these criteria are not met, and games do not provide high agency experiences. This is most prevalent in narrative games. For example, the story and cutscenes in a game like Grand Theft Auto V (Games 2013) present detailed characters with personalities and things to say, but in gameplay, characters mostly serve as targets and ragdolls and do not respond to the player’s actions or the resulting dynamics.
Much work has been done in academic game and autonomous agent research to address this, and it is a difficult technical problem. However, it should be noted that agency can also be achieved through design choices. For example, lower fidelity character content (graphics, dialogue, etc.) arguably suggests to a player that the game is not meant to respond to everything that you would expect a human-like agent to respond to. Genre convention also informs what players will expect and thus their sense of agency. The degree to which a game can be said to provide agency results from a combination of technical, design, and cultural factors. Statistical Approaches A purely statistical approach to physical behavior in a game would inherently present a problem for agency, as part of what a player expects from a game is tied to their experience of reality (object permanence, consistent laws of physics, etc.). However, other areas of behavior are perceived as less static, and statistical methods can significantly alleviate authorial burden and provide surprising and interesting experiences. For example, learned models can drive animation in unanticipated environments, and dialogue can be generated dynamically.
However, for AI-based games, the player is intended to understand how and why the system performs as it does, and statistical methods are black boxes that the player has little access to understanding. Because agency is tied so closely to what a player perceives and expects, this is another area where a system’s processes are more important than its output, and because statistical methods are generally not interpretable, they can impede how much agency a game can provide.
Symbolic Approaches On the other hand, more symbolically oriented systems provide a structure that players can discover. Symbolic systems arguably have more consistent “hooks” that a player can ascribe meaning to and extrapolate about the system’s overall operation.
As an example, consider the 2013 version of SimCity
(Maxis 2013). In order to grow their city as desired, the
player must be roughly cognizant of the operation of the
underlying Glassbox system
Symbolically represented systems contain a theory of what they represent and are prescriptive, rather than just descriptive. Experientially, this lends symbolic systems a degree of legitimacy when the system does not behave as a player might expect. Without this, a player experiences a sense of helplessness and distrust that their actions or experience are meaningful to gameplay (i.e., a loss of agency).
Whether a game aims to immerse the player in a fictional environment or engage players through strategic problemsolving scenarios, it is crucial that players accurately learn about its operation is consistent. Common consistent features might include object permanence and gravity in representational worlds, and in abstract games, the features might be the game’s rules or win conditions. This is not to say that players must always have perfect information about how the system operates, that is seldom the case, but very often, good game design involves players feeling like they are learning about how to play the game better as they play (i.e., learning how the system operates). When a game presents players with information that invalidates player beliefs too often without convincing justifications, players can begin to feel that the game is choosing behavior randomly and ultimately feel that their input does not matter.
Statistical Approaches AI Dungeon 2 is a text-based
game that has the player control an avatar by typing any
command into a prompt
Symbolic Approaches Unlike statistical methods, symbolic approaches can explicitly track context and consistently react to it while in operation. This allows for the system to enforce hard preconditions and to better ensure that appropriate content is presented. A downside of this is that the higher-order context needs to be supported through architecture and often requires more authored content. In other games, the system’s reaction to player choice is critical, and invalidating a player’s mental model can ruin the experience. AI Dungeon 2 is an exception to most interactive fiction games, as most interactive fiction games make use of heavily symbolic systems such as Inform 7 (Nelson 2006). Inform 7 roughly represents the fictional worlds as objects, containers, and actions that can be performed upon them. State change and descriptions of actions are chosen based on a rule system. As a result, games based on Inform 7 are deeply strategic and puzzle-like.
An AI system’s authorability depends on how well it supports the creation or addition of content. As the design and development of other video games, authoring is an act of creation that players ultimately judge subjectively. It is also subject to hard constraints imposed by the game’s systems and by soft constraints from existing and future content. Additionally, authors have a limited ability to create even when obeying those constraints. Not only do authors have to work with these constraints while being judged by players, creating content for AI-based games requires interfacing with complex technologies.
To create, authors need a channel to communicate design
decisions with the AI system. Meaningful communication
to the AI system is central to these aspects, varies widely,
and includes text, reactive planners for character behavior
(Mateas and Stern 2002), bespoke tools (Isla 2005), data
sets or even mixed-initiative tools
We will focus on asset creation and narrative design to explore how statistical and symbolic approaches intersect with authorability. Each of these areas brings a different aspect of authorability into perspective. Asset creation, such as generating textures or geometry, focuses on ease of creation, amplifying authorial power, and reducing authorial burden. Narrative design requires contextual appropriateness and creating and modifying complex artifacts. Statistical Approaches With plenty of examples and the algorithms explored by the computer vision community, statistical methods are well-suited to the needs of generating visual (Li and Wand 2016) and audio assets (McDonagh et al. 2018). On the other hand, authoring stories or social behavior relies heavily on context, temporarily disconnected information, and consistency as judged by the player. Even the current best natural language generation models and dialogue systems have difficulty keeping consistent context or answering questions that require small amounts of reasoning.
Symbolic Approaches Asset authoring with symbolic approaches works well with tasks that can be abstracted into portions and annotated with descriptors like puzzle design (Smith et al. 2012) or procedural music composition (Brown 2012) but perform poorly in generating textures and sound files. The contextual nature of, and through lines required for, social (McCoy et al. 2014) and story games (Reed et al. 2014) plays to the strengths of symbolic approaches, and its application to this space has a rich history.
Aside from the comparisons made in the previous section,
there are many intersections and differences to be explored,
each with its impacts on game AI. One with the most
impact on games is how the AI system encapsulates design. In
data-driven approaches, the act of creating and employing
the system is wholly design-free, and the design knowledge
is implicit within a trained model or policy. Other systems
are dependent on design either in their construction
Environmental impact and reproducibility are
problematic for both approaches to varying degrees. Deepmind’s
AlphaStar Final serves as an example for both because it
required an enormous amount of power to train (though a
large portion of that energy was offset renewably
There are many other challenges ready for future analysis, including using learning as a hammer when not every problem is a nail; impacts of AI-based game design on the knowledge level (Newell 1982); and the function of hierarchy in reasoning and learning on game design.
As argued above, symbolic methods in AI-based games help
achieve meaningful play, trust, and other important factors
that involve the player interacting with the AI processes
themselves. Such methods provide the interpretative
affordances needed to build actionable mental models and are
conducive to high agency experiences. Statistical methods
tend to be successful when AI is used to produce content
where the process involved to create it is not central to the
player experience and authoring. When statistical methods
are used as part of the gameplay loop, such as with
generating dialogue, there is a risk that the system will present
1Estimated training coast of approximately $13,000,000 based
on Google Cloud prices taken from 21 May 2020 from
https://cloud.google.com/tpu/pricing and 44 days each for training
12 agents on 32 3rd generation TPUs
That said, these two approaches need not be
considered entirely separate. Connections between symbolic and
statistical systems (i.e., hybrid intelligent systems) have
taken forms like fuzzy logic
While evocative, comprehensive solutions to game AI may not be possible. Each subproblem has its domain and representation needs that may not generalize or be legible by other problems. Because they all could have this quality, designers should deeply consider their approaches in each subproblem and use the technique that is the best fit for that space.
We believe that games that place AI in the forefront by requiring the player to build a mental model of their operation are beneficial as they expand our conception of what games can be and are meaningful cultural artifacts in themselves. This paper was written to promote this AI-based game design research and to lay out a roadmap to the techniques and challenges within the space. We specifically compared and contrasted statistical and symbolic approaches to using AI in this space and concluded that symbolic approaches are more conducive to player understanding in AI-based games.
Making games, especially with AI, is a complicated process with many technical challenges. While symbolic approaches may assist with mental model building, the design considerations addressed above can indeed be broken into subproblems where statistical problems can be beneficial. Furthermore, hybrid systems show great promise.
Aamodt, A., and Nyga˚rd, M. 1995. Different roles and mutual dependencies of data, information, and knowledge — an AI perspective on their integration. Data & Knowledge Engineering 16(3).
Adams, T., and Adams, Z. 2006. Slaves to Armok: God of Blood Chapter II: Dwarf Fortress.
Andersen, K. A., and Hooker, J. N. 1994. Bayesian logic. Decision Support Systems 11(2):191–210.
Bach, S. H.; Broecheler, M.; Huang, B.; and Getoor, L. 2017. Hinge-loss markov random fields and probabilistic soft logic. The Journal of Machine Learning Research 18(1):3846–3912.
Booth, M. 2009. The ai systems of left 4 dead. In Artificial Intelligence and Interactive Digital Entertainment Conference at Stanford, 2009.
Brown, D. L. 2012. Mezzo: An adaptive, real-time composition program for game soundtracks. In Eighth Artificial Intelligence and Interactive Digital Entertainment Conference, FDG ’12.
Carter, M., and Gibbs, M. R. 2013. esports in eve online: Skullduggery, fair play and acceptability in an unbounded competition. FDG 2013(May):47–54.
Crawford, C. 2003. On Game Design. New Riders Publishing.
Eladhari, M. P.; Sullivan, A.; Smith, G.; and McCoy, J. 2011. Ai-based game design: Enabling new playable experiences. UC Santa Cruz Baskin School of Engineering, Santa Cruz, CA.
Gagne´, A. R.; El-Nasr, M. S.; and Shaw, C. D. 2011. A deeper look at the use of telemetry for analysis of player behavior in rts games. In International Conference on Entertainment Computing, 247–257. Springer.
Games, R. 2013. Grand theft auto v [video game]. Edinburgh, United Kingdom: Rockstar North.
Games, H. 2016. No man’s sky [video game]. Guildford, United Kingdom: Hello Games.
Gunning, D. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web 2.
Isla, D. 2005. Handling complexity in the halo 2. Game Developers Conference.
Koster, R. 2004. A Theory of Fun. O’Reilly Media, Inc. Kreminski, M., and Wardrip-Fruin, N. 2018. Gardening games: an alternative philosophy of pcg in games. In PCG Workshop.
Langley, P. 2006. Cognitive architectures and general intelligent systems. AI Magazine 27(2):33–33. Number: 2. Li, C., and Wand, M. 2016. Precomputed real-time texture synthesis with markovian generative adversarial networks. In Computer Vision – ECCV 2016, Lecture Notes in Computer Science, 702–716. Springer International Publishing. Lin, C.-T., and Lee, C. S. G. 1991. Neural-network-based fuzzy logic control and decision system. IEEE Transactions on Computers 40(12):1320–1336.
Mateas, M., and Stern, A. 2002. A behavior language for story-based believable agents. IEEE Intelligent Systems 17(4):39–47.
Mateas, M. 2001. A preliminary poetics for interactive drama and games. Digital Creativity 12:140–152. Mateas, M. 2003. Expressive ai: Games and artificial intelligence. In DiGRA Conference.
Maxis. 2008. Sims 4 [video game]. Redwood Shores, CA: Electronic Arts.
Maxis. 2013. Simcity [video game]. Redwood Shores, CA: Electronic Arts.
McCoy, J.; Treanor, M.; Samuel, B.; Reed, A. A.; Mateas, M.; and Wardrip-Fruin, N. 2014. Social story worlds with comme il faut. Computational Intelligence and AI in Games, IEEE Transactions on 6(2):97–112.
McDonagh, A.; Lemley, J.; Cassidy, R.; and Corcoran, P. 2018. Synthesizing game audio using deep neural networks. In 2018 IEEE Games, Entertainment, Media Conference (GEM), 1–9.
Nelson, G. 2006. Inform 7.
Newell, A. 1982. The knowledge level. Artificial Intelligence 18:81–132.
Nguyen, T.-H. D.; Chen, Z.; and El-Nasr, M. S. 2015. Analytics-based ai techniques for a better gaming experience. Game AI Pro 2:481–500.
Perez-Liebana, D.; Samothrakis, S.; Togelius, J.; Schaul, T.; and Lucas, S. M. 2016. General video game AI: Competition, challenges and opportunities. In Thirtieth AAAI Conference on Artificial Intelligence.