Effective cooperation between humans and technologies powered by Artificial Intelligence (AI) is decisive to fully exploit AI's economic and social potentials. However, the adoption of AI is often opposed by a lack of humans' trust in AI systems and a dearth of interest in working with them. Turning to games for getting inspiration on how to optimize human-AI cooperation seems promising, since games engage humans almost effortlessly in interacting and cooperating with artificial non-player characters (NPCs). However, a structured overview on how game design can optimize human-AI cooperation is missing in existing gamification research. Therefore, this paper presents a systematic review of NPC design patterns and elaborates on what developers of AI systems can learn from game design. Guided by a thematic analysis, we present a structured overview of relevant design patterns clustered along six focus fieldsnamely I) NPC responsiveness, (II) appearance of NPCs, (III) NPC communication patterns, (IV) emotional aspects, (V) behavioral characteristics, and (VI) player-NPC and NPC-NPC team structureswhich advance our understanding of designing and investigating cooperation between humans and NPCs. The insights of this paper can guide practitioners and future research regarding the design of more effective AI systems, the gamification of human-AI cooperation, and the development of innovative NPC approaches.
With the rise of Artificial Intelligence (AI) and increasingly autonomous machines, human-AI cooperation has received a surge in attention in industry and academia. In areas as diverse as human-robot interaction, autonomous driving, or the assistance of humans in complex decisionmaking with expert systems, seamless cooperation between humans and AI technologies is decisive to enable society and businesses to fully exploit AI's benefits and potentials. The growing interest is reflected by a rise of research papers elaborating on this topic. Despite this interest, we lack a clear understanding of how specific design aspects of AI systems can optimize the human-AI cooperation and establish trust between humans and AI systems [1].
One context where cooperation between humans and AI appears to emerge effortlessly is video games. Existing research demonstrated that specific game design features could engage players in developing strong emotional relationships [2] with non-player characters (NPCs), support the perceived closeness, and even build trust. Design knowledge and patterns from game design and, in particular, NPC design can thus provide a hitherto scarcely explored treasure of knowledge for designing more effective human-AI collaboration outside of games. Lending elements from video games and utilizing them in other contexts, such as AI systems, has become popular in recent years. This trend is called gamification and refers to the use of design principles and features of games outside traditional video game environments with the intention to afford similar experiences as in games and to influence behaviors [3].
While various studies indicate, that game design knowledge can help improve the design of AI systems, the bulk of the gamification research that has emerged over the past ten years missed to provide a structured overview of gamifying human-AI cooperation [4]. Therefore, this study aims at answering the research question:
This paper's major contribution is conducting a systematic literature review and thematic synthesis as well as investigating which patterns game developers and designers exploit for building rich social interactions between NPCs and humans (i.e., player characters (PCs)). Our results are based on performing axial and selective coding to derive subcategories and linkages between the codes and summarize the current body of knowledge in a systematic way. Finally, we offer practical recommendations as to what AI software developers and experts in Humancomputer interaction can learn from the gaming industry.
The term NPC refers to any character found in a game not controlled by the players [5]. In many games, players play with or against NPCs. NPCs are used to increase the believability of games and a player's immersion in the virtual game world [6][7][8]. Human players are keen to interact with realistic NPCs and research indicates that players can even establish strong relationships with NPCs [19].
In the last decades, game developers and designers have placed a primary focus on increasing NPC believability [11] and creating the illusion of playing with human-like fellows. NPCs traditionally follow a deterministic AI behavior and players can compete or cooperate with NPCs; however, humans can quickly become frustrated with NPCs that show deviating, non-human-like or predictable behaviors [12][13][14][15][16]. Recently emerging developments in the field of advanced AI pave the way towards more realistic NPCs and thus more immersive gameplay [17,18].
Even though NPCs are prevalent in games and interest in developing more robust NPCs [23] is high, game research missed studying NPC design patterns in greater detail [24]. One recent work investigates central design components of companions in video games [25], expanding a design space proposed in [26]. While these contributions are relevant for this paper, companions only resemble one category within the broader class of NPCs. Evidently, there is a gap of systematic review papers that deal with design patterns of NPCs. Current literature in NPC design remains fragmented and little is known on how to transfer the insights gained from NPCs to other non-game contexts. Different studies indicate, however, that game design knowledge could optimize future human-AI cooperation and improve AI systems [32,35]. Gamification research has overlooked to provide structured knowledge on the gamification of human-AI cooperation thus far [4].
In this paper, we present a systematic literature review on the topic guided by Webster and Watson [27]. The literature review has been conducted on the Scopus database. The choice of scientific database is justified by two reasons: First, Scopus aggregates several relevant databases such as ACM, IEEE, or Springer. Second, the focus on one single scientific research database allows a replicable process and thus supports the rigor and objectivity of the procedure [28].
We performed the literature search on August 26 th in 2021, querying the Scopus database in the following manner: TITLE-ABS-KEY(NON-PLAYER CHARACTERS AND DESIGN*). The search yielded results focusing on non-player characters and any permutation of the term design. By carefully limiting the search to the metadata, this approach enabled us to scan literature only for publications concentrating on our intended search terms. The search resulted in 295 hits. Next, we performed several screening steps based on the following criteria to include only relevant papers: 1) Removal of duplicates and false hits (-22 papers); 2) Abstract and title screening and subsequent removal of papers with a focus not in line with the research question at hand (-77 papers); 3) Removal of papers not written in English (-4 papers); 4) Removal of papers that are not full papers (-17 papers) and 5) Papers that cannot be acquired (-1 paper). This screening process resulted in 174 full papers. Then, we coded the works by accumulating information on bibliometric and descriptive information. Subsequently, we applied thematic synthesis according to [29]. This approach was chosen as it allows to investigate phenomena in qualitative data, such as prototype descriptions, and aims at generating implications for practice. This is in line with our goal to encourage designers to draw inspiration from NPC design for improved human-AI cooperation. The synthesis comprises three stages:
Free coding: A sample of ten articles was read and reviewed. Inductive line-by-line coding led to the identification of multiple design features. Based on the number of papers in the dataset, we applied an additional second round of open coding with five articles. Next we added codes to the fragments. The plausibility of this preliminary coding scheme was checked by carefully reading all papers in the dataset. As a result, six additional codes were added to prevent neglecting relevant design pattern subcategories.
Construction of descriptive themes: The obtained codes were iteratively compared. The findings were synthesized and similarities as well as differences between the obtained codes, were identified. Descriptive themes were generated through axial coding.
Development of analytical themes: We reviewed the entire body of knowledge and mapped the content on the defined themes. All data was classified along with the following overarching themes: design patterns on (I) NPC responsiveness, (II) appearance of NPCs, (III) NPC communication patterns, (IV) emotional aspects, (V) initiative of NPCs, and (VI) PC-NPC and NPC-NPC team structures. These analytical themes comprised several subcategories and thus resulted in a tree structure.
In terms of the presentation of the results, we follow Paré's assessment [30] and present the synthesized evidence mainly in tabular form.
Out of the 174 reviewed full papers, 116 are empirical studies. 72 papers contain empirical results related to NPC design patterns and human-AI cooperation. 26 papers are conceptual or present frameworks, methodologies, or models. 22 papers are reviews, while 14 studies introduce preliminary results, describe systems, case studies or prototypes. 113 papers fall into the domain entertainment. The second largest category is education with 46 papers. Seven papers belong to the domain of culture/history/ethics, four papers deal with medicine and health, and two papers belong to the domain engineering. The domains sports and tourism each comprise one paper.
The structured review of previous research on NPC design reveals that the majority of the empirical studies employ design patterns related to the responsiveness of NPCs. In this field of research, the most popular design features can be clustered in features related to how NPCs provide feedback and are able to learn and respond. The design patterns in these categories aim directly at facilitating more effective cooperation between NPCs and human players.
A total of 38 studies investigates or employs NPC feedback mechanisms. Feedback has been shown to be powerful in influencing people's decision making [31] and bringing about behavior change. The review indicates that NPC feedback can further be divided into four thematic groups (as visible in Table 1): Direct feedback (e.g., [32]), delayed feedback (e.g., [33]), NPC-PC cocreation (e.g., [34,35]), and persuasion of the player (e.g., [31,36]).
In appropriate contexts, specific NPC feedback seems to be able to serve as a stimulator of curiosity or even an augmenter of human creativity. For instance, Ali et al. [31] demonstrate that NPCs designed as artistic playmates providing creative feedback can increase kids' creativity compared to similar playmates, which provide less creative feedback. This type of feedback is shown to significantly increase the participant's creativity and consequently improves the quality of the human-AI cooperation.
29 studies indicate that especially three patterns related to feedback are highly relevant for achieving effective PC-NPC cooperation: assessment of player's performance/progress (15 studies), immediate feedback (11 studies) and unpredictability (13 studies). Embedding socioemotional elements and unexpected moral questioning prompts can help augment NPC believability and the level of player immersion, as called for in [12,13]. The insights of [42] demonstrate that design features allowing players to observe an NPC's vulnerability and experience its decision-making process first-hand can trigger reflection on the player's side and increase the emotional investment in the game. This can be achieved through perspective switching exercises that serve to confront a player with several daily social dilemmas (such as stealing in a shop, being bullied by peers) that NPCs face and make him assess the NPC's decision-making. Moreover, this review indicates that the NPC's ability to learn and respond is crucial for enhancing both the level of game immersion [51] and the interestingness [71] of the PC-NPC interaction. 38 papers are dealing with this theme, as depicted in Table 2. This category can be divided into three subcategories, differentiating between humans learning from AI/NPC (through social comparisons, switching perspectives, or triggering emotions), NPCs learning based on human gameplay (for example learning by demonstration, utilizing external hardware), and NPCs learning from fellow NPCs/AI (within or outside the current domain).
The generated overview reveals that while a large diversity of approaches exists, several focus fields can be identified (cf. Table 2). For example, only three papers [68,72,73] deal with inter-and cross domain learning where NPCs can learn from fellow NPCs. This is probably because this approach is quite new and complex to realize. The results of [72] and [73] indicate, however, that minimizing the NPC's learning and training times can lead to faster acceptance and the player can exploit the NPC's skill sooner and interact more naturally. The empirical findings demonstrate the benefits of this approach compared to scenarios where a new NPC in a game cannot lend skills from a fellow NPC and needs to be trained based on human performance from scratch. For instance, inter-domain learning allows sharing of knowledge as well as prior experiences among fellow NPCs within one domain. The second subcategory is cross-domain learning. Studies [68,73] demonstrate that this design pattern can facilitate the human-AI cooperation in the long term because it allows NPCs to transfer their skills from one domain onto new fields and enable NPCs with more general capabilities. This can positively influence trust of humans in NPC by simulating human learning processes and creating a sense of likeness in terms of cognitive capabilities. Inter-domain learning is particularly important in games that feature importing NPCs from one game to another. For instance, agents that were trained on how to ride a bike could explicitly utilize that knowledge for riding a motorcycle in a new context. However, the bulk of the empirical studies apply features that allow humans to learn from an NPC or vice versa. In the empirical papers, NPCs training has been achieved through e.g., the usage of learning by demonstration [35], optimization algorithms (such as Reinforcement Learning [81][82][83][84]), or Supervised Learning (e.g., Artificial Neural Networks [78,79]) approaches.
Several studies apply design patterns that enable NPCs to learn from human gameplay. These approaches serve to create profiles of human players and train for imitation (used in 11 papers), exploiting shared memories (used in 6 papers), or utilizing external hardware (such as EEG-based BCI devices [48], webcams or Kinect systems [45], used in 4 papers) to capture movements and emotions in real time.
The review emphasizes that certain design approaches are particularly suitable for supporting humans to learn from their NPC counterparts. In 7 studies, this is accomplished through either social comparison, confronting human players with NPC decision-making, or deliberately controlling the pace of the learning process (e.g., through inobtrusive buttons to deliberately call NPCs for help [69]). Three features, however, are especially prominent in the reviewed studies: perspective switching (7 studies), deliberately triggering emotions (6 studies), and monitoring and adapting difficulty levels (7 studies).
NPCs learn based on human gameplay
The review highlights the importance of responsiveness for facilitating effective NPC-PC cooperation, which was reflected in the amount of coverage across the studies. Nonetheless, this review identifies five further categories with design features that can improve the human-AI interaction.
Appearance comprises features related to anthropomorphism, such as human likeness, customization, tone of voice, facial expression, and embodiment (cf. [38,44]). The patterns of this category can play a vital role in the cooperation because the player's perception of the NPC highly affects the team dynamics [2].
Moreover, several different communication patterns are found: a) the applied modalities (such as text-based, natural language or BCI), b) verbal/non-verbal communication enriched by gestures, body language, levels of assertiveness, and c) the direction of communication (e.g., PC-NPC, NPC-NPC, PC-PC, see [31] and [56]). The results indicate that lively conversations with references to real-world experiences [17] and situations are more effective in terms of engagement and player enjoyment than noninteractive, pre-programmed NPC conversations.
The category emotional aspects comprises patterns related to empathy, the power of narrative and backstories, embedding motivational elements such as points, scores and leaderboards, humor/satire, and love [6,26,70]. This is visible, for instance, in the study of Mallon and Lynch [62] that recommends integrating elements of romantic relationships with NPCs to add an additional dimension of human experience and creating more intriguing PC-NPC partnerships.
Further, our data show that the NPC's degree of autonomy and personality traits are relevant design patterns we summarize as behavioral characteristics. These contain the degree of involvement of an NPC in the PC's game experience and an NPC's own agenda (cf. e.g., [50,52]). Creating unique NPCs and controlling when and how they intervene are demonstrated to be promising ways to increase the player's curiosity and facilitate immersion [31,44].
Lastly, the category PC-NPC and NPC-NPC team structures captures features related to the team dynamics and the role of each actor in sociotechnical systems. Our results indicate that NPCs that possess knowledge about previous incidents and preferences of the player can more easily create a personalized game atmosphere. Through design patterns that allow NPCs to build memories, a collection of relevant shared experiences with the player are created. By taking advantage of this wealth of shared experiences and proactively suggesting actions based on previous player preferences, the NPC comes across as a non-static and adaptable counterpart [72]. This, in turn, serves to strengthen and mature the relationship with the player [56]. Additionally, taking turns with the human can create a more captivating experience since the NPC's reactions appear more natural and may remind the player of human-human conversations. This pattern is especially useful in dialogues [38] or when elaborating choices at decision points [44].
This study investigates design patterns of NPCs that facilitate cooperation between NPCs and human players in existing research. This adds to previous research in the field of companion design [25,26] through a broader consideration of this relevant phenomenon. The study's main contribution is an explorative elaborated novel overview of categories and design patterns that advance our understanding of how specific design features facilitate human-AI cooperation.
This research illustrates that reaching a high level of NPC believability is a difficult mission. It involves elements such as goals, proper reaction abilities, non-verbal communication [9], emotion and social-emotional cognition [10], dynamic dialogues [20], adapting to the player [21], and the quest for more meaningful interaction [22]. The study discovers that several clusters exist, such as feedback mechanisms that aim to influence player behavior or approaches of mutual learning.
To the best of our knowledge, this research is the first work to holistically investigate NPC learning processes in video games. The systematic screening of the existing body of knowledge reveals that learning can occur on several levels: a) NPCs being either directly responsible for it by triggering emotions or allowing for perspectivetaking, stimulating, or teaching humans, b) NPCs learning based on human behaviors and gameplay, c) inter-domain and cross-domain learning with NPCs learning from fellow bots.
Our study adds to previous research in several ways: Firstly, this study can offer new pathways for developing more compelling NPC characters in games and serious games. We recommend that designers actively embed NPC feedback elements, including direct/delayed feedback or NPC-PC co-creation. These features are shown to be powerful in influencing people's decisionmaking and behaviors [31]. Consequently, game designers should diversify and enrich their NPC-PC interactions through timely feedback, emotional-triggering elements, and by increasing the unpredictability through unforeseen actions and plot twists.
Secondly, the results reveal novel approaches to human-AI cooperation and can offer practical guidance for software developers of AI-based solutions. For instance, the review identified that certain aspects of NPC design have already been implemented in human-robot interaction with positive outcomes (cf. [31]). Furthermore, the presented design patterns can guide the design of future AI systems outside games. For instance, designers of AI systems could implement aspects of perspective switching with an AI system, as shown to be promising in NPC design by [51]. Further, design patterns such as the active design of perceivable vulnerable AI, reinforcing players' learning processes through deliberately triggering emotions, or actively confronting users with the reasoning behind an AI's decision-making could guide future AI design for supporting human-AI cooperation.
Thirdly, we found that NPC design increasingly employs various patterns related to an AI's learning from the player behavior. This trend is illustrated through 23 empirical papers in which NPC learning is triggered by human gameplay. The corresponding approaches can also be very valuable in gamification design. They could guide future research on further personalization of gamification which is required to prevent a one-size-fits-all approach [86]. Applying NPC learning approaches in gamification may support personalized need satisfaction and increase the effectiveness of gamification for various target groups.
Further, our results reveal several shortcomings in the current body of knowledge that could guide further research in this field:
1. Future studies should empirically investigate the effects of single design patterns. The isolated consideration is important to assess the applicability as well as the actual effectiveness of the identified patterns.
2. Also, gamification research has largely overlooked applying NPC designs in nongame contexts [87]. Future research should develop empirically evaluated frameworks that can guide scientists and practitioners in further leveraging the potentials of NPC design outside games.