=Paper=
{{Paper
|id=Vol-2480/GHItaly19_paper_05
|storemode=property
|title=Towards a Model to meet Players' Preferences in Games
|pdfUrl=https://ceur-ws.org/Vol-2480/GHItaly19_paper_05.pdf
|volume=Vol-2480
|authors=Mattia Bellini
|dblpUrl=https://dblp.org/rec/conf/chitaly/Bellini19
}}
==Towards a Model to meet Players' Preferences in Games
==
Towards a model to meet players’ preferences in games Mattia Bellini University of Milan Milan, Italy bellinitia@gmail.com using a Recurrent Neural Network [6]) and music (e.g. ABSTRACT through «a probabilistic model based on distribution Different have been the attempts to use Procedural Content estimators conditioned on a recurrent neural network» [1]). Generation via Machine Learning in game development. Some of the techniques used in such studies have been Among the others, some researchers have tried to adapt a applied in other domains, by knowledge transfer. The game, or some part of it, to the user playing it. This target1 domains of these transfers included game approach has been called “adaptive game design”. development. More precisely, PCGML has been applied to Contrarily to what it may seem, apparently the most game contents rather than games themselves, or, better interesting findings in this field have been made for drama told, on level design rather than game design. Researchers managers, i.e. for the artificial intelligences that generally tried to automatically generate contents procedurally generate story flow. The paper takes the move constituting levels, like maps, while the attempts to from what seems to be a missing in current literature and it automatically generate or to adapt entire game is aimed at proposing and discussing a possible procedural environments have been less frequent. This is a direct content generation via machine learning model that takes reflection of the limits of present ML: the scarcity of the latest approaches in machine learning applied to drama available data for full game generation and the difficulty of managers and combine them with findings from adaptive creating a model able to generate an entire game from game design. The objective of the proposed model is to give scratch. players the best possible gaming experience of a highly branched game, depending on their attitudes towards the As Summerville et al. [18] outline in their work, researchers gaming world. of PCGML have applied different machine learning approaches in game studies, including artificial neural networks, Markov models, clustering and matrix KEYWORDS factorization. Most of the works focused on the autonomous Video games; Storytelling for video games; Procedural generation of levels, particularly for platformer games like Content Generation via Machine Learning; PCGML; Super Mario Bros. [11] (e.g. [18], [8]). There had been Adaptive game design; Drama manager. attempts to generate also contents different from mere game level, like Magic: The Gathering [24] cards [17] or stories for interactive fictions [7]. Some of the most INTRODUCTION interesting approaches in the field applied ML to drama Procedural Content Generation via Machine Learning managers (DM), to procedurally generate stories following (PCGML) is a new paradigm for the self-driven creation of players’ behaviour. Similar studies have been made in the new content. The main difference with the mere procedural field of adaptive game design, e.g. to balance game difficulty generation is the generally higher quality of the created to player’s abilities. Following the idea of applying ML for content, achieved by integrating the procedural content a recognition of a player’s attitudes, the focus of this paper generation (PCG) algorithm with a machine learning (ML) is to discuss whether is possible an application of PCGML model trained on existing content. for the creation of a player-aware model capable of PCGML has been applied to a variety of different content predicting user preferences and serving an adapted level types and, by the time this paper is being written, it progression, to maximize appreciation. performed well particularly for the creation of images (e.g. GHItaly19: 3rd Workshop on Games-Human Interaction, September 23rd, 2019, Padova (Italy) Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). ADAPTIVE GAME DESIGN changes happening in run-time, just like the Mario Kart Wii Using Procedural Content Generation, some researchers [12] AI example seen above. tried to adapt games to the player. Traditionally, these In the survey made by [10], however, an important lack in works were interested in modifying difficulty settings, research emerges: the procedural generation of quests has using different techniques. Examples include the mechanics been studied only from the point of view of placing goals of different published games, like the aim assistant in Max and “keys” to reach them. Even though that is a relatively Payne [13] that is more precise the less the player’s abilities, out-to-date survey, being dated back in 2011, this lack or the opposing AI in Mario Kart Wii [12] that increases its seems to be still present. Indeed, at the best of my skills when the player is performing too well. These studies knowledge, no new impactful studies have been conducted are of little interest for the purposed of the model, since in this sense. On the contrary, many have been the works their aim is to adapt the game mechanics and difficulty but on improving the storytelling mechanisms on which DM in no way the game itself, that is the aim of this research. are based, as we have already mentioned. More interesting are semantic and declarative approaches, like the one found in Tutenel et al. [21]. Their model is based on a semantic definition of objects that includes all APPROACHES IN DRAMA MANAGERS game-relevant information of a particular game object. There have been several attempts in the field of DM to These include functional information, possible relations to identify and classify a player in order to give her the story other objects and metadata of the game content. After progression that best fits her tastes. In the survey made by having declared and assigned these data to the objects, it is Roberts and Isbell [14] we can find multiple examples, as possible to use them to better drive the new content also outlined by the more recent study conducted in [22]. creation process. Giving objects a semantic layer «helps For analytical purposes, the same four features presented in convey the meaning and the role of an object in the virtual this lastly mentioned paper will be used to describe models world, and consists of generic descriptions of classes of found in literature. The four features are: replayability features, including attributes, properties, roles, relations, (possibility to play again the game without receiving the etc. This encourages the incorporation of further semantic same gaming experience), authorial control (control over the information about player-dependent gameplay purposes, game design process left in the hands of the author), player and how these can be used to control object generation» autonomy (freeness of the player in the gaming experience) [10]. and adaptability (capability of modifying the game to meet player’s tastes). Later in the paper, we will also address the [10] points out also that a model that aims at providing a problem presented by a fifth feature, namely the better game experience to the players by adapting the game coordination, i.e. the ability to orchestrate Non-Player to their play-time behaviour, needs a firm knowledge over Characters (NPCs) and other game elements to present what a player expects to play, to feel and in general to find specific experiences to the player. in a product. This means, basically, three needings: Researches on DM present the closest approaches towards 1. Have a solid player model and a way to capture the model that this paper is aimed to propose. In particular, player’s expectation; the approach of the PaSSAGE system [20] provides a good 2. Quantify the expectation to a measurable level; degree of adaptability through the identification of pre- 3. Process them and adapt the game consequently. defined players’ styles. By assigning at each event of the These three steps are of essential need for the creation of a game a weight for each style, the model chooses the most model really capable of adapting contents to the player. attractive event at every stage, ensuring autonomy to the Charles et al. [5] supports the idea of shaping a player player. To be noted is that this approach is deterministic, i.e. model to capture her interests and playstyle, but also points there is no degree of randomness. Thus, replayability is a fourth need: the necessity of monitoring the player to virtually zero, since the same actions will always result in constantly check for the effectiveness of a generated the same reactions in the interactive fiction. This is also the solution. reason for the high degree of authorial control of this model. A last useful distinction made by [10] is the one between off-line customized generation and on-line adaptivity: the Very interesting is also the approach that emerges in former is intended as a generation of contents while the Implementation and Analysis of a Non-Deterministic Drama game is not running, typically during the loading of a Manager [22]. The aim of the authors here is to serve the gaming session; the latter, on the contrary, describes the best possible match between emergent player attitude towards the story and story progression, to provide a high way to maximize player’s appreciation of a game by degree of adaptability. The job is entrusted to a DM built on adapting it at the levels of game design, story development a genetic algorithm that ensure non-deterministic results, and «the logical flow of events and actions that follow» [10]. thus replayability, and the possibility to deal with a large However, Yu and Riedl’s [26] decision to not rely on a pre- number of blocks (here quests). To match player’s defined PM presents us a huge knowledge gap between what preferences, the developers rely on a player model (PM) can be designed and what really players want. composed by a vector of three dimensions, representing the Notwithstanding the advantages of having self-built patterns attitudes of the player towards the three regiments that do not rely on any abstract theory, this approach is not theorized by Durand in The anthropological structures of the returning any clearly readable data on user preferences but imaginary. Quests are human-authored and described using just, indeed, opaque patterns. For this reason, it might be similar vectors. This way, it becomes possible a quite more profitable to rely on a well-established PM. For computation of the concatenation, to evaluate the distance the purpose of this model, I decided to base the PM on of the results of each possible sequence with player’s Stewart’s theory[16], grounded in turn on Bartle’s preferences. After a probabilistic tournament selection, the psychographic taxonomies [1]. Other theories might have tournament-winner concatenation is manipulated to been used, among the others Yee’s model [25], or Bartle’s minimalize the possibility of two identical results given the three dimensions model [2]. I decided to discard Yee’s same premises. The model grants player autonomy by because, with Bartle’s word [3], «if you want a theory for setting a minimum number of choices available to the […] studying player psychology, then you may be better player at each stage of the interactive fiction. served by a straight taxonomy […] such as Nick Yee’s The model presented in [22] is an extremely useful starting motivations». It should be clear that my intent is not to point for the present theory. However, in the field of DM, study the psychology of the players, but rather their the most interesting approach is the one showed in preferred game style. On the other hand, Bartle’s three Personalized Interactive Narratives via Sequential dimensions model [2] presents a too-broad categorization, Recommendation of Plot Points [26]. In the paper, the authors that becomes nearly impossible to handle in the present a collaborative filtering approach, similar to the development phase. Another possibility is to develop a ones used in recommendation systems for services like custom categorization of players, but this is going further Netflix and YouTube, applied to DM for the autonomous beyond the scope of the paper. I decided to base the model building of a story. Thanks to the collaborative filtering on Stewart’s expansion of Bartle’s taxonomies mainly for algorithm, the developers handed-off the complex problem two reasons: firstly, it provides a clear and simple of defining PMs to determine current users, as the different categorization of players that is well-grounded in literature categories of users were grouped by the algorithm itself. An and which validity has been tested different times. interesting advancement of this model is that it Secondly, describing the PM with four values guarantees an demonstrates how a progression-aware model has amount of information that is easily processable both by the impactful benefits in the recommendation of subsequent algorithm and, more importantly, by the developers. plot nodes. On the other hand, a limit of the approach is that However, the proposed model is not dependent on the way it is largely based on manual and explicit expression of of describing the players and it is possible and easy to positive and negative feedbacks via a review system, due to change the PM description if a better method is found. the non-pre-defined PM. The authors also implicitly Stewart in [16] describes the four taxonomies (Socializer, pointed out a good practice to retain authorial control over Killer, Explorer and Achiever) by binding them with story generation, i.e. the use of a branched scheme as a specific actions performable in a generic game. On the basis starting point from which to pick the blocks to be appended of this theory, a player of the hypothetical single-player at each stage. game based on the proposed model will be described by soft clusters, «in the sense that a particular player can have a degree of membership in each player type» [26], i.e. she will THE PROPOSED MODEL be characterized by a vector of four values ranging from 0 The approach presented in [26] points out an extremely to 1, that quantify her degree of membership to each of the promising case of knowledge transfer: they demonstrated four taxonomies. The vector will then describe the gaming that sequential recommendation, frequently used to suggest persona of the current player in a pretty accurate way. complete fictional artefacts, is eligible to be applied also to shape only parts of a product, to best fit the tastes of audience in almost real-time. This promises to be a smart The proposed model relies therefore on PM recognition to procedurally generate content. The content is customized on player’s profile and based on a PM built during an online (in-game) opaque survey: while normally playing, the algorithm registers player’s path, formed by each action she chooses to perform, and updates in run-time her profile accordingly. However, the proposed procedural generation is offline (pre-game), meaning that the algorithm will select a block to present to the player as next level during the loading screen (i.e. in the time moving from one level to the subsequent) and not during an active game session, mainly to avoid slow-downs. To decide which block to select, it will be used an approach similar to the one that can be found in [22]. The model will Figure 1 – Visual description of Stewart’s expansion of have a pool of developers-defined levels at each stage, in Bartle’s taxonomies [16] which are encapsulated n number of actions, having each a Using such a model would surely require some care during vector of traits corresponding to Bartle’s taxonomies. A level design, particularly for early-in-the-game levels. What fitness function evaluates the levels in the pool by would be needed are multi-solution problems to overcome. calculating the distance between available actions vectors This might include both enemies and riddle to solve, and player’s profile vector and return for each level a fitness obstacles, pathfinding moments, etc. The first step than is rate. The fitness rate is then downscaled to a percentage, to assign a value for each of the taxonomies to the different that is in turn used to probabilistically pick the level to show possible actions, i.e. to the different ways of progressing in to the player as game progression. It might be more efficient the events. Later, we can easily obtain a well-defined to evaluate each block in advance, attaching a vector of gaming persona of the player by adding those values to the properties to the levels and not to the single actions, in player’s profile. Indeed, to evaluate her disposition towards order to accelerate the concatenation process. This way, the a taxonomy it will suffice to register the actions she algorithm would only have to evaluate levels as a whole, performs and their semantic description, made using instead of each action separately. However, unfortunately, Stewart’s expanded taxonomies. For example: a player that this approach probably presents its drawbacks, too: even if during the game tends to speak with all the NPCs and to it is true that the algorithm could work faster, presenting solve problems in a “diplomatic way” might be labelled as a the next level a few milliseconds in advance, on the other Socializer, while a player who tends to attack whatever is hand the blocks may need to be specifically designed to in sight might be labelled as a Killer. Again, this is an please a particular part of the audience. This might lead, in example: as said, the gaming persona are not defined as a set turn, to a regression to an almost deterministic model, of mutually exclusive booleans, but rather as a set of values especially after the PM will have reached a good level of floating between 0 and 1. Indeed, a much more realistic precision. In addition, such an approach could mean more representation of the player would be formed using “floats restricted possibilities in designing actions due to the approach”. specificity of the preferences of the audience for that block. This could make level blocks even more specific. In turn, it Furthermore, to better represent a player’s attitude, it will could lead to an even more specifically designed level, be needed to weight different actions in a reasonable way. ending up with a system that, due to a “wrong” choice of An accurate weighting is necessary to not overbalance an the player in the early stage, could keep her in an unwanted action regarding the others and thus to obtain a valid path for the entire game. On the other hand, by evaluating gaming persona of the player. Taking back the previous each single action the designers can also include different example: if actions are not differently weighted, a player taxonomies in each level. This way it will be possible to not that kills an evil slaver would end up with the same “Killer present the player with actions belonging only to her rate” as one who murders an innocent just for fun. In preferred taxonomy, in order not to bore her with too addition, a single action might have a (positive or negative) similar tasks. To obtain the same result, if needed, it will weight in two or more taxonomies, thus actions, too, need also be possible to include a random factor during the pick to be described as vectors of weights. of the blocks. Picking from a set of author-made blocks, the algorithm will scalability, as it is possible to add or remove levels choose the one that would probabilistically provide the best at each point of the game without impacting on the possible experience for the player. Thus, for our previously game progression, since the algorithm picks the instantiated player booleanly labelled as Socializer, the best-fitting block in the provided pool. This being algorithm will more likely choose to concatenate levels with said, a nota bene is that the model does not evaluate the most “problems” solvable via socializing, while for the the coherence of the game progression, that has to Killer are more likely to be chosen the levels with the be addressed during the game design phase. This greatest number of possible enemies, and so on. approach is scalable also in the sense that it is possible to modify the PM description to best fit The advantages of the proposed approach are multiple and the needing of each game built on the model; can be summarized as follows: ▪ Single-person collaborative filtering: taking the ▪ Authorial control: using a defined pool of possible example of the collaborative filtering approach levels at each stage ensures a high degree of found in [26], the proposed model will rely on a authorial control over the result. With this “single-person collaborative filtering”. The model approach, each level is entirely created by authors: is based on a prediction of likeliness built on a the PCG applies only on the concatenation. It series of positive and “non-positive” feedbacks. generates the game, but not single levels. Authors The feedbacks are given by player’s choices of the can design the game and its story with a normal actions to perform: the chosen action is a positive branching tree, just as [26] suggested; feedback, while all the other discarded possibilities ▪ Player autonomy: the player keeps the autonomy are “non-positive” ones; she has in a general game, since there is no ▪ Data scarcity: the main issue of the approach found autonomy retention in the model itself. in [26] is the heavy reliance on human-provided Constraints might be decided in the phase of actual data quantity. For the model proposed in this development of game and levels; paper, data scarcity is not an issue, mainly for two ▪ Adaptability: the whole model is intended to have reasons: 1) the concatenation can be delayed until a good level of adaptability, given the constraints a certain amount of data over the player are of human capabilities to create levels. The collected, and 2) is virtually possible to design proposed approach is not a PCGML model aimed extremely dense levels that would give a relatively at the autonomous generation of an uncountable huge amount of information. number of games, or levels, of stories, but rather it This model finds its place between level design and game is aimed at the maximization of the appreciation of design. Taking player’s choices in levels as inputs, and a widely branched game. In addition, thanks to the outputting game design options through a recommendation evaluation of the single next step, players can system, might be a play-changing approach in PCGML change their attitude towards the game and its applied to games. However, this approach shows us a fictional world without being constraint in a challenge: on the one hand we would end up with a game narrow path, pre-determined by her early-in-the- that has the highest possible appreciation rate, due to the game choices; very fact that the game itself is shaped on the individual ▪ Replayability: due to the probabilistic player attitudes. On the other hand, however, designing the concatenation of levels, the model keeps a medium proceeding of such a game requires particular care, above level of replayability, since the concatenation is all for story progression. This is the main weak point of the not deterministic but, indeed, probabilistic; model: it needs an expertise in storytelling and game design ▪ Coordination: coordination in the model is to keep the consistency of the story. To address this addressed incidentally, since there is no direct problem is probably preferable to keep levels relatively control of the model on the behaviour of NPCs and small-scaled: keeping in mind the Aristotelian unities of other game elements. The coordination arises here time, place and action when designing levels might be a from the very fact that depending on the actions of good practice when this model is applied. Of course, the player, the concatenated levels will ideally be constraints to the level concatenation can be applied in built to present a reaction of the environment to order to prevent a certain level being shown after another player’s actions; one that has nothing to do with the previous story. This ▪ Scalability: the evaluation of vectors of the does not mean, obviously, that the player cannot occur in individual actions found in a level ensures major crossroads in the story. FINAL CONSIDERATIONS AND FUTURE STUDIES [3] R. A. Bartle. 2009. Understanding the Limits of Theory in Beyond Game Design: Nine Steps to Creating Better Videogames. The current paper was aimed at proposing a PCGML model Chris Bateman. Delmar. capable of adapting a game to players’ attitudes and [4] N. Boulanger-Lewandowski, Y. Bengio, and P. Vincent. 2012. preferences. The starting point has been different models Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and aimed at an adaptive game design. Notwithstanding some transcription, arXiv preprint arXiv:1206.6392. extremely useful best practices pointed out by such [5] D. Charles, A. Kerr, M. McNeill,M.McAlister, M. Black, J. Kücklich, A. Moore, and K. Stringer. 2005. Player-centred game researches, no models actually similar to the one I was design: Player modelling and adaptive digital games, in Proc. aiming to obtain have been found in this field. Instead, DiGRA Conf. Changing Views—Worlds in Play, Vancouver, BC, researches on DM seemed to be oriented more towards the Canada, pp. 285–298. [6] K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. direction my model was aimed at facing. Indeed, approaches Wierstra. 2015. Draw: A recurrent neural network for image that relied on a non-deterministic blocks concatenation [22] generation arXiv preprint arXiv:1502.04623. and on a collaborative filtering recommendation system [7] M. Guzdial, B. Harrison, B. Li, and M. O. Riedl. 2015. Crowdsourcing open interactive narrative in the 10th [26] have been extremely useful for the theorization of the International Conference on the Foundations of Digital Games, proposed model. This is based on the definition of a PM as Pacific Grove, CA. [8] R. Jain, A. Isaksen, C. Holmgård, and J. Togelius. 2016. a vector of four dimensions: each time the player performs Autoencoders for Level Generation, Repair, and Recognition in an action, the PM is updated accordingly. At each new level, ICCC. the algorithm probabilistically picks a subsequent block [9] B. Li. 2015. Learning knowledge to support domain- independent narrative intelligence, Ph.D. Dissertation, Georgia that is more likely to be interesting for the player, according Institute of Technology. to her current PM. A particular care is needed during the [10] R. Lopes, and R. Bidarra. 2011. Adaptivity Challenges in Games storytelling and game design phases, but every game that is and Simulations: A Survey in IEEE transactions on computational intelligence and AI in games, vol. 3, no. 2. not linear-paced needs expertise in game design and [11] Miyamoto, T. Tezuka. 1985. Super Mario Bros. storytelling and the little additional care needed here is a [12] Nintendo EAD. 2008. Mario Kart Wii. [13] Remedy Entertainment. 2001. Max Payne, Gathering of little mite to be paid for what the model promises to do. Developers. [14] D. L. Roberts, and C. L. Isbell. 2008. A survey and qualitative Further progress of the research will be, first of all, the analysis of recent advances in drama management. in development of the proposed model and its implementation Transactions on Systems Science and Applications, no. 3. in a game. However, there are also different other [15] M. Sharma, S. Ontañon, M. Mehta, and A. Ram. 2010. Drama Management and Player Modeling for Interactive Fiction advancements that might be needed to obtain a completely Games in Computational Intelligence Journal, Volume 26 Issue 2 valid result. Among the others, a better way of drawing a [16] B. Stewart. 2011. Personality and Play Styles: A Unified Model, PM might be found. Indeed, as mentioned, the model is not Retrieved June 03, 2019 from: https://www.gamasutra.com/view/feature/134842/personality_ dependent on the way of describing the players and it is and_play_styles_a_.php always possible to modify the way the PM is calculated, [17] A. Summerville, and M. Mateas. 2016. Mystical tutor: A magic: The gathering design assistant via denoising sequence- described and stored. In addition, from the point of view of tosequence learning. a storyteller, it might be very useful to conduct a proper [18] A. Summerville, and M. Mateas. 2016. Super Mario as a string: research to analyse the new paradigm for addressing Platformer level generation via LSTMs, in Proceedings of 1st International Joint Conference of DiGRA and FDG. interactivity that emerges from the application of the [19] A. Summerville, S. Snodgrass, M. Guzdial, C. Holmgård, A. proposed model. Also, it might be interesting to examine Hoover, A. Isaksen, A. Nealen, and J. Togelius. 2017. Procedural whether my approach presents restrictions in the stories or Content Generation via Machine Learning (PCGML) in IEEE Transactions on Games. in the mechanics of a game based on it. Lastly, contrarily to [20] D. Thue, V. Bulitko, M. Spetch, and E. Wasylishen. 2007. what might be found in many researches on the field, I Interactive storytelling: A player modeling approach in Proc. of the 3rd AIIDE07. strongly believe that a PCGML model able to improve [21] T. Tutenel, R. Bidarra, R. M. Smelik, and K. J. de Kraker. 2008. human design - rather than substituting it - can help the The role of semantics in games and simulations, in Comput. improvement of such approaches both in literature and in Entertain., vol. 6, no. 4, pp. 1–35. [22] M. N. R. Utsch, G. L. Pappa, L. Chaimowicz. 2018. the industry. As of little help as it might be, I hereby Implementation and Analysis of a Non-Deterministic Drama encourage any studies aimed at this purpose. Manager in Proceedings of SBGames. [23] J. Valls-Vargas, S. Ontañon, J. Zhu. 2013. Towards Story-Based Content Generation: From Plot-Points to Maps, IEEE Conference on Computational Inteligence in Games (CIG) REFERENCES [24] W. of the Coast. 1993. Magic: The Gathering. [1] R. A. Bartle. 1996. Hearts, clubs, diamonds, spades: Players who [25] N. Yee. 2006. Motivations for play in online games in suit MUDs, in Journal of MUD research, Colchester. CyberPsychology & Behavior, 9, 6, pp. 772-775. [2] R. A. Bartle. 2005. Virtual Worlds: Why people play in Massively [26] H. Yu, and M. O. Riedl. 2014. Personalized Interactive Narratives Multiplayer Gamer Development, 2, 1. via Sequential Recommendation of Plot Points in IEEE Transactions on Computational Intelligence and Artificial Intelligence in Games.