Dynamic game difficulty balancing in active ageing systems Alina Delia Călin alinacalin@cs.ubbcluj.ro Babeş-Bolyai University, Department of Computer Science Cluj-Napoca, Romania Abstract This study focuses on dynamic game difficulty balancing (DGDB) in systems oriented towards older people which are using it for cogni- tive training. As these systems have an important therapeutic effect for the user, increasing their engagement and satisfaction by means of DGDB is of utmost importance. We analyse the specific requirements of such systems and their main differences from traditional dynamic video games difficulty balancing, considering that they are based on serious video games, which aim to train the cognitive functions, and their main target group are older people. As active ageing games dif- fer in structure and composition from traditional strategy video games that have been so much studied, in this paper we propose a new game balancing model tailored to meet the requirements of active ageing systems, considering the educational-cognitive characteristic, as well as the age user group (over 60 years old). As a case study, we apply this model on three exergames training cognitive functions, that are part of the MIRA rehabilitation software platform. 1 Introduction Dynamic game difficulty balancing (DGDB) [Wiki16, Andr06] is a research topic that refers to automatically changing game settings, scenarios and parameters based on the players’ ability and gameplay. The purpose of DGDB is to keep users engaged and challenged by the game, and to avoid them becoming bored (which usually happens when the game is too easy and not sufficiently challenging) or frustrated (when the game is too difficult and they lose too often). Researches [Andr06, Bakk12] emphasize that this is an essential and important factor in increasing user satisfaction in games, which is greatly desired in educational, therapeutic, medical or other serious games. DGDB of traditional strategy video games have been widely studied and several aproaches based on methods such as genetic algorithms [Dema02], reinforcement learning [Andr06], multi-layered perceptrons [Chan13] or environment variables manipulation have been proposed and developed, by adjusting elements like non-player characters, number of enemies, resources or task specific time limits [Andr06]. However, active ageing games differ in composition and aim from traditional video games, often lacking the concepts specific to complex First Copyright c by the paper’s authors. Copying permitted for private and academic purposes. In: E. Vatai (ed.): Proceedings of the 11th Joint Conference on Mathematics and Computer Science, Eger, Hungary, 20th – 22nd of May, 2016, published at http://ceur-ws.org 56 Person Shooting games, such as enemies or non-player charaters, thus game balancing in this case requires a different approach [Frei13]. This paper proposes a new emotional-motivational game balancing model tailored for active ageing game based systems, by addressing two main differentiating aspects: • Educational-cognitive characteristic: active ageing games follow different patterns and have different com- position elements than traditional video games. The difference is not only in comparison to strategy video games, but even between themselves, as cognitive games follow various patters which are not necessarily similar. • Specificly targeted user age group: older people are subject to anatomical, cognitive and functional changes related to ageing, that influence game play and interaction, such as vision, audio or sensorial sensitivity decline [Gamb06]. We analyse this model through a case study on Kinect based video games targeting executive functions for promoting active ageing. This was done by identifying the key elements involved in difficulty balancing and the way in which they should be adjusted dynamically, based on emotional indirect input from the users and their sensory acuity. Next, we have proposed some future directions for this research. 2 Background As we have emphasized before, DGDB’s main role is to keep users engaged and entertained by the game, while trying to avoid them becoming bored or frustrated, which is greatly desired in educational, therapeutic, medical or other serious games, so they may reach their purpose. There are several researches proposeing different approaches for DGDB of video games, for example targeting First Person Shooter games [Chan13], real-time fighting games [Andr06] or artificial game presenter characters for computer-based tabletop games (like Wheel of Fortune, Power of Ten, Who Wants to Be a Millionaire, etc.) [Dema02]. However, serious games (or exergames) based on clinical expertise have started to be widely used in the past years [Gamb06, Pari14] for the purpose of physical and cognitive rehabilitation, thus requiring specific attention. For these types of games, increasing user satisfaction ensures that they reach their clinical therapeutic purpose. One such system, using Kinect based interaction, is MIRA, a clinical software platform based on video games, targeting rehabilitation, especially physical therapy. It contains an active ageing package in development aimed to prevent falls in the elderly, as well as to train cognitive function. We have selected three of the cognitive games contained by this platform and analysed, as a case study, an approach for applying our model to dynamically adjust game difficulty for these particular cases. Figure 1: MIRA ActiveCore cognitive exergames package: Memory Scape, Color Clouds and Seasons. The advantage of using the Kinect sensor for interaction in active ageing games is more than a natural, intuitive and interactive way of playing and training, provided by its capability to detect and track human body 57 joints [Micr16]. As the sensor tracks the body motion, it is able to provide this stream for the system not only for interaction, but also for determining secondary movements and actions of the player, which might suggest their state or emotions during gameplay (from their poses or gestures) and help adjust the system to improve their experience in real time. As we have found in our previous studies that Kinect based gesture recognition can provide results up to 99.10% precision and 99.08% accuracy [Cali16] for pose recognition and up to 97.85% for one-hand gesture recognition [Caln16], this proves it has a great potential for identifying meaningful gestures as secondary input. Active ageing games are serious games and, as we have emphasized before, DGDB in their case requires a different approach [Frei13]. In this study, we refer to cognitive games aimed at training some cognitive functions, like working memory or task inhibition. On one hand, these games are much simpler, designed to provide an easy and intuitive interaction for users less familiar with technology and gaming, offering a balanced amount of feedback that engages users without overwhelming them, while avoiding excessive amounts of design complexity. Most of them are adaptations of science-based paper exercises or tasks (such as Color Clouds [Ridl35]), because of their educational-cognitive characteristic, so they follow different patterns and have different composition elements than traditional video games that have have been widely studied because of their commercial popularity. On the other hand, the games are aimed at improving cognitive functions in elderly people, which are a group of users with distinct particularities as opposed to the majority of gamers, which are young adults. Older people are usually subject to anatomical, cognitive and functional changes related to ageing, that influ- ence their capacity of playing video games and interacting, such as vision, audio or sensorial sensitivity decline [Gamb06]. Thus, DGDB must consider adapting elements according to these aspects as well. Table 1 presents different anatomical changes, their impact on the older people’s perception and how they should be considered when adapting DGDB, based on [Gamb06] and [IJss07]. Moreover, there are psychological and cognitive abil- ities that are generally affected by age, such as attention (selective and focused attention, dived attention and attentional switch), automatic and voluntary processing, learning and memory, working and semantic memory and everyday cognitive tasks, most of these being functions we aim to train with our system. Some of these are detailed in Table 1, which also gives some solutions towards them, by compensating through the game design. Table 1: Anatomical changes affecting older people Affected perception with ageing Compensating design factors -adaptation to darkness -adapt contrast -visual acuity -increase environment illumination -contrast sensitivity -increase font size Vision -peripheral vision -minimise the use of peripheral vision -motion perception -shape objects more clearly -colour perception -adapt motion graphics -absolute hearing sensitivity -avoid very low/high frequency sounds -sound localisation -use natural speech instead of generated Hearing -speech recognition speech; adapt words rate -frequency & intensity discrimination -control background noise The challenges of DGDB in exergames are very complex, as from our end we are interested whether they train the cognitive functions, while the user will benefit from them as long as they are still engaging and fun. Moreover, the main question in adjusting the difficulty of the game is when and how to adjust the difficulty of its components, as in when we choose to adjust cognitive task difficulty and when the difficulty of interaction with the system (using mostly the physical abilities and the perception/interaction abilities of the user which are age/user specific and have no correlation with the cognitive function we aim to train). 3 Proposed Action Model Based on these, we propose a new emotional-motivational game balancing model tailored for active ageing game based systems, described in Table 2. This model we propose considers only some basic user states, which are also important for creating the user’s personal preferences map, matching the game scene according to the perceived reaction as liked, disliked, engaging or boring. The user’s reaction sometimes gives information on what exactly generates the emotion/state, for example dislike caused by loud sounds or unpleasant auditory feedback. Most of them are based on the poses 58 and expressions that are body language related, but also on facial emotions, mainly micro expressions that accompany the poses, which we aim as future work based on Kinect’s facial recognition capabilities. Table 2: The DGDB action model we propose based on user states State/emotion Distinctive elements DGDB Action proposed - repeat instructions/tutorials Hands out in why, - highlight main game elements scratch head, (use magnify tool, increase fonts and Confusion/uncertainty hand on chin, contrast, simplify game dynamics) (what should I do?) thinking, narrow - increase game feedback (correct and eyes, eyebrows incorrect action indicators, sounds, pulled together colours, elements) - decrease the volume and amount of Hands on ears (if auditory feedback it is sound related), - decrease complexity and difficulty hands crossed, Discomfort/dislike - add more achievements/rewards hands on hips (both (sad, angry, hearing) - repeat playing rules, highlight game or only one), elements and feedback sad/angry face - add some different game elements expression, and sounds to create a surprise Hands out winning, - keep or increase the level of game praying, Happy/engaged complexity and feedback (difficulty, smile/laugh/ game challenge) loudness Talking on the - propose the user either to pause this phone, looking game or to play a different game Distracted away from the - resume the game with a more screen, hands challenging or clearer task and visual in pocket environment (big fonts, colours, etc.) Eyebrows raised, mouth slightly Surprise - explain rules and keep the new challenge open, standing straight Analysing this model through a case study on three of the games (Figure 1) targeting executive functions for promoting active ageing, we have identified the key elements that could be involved in difficulty balancing and adjusted them dynamically based on emotional indirect input from the users and their sensory acuity, just like in the model presented above. 4 Case Study on Three Active Ageing Exergames According to the emotion and the user state detected at each time from the specific indicators (poses, gestures, facial expressions), game parameters can be adjusted for each of the three following games in particular, in order to increase or decrease difficulty and keep the best level of challenge for the user, as described in Table 2. In order to assess the model we propose, we have applied it on three MIRA [Mira16] cognitive exergames. 4.1 Memory Scape This is a working memory game in which the user has to establish if the new card on the right matches the previous card now turned over on the left. Each card contains different shapes and colours. For this game, difficulty can be adjusted by choosing only certain types of shapes and colours, as well as controlling colour contrast and cards motion (when they are turned over). This is a game in which points are given according to the correctness of the answer and the speed of answering. It can be played with one step (like this one) and with two or three steps (matching the card which was 2 or respectively 3 cards away from the current one). 59 4.2 Color Clouds This game targets cognitive inhibition. The user has to answer with Yes if the meaning of the word in the left corresponds to the font colour of the word in the right, and No otherwise. For this type of game, small and thin fonts can be an impediment in quickly establishing the meaning of the left word and the colour of the right one, for which the obvious reason is to increase font and its boldness. Colour contrast and shades can also influence perception (for example some shades of yellow on a blue background might be confused with a green), for which reason this should also be considered for design. This is a game in which points are given according to both correctness and speed of the answer. 4.3 Seasons In Seasons there is a screen containing several random objects. The purpose is to choose the objects that have not been previously selected, one at a time. After each selection, a new screen appears with a different random selection of objects. As these objects are quite small, we can use a magnifying tool as an aid when they are selected, to better distinguish between very similar objects, in order to establish correctly if they have been previously selected. As the purpose of the game is to train working memory and not visual acuity, this makes much sense. Auditory feedback could also be provided when the user hovers over an item or selects it, so that the user is not restricted in performing well because of a visual impairment. This game is not time constrained. Figure 2: Screenshots of the cognitive games, from left to right: Memory Scape, Color Clouds and Seasons. 5 Conclusions and Future Work The domain of cognitive games for the elderly is one that is still at its beginnings; further development and study need to be done in order to standardize and establish a best practice. In this work we have emphasized some of the challenges, opportunities and approaches that are worth con- sidering in the case of serious games and how different they are from traditional video strategy games that have been so much studied and improved for attaining a good level of DGDB. We have also proposed a model based on indirect input from the Kinect sensor, which proves to be very accurate for gesture and pose recognition, and shown how to apply it in practice on three MIRA cognitive games. For further work, we will also consider using audio feedback and exercise difficulty balancing (as in the difficulty of the task in itself, and not the way it is displayed, perceived or interacted with). Also, this model would require further validation with users and clinicians, as well as further improvements, based on their feedback. 5.0.1 Acknowledgements This work was partially supported by a grant of the Romanian Ministry of Education and Scientific Research, MECS - UEFISCDI, PN II - PT - PCCA - 2013 - 4 - 1797. References [Andr06] G. Andrade, G. Ramalho, A. S. Gomes and V. Corruble. Dynamic Game Balancing: an Evaluation of User Satisfaction, Proceedings of the Second Artificial Intelligence and Interactive Digital Entertainment Conference, 3–8, California, June 2006. 60 [Bakk12] S. Bakkes, C. T. Tan and Y. Pisan. Personalised Gaming: A Motivation and Overview of Literature, ACM Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System, 2012. [Cali16] A. Călin. Variation of Pose and Gesture Recognition Accuracy Using Two Kinect Versions, International Symposium on INnovationsin Intelligent SysTemsand Applications, Sinaia, Romania, 2-5 August 2016. [Caln16] A. Călin. Gesture Recognition on Kinect Time Series Data Using Dynamic Time Warping and Hidden Markov Models, submitted to SYNASC, September 2016. [Chan13] D. M. J. Chang. Dynamic Difficulty Adjustment in Computer Games, Interactive Multimedia Systems Conference, 2013. [Dema02] A. Cruz. Online Coevolution for Action Games, Proceedings of The 3rd International Conference on Intelligent Games And Simulation, 113–120, London, 2002. [Frei13] S. de Freitas, M. Ott, M. M. Popescu and I. Stanescu. New Pedagogical Approaches in Game Enhanced Learning: Curriculum Integration, Information Science Reference, 2013. [Gamb06] L. Gamberini, M. Alcaniz, G. Barresi, M. Fabregat, F. Ibanez and L. Prontu. Cognition, technology and games for the elderly: An introduction to ELDERGAMES Project, PsychNology Journal, 4(3): 285–308, 2006. [IJss07] W. IJsselsteijn , H. H. Nap, Y. de Kort, K. Poels. Digital Game Design for Elderly Users, Proceedings of the 2007 conference on Future Play, ACM New York, 17–22, 2007, USA. [Micr16] ***. Kinect for Xbox Official Website Microsoft Corporation, https://www.xbox.com, 2016. [Mira16] ***. MIRA Software Plarform, MIRA Rehab Ltd, https://www.mirarehab.com, 2016. [Pari14] H. Parisod, A. Pakarinen, L. Kauhanen, M. Aromaa, V. Leppanen, T. N. Liukkonen, J. Smed, and S. Salantera. Promoting Childrens Health with Digital Games: A Review of Reviews, Games For Health Journal: Research, Development, and Clinical Applications, 3(3): 145–157, 2014. [Ridl35] S. J. Ridley. Studies of interference in serial verbal reactions Journal of Experimental Psychology, 18(6): 643–662, 1935. [Sapo10] G. Saposnik, R. Teasell, M. Mamdani, J. Hall, W. McIlroy, D. Cheung, K. E. Thorpe, L. G. Cohen, M. Bayley, Effectiveness of Virtual Reality Using Wii Gaming Technology in Stroke Rehabilitation- A Pilot Randomized Clinical Trial and Proof of Principle, American Heart Association, 2010. [Wiki16] ***. Dynamic game difficulty balancing, Wikipedia, https://en.wikipedia.org, 2016. 61