Heuristic evaluation for playability and PX: contextualized analysis for game-based systems Juan Antonio Trillo-Manzano, Johnny Alexander Salazar-Cardona, Francisco Luis Gutiérrez-Vela, Patricia Paderewski-Rodríguez Abstract Heuristic evaluations are one of the most used tools to analyze the quality of videogames. However, the general approach to this evaluation doesn’t take into account the type of game- based system it pretends to evaluate, often leading to not applicable heuristics or results lacking context. Considering evaluation variables such as the player profile the game is made for, the goals of the game-based system, the game genre, etc. can be used to improve the displayed heuristics for a game, as well as giving some heuristics more importance than others when calculating the results of the analysis. We propose the extension of an existing heuristic evaluation tool implementing this functionality and making it open to more variables that may be able to help to contextualize the analysis even more. Keywords 1 Heuristic evaluation, playability, player experience, game-based systems, videogames, serious games, gamification, player profile, gameful experiences, game genres, game design 1. Introduction Game-based systems (GBS) variety make them hard to study and analyze in a general way, because some aspects that may be core to a game can be non-existent in another one. This can be a problem when trying to evaluate the playability and player experience (PX) of a game, as the methods used are usually designed after a specific GBS, making it hard to extend these evaluations to other systems. The solution to this problem is to design a heuristic evaluation that can adapt to any kind of GBS that can be designed. However, in order to do so, we first need a classification of the GBS to be able to distinguish the differences between them. We also need to establish a relation between those characteristics and how the results of the evaluation are obtained. 2. Background We have used the term game-based systems, but we haven’t presented a definition of them. GBS have been used in previous instances [1] as a way to not only refer to the usual concept of videogames, but also another game forms like serious games [2]. When we use the term GBS, we consider all software systems that include game design elements in them. GBS can be classified according to various criteria. One of them is the main objectives of the GBS as a game experience. The general concept of videogame is the standard game experience, and its primary goal is to be fun to its players. With the evolution of videogames, some other objectives have been included in the game experience they offer, and nowadays there are GBS that share their main objective of fun with other primary goals. These are the GBS considered as «serious games», like the learning games or purposeful games mentioned in Marczewski’s classification, but there are more, like therapy games [3]. There are also GBS that don’t mind as much being fun, mostly because they’re not I Congreso Español de Videojuegos, December 1–2, 2022, Madrid, Spain EMAIL: juananbeas@correo.ugr.es (J. A. Trillo-Manzano); jasalazar@correo.ugr.es (J. A. Salazar-Cardona); fgutierr@ugr.es (F. L. Gutiérrez-Vela); patricia@ugr.es (P. Paderewski-Rodríguez) ORCID: 0000-0003-0688-6458 (J. A. Trillo-Manzano); 0000-0002-6048-740X (J. A. Salazar-Cardona); 0000-0001-6629-7597 (F. L. Gutiérrez-Vela); 0000-0001-6626-9633 (P. Paderewski-Rodríguez) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) designed as games, but instead they use game elements in a non-gaming context. This use of game elements is called «gamification» [4], and it can have a lot of different goals depending on what the gamified system was designed for. Another factor that needs to be considered is the type of player the GBS is directed to. Lots of authors distinguish player profiles based on their expectations for the GBS and the behavior they show while gaming [5]. Marcewzski [2] also gives a very detailed classification using the motivations the players have when playing a game. There are some characteristics of the player that influence the player profile they present, like the age, and the playability and PX evaluation can be analyzed from this perspective to find the right public the game should be directed to. There are also differences between how GBS present their game. Some of them have really advanced graphics while others look very simplistic; some encourage the player to think strategically to solve the game challenges while others require good reflexes; some use a realistic setting while others delve into fantasy... This is what is called «game genres», and there is a lot of variety between them, making their classifications long and complex [6]. There is also to consider the growing presence of pervasive games, i.e., GBS that extend the concept of game in one or various dimensions [7]. As an example, a geolocalization game uses spacial pervasivity, and so do the VR systems. These kinds of games have very unique characteristics that need to be considered in order to evaluate them with reasonable results. Regarding the existing heuristic evaluations tools for videogames, we consider González Sánchez tool [8] as the base for the extension to GBS. This author proposes a tool based on his playability and PX characterization, PHET (Playability Heuristic Evaluation Tool). The tool, implemented with Excel as a prototype, uses weights to contextualize the results of the evaluation based in things like the player profile or the genre of the game. This idea can be easily adapted to consider GBS in general, so the GBS playability and PX evaluation tool can be based on PHET and modified to include all the types of game-based systems we discussed before. 3. Proposed heuristic evaluation tool (extension of PHET) As we just said, PHET is a tool which can be applied to GBS with the right modifications. In this section, we will study what changes are necessary to give enough context to the evaluation given a specific GBS, considering its goals, the player profiles it is directed to, and its genre as a game. 3.1. Playability and PX characterization Before considering how the GBS goals, the player profile and the game genre are involved in the analysis, we need to establish the playability and PX concepts and characterization we are working with. In González Sánchez thesis, the PX is considered as attributes of playability, and there is another classification with «facets» of playability that differentiates between the game elements. This work will characterize playability using game design elements and PX with player experiences, similarly to the González Sánchez classifications, but with a clearer distinction between them. An early version of this characterization can be seen in Figure 1. The goal with this classification is that it must be general enough to be able to describe all the game design elements and player experiences that can be present in a GBS, but also it must be specific enough to be able to include weights depending on the GBS that it is being analyzed. Not all the characterization elements need to be included in a GBS: for example, pervasivity can only be found in pervasive games, but it must be present in the analysis if that is the case. Figure 1: Playability and PX characterization 3.2. Type of GBS As we said earlier, the player profile the GBS is directed to, the goals of the GBS and its genre are some of the aspects that can influence the analysis of the system quality. They are also mentioned as variables to consider in the final model of his PHET, so it is reasonable to include them in its extension. Starting with player profiles, the principal types of player Marczewski [2] considers are the following: philanthropist, achiever, socializer, free spirit, self-seeker, consumer, networker and exploiter. The first four are player profiles with intrinsic motivations to play the GBS, and the last four are the respective counterpart for player profiles with extrinsic motivations. We can reason some of the relationship between the playability and PX characterization presented and those profiles based on their names. For example, multiplayer aspects will be more relevant to socializer players, as well as group experiences, but the challenge and entertainment experiences like motivation and skill may be the focus of the achievers. These preferences would be carried over to the evaluation using bigger weights with the corresponding game design elements and player experiences, but player profile is only one of the variables considered, so the GBS objectives and game genre can make those weights lose value or get even more relevance. GBS goals are also discussed in Marczewski’s work. He addresses them as “gameful experiences”, but his classification is based on the goals each of those experiences have. Apart from fun, which has been the principal goal of games since they are an entertainment product, we can extract the following game objectives from Marczewski’s classification: learning, meaning, purpose and simulation. It is interesting to also consider therapy as a possible GBS goal, as studied by Horne [3]. Like with player profiles, the GBS goals influence the relevance of the game design elements and player experiences the system offers. A meaningful game will rely more on the sentimental experiences, whereas a simulation may give more importance to structural experiences like immersion or coherence. As a last way to give different weights to certain attributes of the GBS in the playability and PX evaluation, we can classify the GBS based on their game genre. The genre of a game can describe aspects of the game like the style or the theme, as the classification by Lee suggests [5], but the main genre tag that makes an impact in the evaluation of playability and PX is the gameplay genre. This tag includes the following genres: action, action/adventure, driving/racing, fighting, puzzle, RPG, shooter, simulation, sports and strategy. The aspects of the game that certain genres focus on more are often evident. For example, fighting games will give more relevance to the challenge of the system, while RPGs usually have the freedom they offer as their principal asset. 3.3. Extension of the heuristic evaluation tool To be able to use PHET to analyze the quality of GBS other than classic videogames, the first thing we need to do is to add a way of selecting the goals of the system, the player profiles it is directed to and the genre of the game. These variables can be determined by the developers of the GBS or can be inferred by a questionnaire about the characteristics that define each element of the classifications. Once these factors are established, the heuristic evaluation should be adjusted to match them. For example, if the game has not multiplayer functionalities, it is useless to have heuristics for the multiplayer attributes and facets of playability. The next thing to do is considering weights for each of those variables. After the questionnaire is completed, the values of each heuristic would be altered depending on the impact of said variables in that specific heuristic, so that the final results of the analysis for each attribute and facet of the playability reflects the GBS goals, the player profiles and the genre that were selected earlier. 4. Conclusions and future work In this paper, we have proposed a way to extend a playability and PX heuristic evaluation tool so that we can analyze the quality of any GBS. This extension is made based on the characterization of a GBS by its goals, the player profiles the system is directed to, and the genre of the game. These variables help to contextualize the relevance of game design elements and player experiences, of which a new classification has also been proposed. This context is implemented in the evaluation tool as weights that are applied to each game design element and player experience. To establish these weights, we can use logical reasoning and the experience of experts in the matter to get an initial approximation, but as future work we should refine the weights by comparing the results obtained with this tool to the results of evaluations designed for specific games, getting more accurate weights with each new comparison. PHET is based on an Excel prototype, and it is natural that its extension will be initially implemented in the same way. However, it would be interesting to bring this tool to a wider audience, so we will consider implementing the extension in a web or an app to be used not only by experts that need to evaluate a game, but also by players that want to submit their perspective on the quality. 5. References [1] J. Salazar Cardona, F. L. Vela, J. Arango, J. Gallardo, Game-based systems: Towards a new proposal for playability analysis 3082 (2021) 47–56. [2] A. Marczewski, Even Ninja Monkeys Like to Play: Gamification, Game Thinking and Motivational Design, 1st ed., CreateSpace Independent Publishing Platform, 2015. URL: https://www.gamified.uk/gamification-framework/differences-between-gamification-and- games/amp/. [3] L. Horne-Moyer, B. Moyer, D. Messer, E. Messer, The use of electronic games in therapy: a review with clinical implications, Current psychiatry reports 16 (2014) 520. doi:10.1007/ s11920-014-0520-6. [4] K. Seaborn, D. I. Fels, Gamification in theory and action: A survey, International Journal of Human-Computer Studies 74 (2015) 14–31. doi:10.1016/j.ijhcs.2014.09.006, pT: J; UT: WOS:000347765000002. [5] R. A. Bartle, Player types, Jeannie Novak: Game Development Essentials (2008) 39–40. [6] J. Lee, N. Karlova, R. Clarke, K. Thornton, A. Perti, Facet analysis of video game genres (2014). doi:10.9776/14057. [7] J. Arango, F. L. Vela, C. Collazos, F. Moreira, Modeling and Defining the Pervasive Games and Its Components from a Perspective of the Player Experience, 2018, pp. 625–635. doi:10.1007/978-3-319-77712-2_58. [8] J. L. González Sánchez, Jugabilidad. caracterización de la experiencia del jugador en video- juegos, 2010. URL: http://hdl.handle.net/10481/5671.