Playability evaluation relies on heuristics as one of the most prominent approaches by virtue of their comprehensiveness and relative ease of implementation. However, due to the vastness of available game-based systems, their varied purposes, and the diversity of the player audience, defining heuristics that focus on thorough playability assessment carries a considerable challenge. This paper presents an online tool designed to evaluate the playability of game experiences. For its development, it has been based on a general-purpose playability assessment tool, which has been improved and adjusted according to previous research results. Efforts have been directed towards a specific population, in this case, the elderly, with the purpose of adapting the tool to a specific practical case. It is expected that the results obtained through this tool will provide a solid means to evaluate the playability of game experiences, not only in the general public, but also in specific populations, such as the elderly. This is intended to provide high quality game-based systems that are attractive and generate a high level of player engagement.
In contemporary times, technological advances have led to the generation of new experiences in multiple facets of human life, such as education, health, employment and entertainment. Within the field of entertainment, specifically regarding game-based systems (GBS), innovative modes of interaction have emerged through various technological devices. This evolution allows the appreciation of highly immersive and enjoyable experiences. A GBS is defined as the application of game design principles and elements to address challenges in a creative, interactive and participatory manner. Its main objective is to enhance experiences and achieve optimal results. To achieve this, GBS makes efficient use of participants' motivations to foster a high level of engagement, stimulate skill development, facilitate knowledge acquisition, generate positive emotions and promote overall wellbeing. These motivations are key drivers of participation and full immersion of individuals in the world of the game [1].
During the interaction between the user and the technology, usability plays a crucial role. This term refers to the "ability of a product to be used effectively, efficiently and satisfactorily by users to achieve their goals in a particular context" [2]. Although there are numerous methods for assessing usability in GBS, none of them are completely objective. GBS incorporate subjective aspects that cannot be accommodated in conventional usability evaluations, nor can they be adequately addressed by assessing positive or negative user impressions from a User Experience (UX) perspective [3]. In this situation, the concept of playability becomes more accurate and appropriate, since it facilitates the evaluation of a GBS as a software product, providing a more accurate measure of the fun that the game can offer [4].
The design and implementation of the tool conceived for the evaluation of playability in game experiences is presented below. The development of this evaluation tool is based on previous research that has been exhaustively carried out, thus providing a solid and objective basis for its conception. Furthermore, existing design guidelines have been considered and special attention has been focused on the specific motivations that guide it. The structure of this paper is presented as follows: Section 2 provides a concise exposition of the concept of playability, including its facets, attributes, and proposed extensions from existing theories, as well as addressing the concept of GBS and pervasiveness. Section 3 focuses on detailing the design and implementation process of the evaluation tool, as well as its availability through the PL/PX web platform. Finally, section 4 includes the conclusions obtained in this work and points out possible directions to be followed in future research.
The following is a conceptualization of the main topics addressed in this document, such as Game Based Systems, playability and its differences with usability, and concepts related to pervasive games. This last topic is addressed because the implemented web platform offers the possibility of measuring the degree of pervasiveness of a game experience.
2.1.
There are several approaches that use GBS to enhance the user experience and achieve different objectives. These approaches include concepts such as "playful design", which refers to the use of designs and illustrations inspired by the world of games or design elements typically associated with that category, to provide a visually engaging experience. The "gamification" is distinguished by the inclusion of mechanics, dynamics and elements typical of games in contexts that, a priori, do not have a playful nature. Its main purpose is to stimulate user engagement and participation in a meaningful way. The "simulation" is based on the creation of virtual representations of real-world elements, thus allowing learning, practicing and experimenting in a safe and controlled environment. The "serious games" deserve special mention, as they go beyond mere entertainment. These games are used for educational, training or behavior modification purposes, giving them a substantial purpose in the context of the user experience. Finally, "games” are mainly conceived to provide fun and entertainment to the user, with no other objectives than to delight and captivate the audience [1]. The various approaches mentioned above provide opportunities for the creation of interactive and engaging experiences for gamers, taking into consideration their individual requirements and preferences. 2.2.
Undoubtedly, several usability evaluation procedures have been devised to quantify the users' experience, perceptions and degree of acceptance of the GBS. However, it should be noted that such methods are not fully adapted to the needs and specificities inherent to the target population, nor to the inherent subjectivity that characterizes these systems. In this context, it is pertinent to mention the existence of applications designed for the assessment of the quality of GBS, making use of traditional usability tools, such as the System Usability Scales (SUS) [5]. For example, there are different researches where players have played a game and then answer the SUS questionnaire to obtain information on effectiveness, efficiency and satisfaction [6]–[8]. There are also cases in which the SUS has been used to analyze satisfaction in the use of games oriented to cognitive training [9].
Although there are several approaches to assessing the playability of a GBS, one of the most comprehensive theoretical frameworks at present is the one proposed by González and Gutiérrez [4]. According to their approach, the quality of a GBS is determined by a series of key elements that encompass its objectives, rules, mechanics and game dynamics, as well as the experiences and emotions experienced by the player during their interaction with the system. Furthermore, these authors define playability as "the degree to which users can achieve their objectives effectively, efficiently and, above all, experience satisfaction and entertainment in the context of the game" [10]. Following this perspective, González and Gutiérrez propose six different facets that make up playability, each of which offers a unique view of the concept: • Intrinsic playability: It refers to the way the game is presented to the player and is evaluated through the very essence of the game itself. This evaluation covers aspects such as the rules that govern the game, its level of difficulty, the timing it establishes, the objectives it sets and the game mechanics employed. • Mechanic playability: It is linked to the quality of the video game as a software system. It focuses specifically on issues such as the response times offered, the fluidity in its execution and the physical characteristics of the characters and the virtual environment that surrounds it. • Interactive playability: It directs attention to the elements that mediate the interaction between the game and the user. This includes the graphical interface presented by the game, the effectiveness and usability of the controls provided, and the input devices used in the game experience. • Artistic playability: It is linked to the quality and artistic and aesthetic relevance of the components that make up the game. This includes aspects such as graphic quality, setting, sound composition and narrative plot. • Personal playability: It is a subjective analysis, focused on the perception and feelings experienced by the player on an individual level. This aspect has a high subjective component, as it varies significantly from one person to another. • Social playability: It examines how the player perceives and experiences the game in a group context, especially when playing in company. This analysis can address competitive, collaborative or cooperative experiences, depending on the game dynamics adopted by the group.
In each facet of playability, an evaluation of attributes covering all these areas is carried out. The authors propose seven playability attributes: effectiveness, learnability, immersion, satisfaction, motivation, emotion and socialization [10]. The concept of satisfaction is related to the degree of satisfaction experienced by the player when participating in the video game. Learning is defined as the ability to easily understand and master the objectives, rules and forms of interaction present in the game. Effectiveness refers to the time and resources required for the game to provide entertainment to the player while the player achieves goals and overcomes challenges. Immersion is linked to the player's conviction to become fully immersed in the virtual world presented by the game. Motivation encompasses the way in which the game stimulates the player to perform actions and repeat them until they are achieved. Emotion encompasses the feelings and reactions that arise as a result of the stimuli provided by the game. Finally, socialization is related to the perception and appreciation that the player experiences when participating in the game in company, whether competitively, collaboratively or cooperatively.
2.3.
Pervasive games represent a novel genre of entertainment that places a paramount emphasis on the
player's experiential engagement. As elucidated by Arango et al. [11], these games deliver a captivating
and enriching experience by orchestrating the evolution of dynamic elements while also extending the
realm of gameplay contingent upon the contextual milieu in which they unfold. This distinctive attribute
affords players the capacity to transcend the conventional confines of the game world, allowing for a
seamless integration with reality and, moreover, permitting real-world elements to exert a tangible
influence upon the overall gaming experience. The conceptualization of pervasive games as articulated
above owes a substantial debt to antecedent definitions in the field. Notably, Montola's definitions [12],
[13] have played a pivotal role in shaping our understanding of this genre. Montola contends that
pervasive games boldly challenge the established parameters of traditional computer games, pushing
beyond the limitations of spatial, social, and temporal dimensions. This conceptual demarcation
underscores a fundamental divergence between pervasive games and their conventional counterparts,
as the former defy the boundaries delineated by Huizinga's notion of the "magic circle" [14].
Conversely, non-pervasive games, firmly ensconced within the confines of the magic circle, remain
steadfast in adhering to specific spatial and temporal constraints, often engaging with a predetermined
set of participants [15], [
The elements aforementioned encompass a diverse spectrum of potentialities in the realm of conceiving and executing pervasive gaming encounters. This pertains to a broad demographic, inclusive of the general populace and individuals with particular requisites. It becomes imperative for these gaming encounters to place unwavering emphasis on the perpetuation of an optimal player experience (PX). Within this context, PX denotes the unique encounter of an individual player during their interaction with a GBS [18]. Furthermore, it is paramount to bear in mind that the mere utilization of pervasive-enabling technologies and apparatus does not inherently confer the attribute of pervasiveness upon the gaming experience.
Based on previous research specifically oriented to the elderly population [
As previously mentioned, the evaluation process defined by Gonzalez and Gutierrez [4] has inspired
the creation of an extension, which considers the different design recommendations related to the
motivations and game elements that should be considered in the older adult population [
Although the original model consists of six facets, three new facets have been incorporated: adaptive playability, pervasive playability and persuasive playability.
The checklists for each existing facet have been completely restructured to adapt to the characteristics and requirements of a specific population, that is, elderly players. Their motivations and the particularities that influence their experience during the game activity have been carefully considered.
Regarding the analysis of pervasiveness, a supplementary analysis has been incorporated that addresses the different dimensions of pervasiveness from a technological perspective. Attributes such as naturalness of interaction, degree of immersion, configuration and security have been considered.
Regarding the original evaluation process, it is important to note that the mechanic playability facet was minimally modified. This circumstance is explained by the fact that this facet focuses on the assessment of aspects such as response times and fluency, which are closely linked to the implementation technology and remain invariable regardless of the target population. 3.1.
The playability evaluation process of a GBS is characterized as an activity based on the application of Nielsen's heuristic evaluation technique. This method requires the participation of an expert evaluator, given the complex and detailed nature of the task, demanding a solid knowledge in the field of games. Each of the heuristics, together with their checklists, are not only considered as design recommendations, but also as concrete evaluation elements aimed at measuring the quality of the experience provided by the game. The design of the playability evaluation process involved a series of stages, the development and description of which are detailed below (see Figure 1).
As previously mentioned, during the conception of the assessment procedure, it was realized that the pre-existing facets were insufficient to effectively address the multiple checklists derived from the motivational heuristics (see Appendix A) and the previously established transversal elements (see Appendix B). Therefore, stage 1 focused on the development of three new facets with the purpose of providing a more structured and comprehensive evaluation.
• Adaptative playability: Adaptative playability is a concept that encompasses a series of components aimed at customizing the game experience according to the player's preferences and requirements. This notion includes various aspects, ranging from the technology utilized to particular design elements, such as the typography used and the ability to adjust the game experience according to the demands and particularities of the target audience. • Persuasive playability: It is a subjective analysis focused on elements that, in general terms, arouse positive emotions, motivate and captivate the target audience. These elements are not quantifiable by other facets of playability. • Pervasive playability: It is an evaluation of the quality of a game experience by virtue of various components that amplify the extent to which the game experience becomes pervasive. Additionally, it provides a means to quantify the degree of pervasiveness along multiple dimensions and attributes found within this domain.
After the conceptualization of the newly defined facets, proceeded to the assignment of the various checklists linked to the motivational heuristics and game elements to the different facets that make up the game experience. This process represents stage 2 of the analysis. In the distribution process, the assignment of each heuristic to multiple facets of playability is found, based on its relevance. This is because each facet represents a unique perspective of the game, which makes it possible to apply a heuristic to one or several facets, as required by the situation. Consequently, it is possible to find similar questions in more than one facet, specifically adapted to the context of that facet. An example of this situation is evident in the generation of "empathy" in the player, an element that can be approached both from the perspective of "artistic playability", since the artistic elements are designed with the intention of evoking emotions in the player, and from the perspective of "personal playability", which involves analyzing the emotions generated in the player during the game experience.
In the context of this study, a detailed analysis of each checklist applied in various facets was carried
out. It was determined whether these checklists originated from the motivation heuristics or from the
pervasiveness and game element heuristics. This distinction was implemented to avoid redundancy in
the questions, thus marking stage 3 of the research process. In addition, the motivation category that
each checklist affects was identified, as well as the type of fun that is sought to be assessed [23] and the
transversal game element that it impacts, according to the fundamental models and theories that support
the definition of the various heuristics and checklists (stage 4) [
Within the framework of the checklists assigned to the various predefined facets, stage 6 was carried out. In this phase, a detailed evaluation of each of these elements was established using a 5-point Likert scale. On this scale, a value of 1 represents a negligible or no rating, while a value of 5 denotes a substantial or considerable rating. It is important to note that meticulous attention was provided to the formulation of the questions in such a way that they were inherently consistent with this type of evaluation. In the outcome of stage 6, the generation of weights was achieved, which enabled the presentation of a variety of descriptive statistical results to the experts. These descriptive results allow a thorough analysis of the results derived from the evaluation in the stage 7.
3.2.
Based on the design of the evaluation process previously described and the advances in research related to the analysis of game experiences and fun, the PL/PX web platform has been developed. The main objective of this platform is to centralize the aforementioned results, facilitating their access through the web. The platform is available at https://plpx.johnnysalazar.net/, is fully responsive and is available in English and Spanish (see Figure 2). PL/PX covers the detailed definition of a standardized evaluation process, as well as heuristic definitions aimed at identifying potential problems in the design of the game experience, with a focus on motivation and game elements to enhance the player's experience. In addition, it offers an evaluation tool that allows the assessment of the game experience as lived by the participants, as well as the evaluation of the playability of a GBS. Although all the elements currently available are oriented to the older adult population, it is expected that the platform in the short term will provide other evaluation approaches, oriented to a general public or other specific populations with different needs.
The platform provides an evaluation tool that enables a thorough analysis of the 9 facets that make up the playability. This evaluation is carried out by means of a set of specific questions, each one aimed at evaluating a particular facet. These questions require to be evaluated using a Likert-type scale, which covers 5 points. In addition, it should be noted that the evaluator has the faculty to express observations and comments regarding each of the evaluation items, also having the option to omit them as required (see Figure 3).
Each item answered is linked to the different attributes of the playability, allowing preliminary results to be obtained for each facet answered (see Figure 4). These preliminary results are shown from different graphs that provide a holistic analysis of the gameplay facet evaluated. The graphs used for this are the radar, doughnut and bar charts. Easily distinguishable colors have been chosen to avoid interpretation problems. In addition, these graphs are interactive, allowing the application of filters in real time, as well as the display of additional information by hovering the mouse over them.
When the expert conducting the evaluation has answered the different facets of playability that are considered necessary, a detailed analysis of the results can be obtained. This analysis makes it possible to compare the different facets analyzed, identifying differences between them (see Figure 5). These graphs, as well as the preliminary graphs, are fully interactive for the application of filters in real time. In addition, they also offer the visualization of additional information by hovering the mouse cursor over them. elaboration.
The general behavior of the different attributes can also be observed through the different facets (see Figure 6). Due to the large number of attributes available for each playability facet, and to the 9 facets available, it was decided to use a bar chart grouped by each facet. This allows to visualize in a concise way the results for each facet, as well as the changes of the attributes behaviors for each one of them. In addition, like the previous graphs, it offers interactivity and the possibility of additional visualization.
If the pervasive playability facet was answered in the evaluation process, additional results can be seen specifically focused on the analysis of the degree of pervasiveness of the experience (see Figure 7). These results allow a comparison of the different expansions of pervasiveness, as well as a technological view through the different defined properties such as naturalness of interaction, safety, immersion and configuration. These elements allow a deeper analysis to determine which specific elements need to be improved to enhance the pervasiveness of a gaming experience.
The conception and execution of the process of assessing playability in game environments focused on a particular demographic was a challenge of considerable proportions. To carry it out in a systematic and objective manner, it was imperative to conduct preliminary research oriented towards the incentive elements and game characteristics relevant to the population group in question. These investigations not only contributed to the definition of the process itself, but also allowed for its improvement and expansion based on a general playability evaluation model that already existed previously.
The paradigm and the reference framework of the playability model underwent significant modifications and expansions to adapt them to the proposed proposal. As a result of these transformations, three new facets of analysis have been generated in the field of playability. These additions enrich the analysis capacity with respect to the level of immersion achieved in a game experience. Additionally, a comprehensive restructuring of the different checklists corresponding to the nine previously defined playability facets was carried out. This process included the redistribution of the various attributes associated with playability, with the primary objective of providing more accurate and appropriate results for the target population. This work of adapting and refining the playability model and tool represents a significant step towards improving the assessment and understanding of the game experience in the context of the older adult population from a serious and academic perspective.
The evaluation tool conceived from this process is freely accessible through a web platform hosted at https://plpx.johnnysalazar.net/. This platform acts as a consolidation center for all research results related to the topic in question. Although the central purpose of this tool focuses on the evaluation of game experiences that present a significant degree of pervasiveness, given their favorable level of acceptance by the target population, it is important to highlight that its applicability is not limited exclusively to this type of games based on pervasiveness, but can be extended to any modality of GBS, adjusting the relevant facets according to the particular context in which it is applied.
The present application case for older adult demographics could be extrapolated to adapt analogous assessments to different population groups, rigorously following the guidelines and methodologies previously used in this study. Future perspectives of this research should focus on the conceptualization of an assessment suitable for implementation in a general population, as well as on the application of the tool for the purpose of assessing game experiences in a context involving expert evaluators. This approach will allow for thorough documentation of the evaluation process and provide substantial insight into the interpretation of the results and their applicability to projects in practice.
• •
This work has been supported by the PLEISAR-Social project, Ref. PID2022-136779OB-C33, funded by the Ministry of Science and Innovation (MCI/AEI/FEDER, UE). Finally, we would like to thank MINCIENCIAS of government of Colombia for providing the necessary resources for the completion of this research.
A. Marczewski, “Game Thinking,” in Even Ninja Monkeys Like to Play: Gamification, Game Thinking and Motivational Design, 1st ed., CreateSpace Independent Publishing Platform., Ed. [20]