=Paper=
{{Paper
|id=Vol-3305/short4
|storemode=property
|title=Heuristics evaluation for playability and PX: contextualized analysis for game-based systems (poster)
|pdfUrl=https://ceur-ws.org/Vol-3305/short4.pdf
|volume=Vol-3305
|authors=Juan Antonio Trillo-Manzano,Johnny Alexander Salazar-Cardona,Francisco Luis Gutiérrez-Vela,Patricia Paderewski-Rodríguez
|dblpUrl=https://dblp.org/rec/conf/cev/Trillo-ManzanoC22
}}
==Heuristics evaluation for playability and PX: contextualized analysis for game-based systems (poster)==
https://ceur-ws.org/Vol-3305/short4.pdf
Heuristic evaluation for playability and PX: contextualized
analysis for game-based systems
Juan Antonio Trillo-Manzano, Johnny Alexander Salazar-Cardona,
Francisco Luis Gutiérrez-Vela, Patricia Paderewski-Rodríguez
Abstract
Heuristic evaluations are one of the most used tools to analyze the quality of videogames.
However, the general approach to this evaluation doesn’t take into account the type of game-
based system it pretends to evaluate, often leading to not applicable heuristics or results lacking
context. Considering evaluation variables such as the player profile the game is made for, the
goals of the game-based system, the game genre, etc. can be used to improve the displayed
heuristics for a game, as well as giving some heuristics more importance than others when
calculating the results of the analysis. We propose the extension of an existing heuristic
evaluation tool implementing this functionality and making it open to more variables that may
be able to help to contextualize the analysis even more.
Keywords 1
Heuristic evaluation, playability, player experience, game-based systems, videogames, serious
games, gamification, player profile, gameful experiences, game genres, game design
1. Introduction
Game-based systems (GBS) variety make them hard to study and analyze in a general way, because
some aspects that may be core to a game can be non-existent in another one. This can be a problem
when trying to evaluate the playability and player experience (PX) of a game, as the methods used are
usually designed after a specific GBS, making it hard to extend these evaluations to other systems.
The solution to this problem is to design a heuristic evaluation that can adapt to any kind of GBS
that can be designed. However, in order to do so, we first need a classification of the GBS to be able to
distinguish the differences between them. We also need to establish a relation between those
characteristics and how the results of the evaluation are obtained.
2. Background
We have used the term game-based systems, but we haven’t presented a definition of them. GBS
have been used in previous instances [1] as a way to not only refer to the usual concept of videogames,
but also another game forms like serious games [2]. When we use the term GBS, we consider all
software systems that include game design elements in them.
GBS can be classified according to various criteria. One of them is the main objectives of the GBS
as a game experience. The general concept of videogame is the standard game experience, and its
primary goal is to be fun to its players. With the evolution of videogames, some other objectives have
been included in the game experience they offer, and nowadays there are GBS that share their main
objective of fun with other primary goals. These are the GBS considered as «serious games», like the
learning games or purposeful games mentioned in Marczewski’s classification, but there are more, like
therapy games [3]. There are also GBS that don’t mind as much being fun, mostly because they’re not
I Congreso Español de Videojuegos, December 1–2, 2022, Madrid, Spain
EMAIL: juananbeas@correo.ugr.es (J. A. Trillo-Manzano); jasalazar@correo.ugr.es (J. A. Salazar-Cardona);
fgutierr@ugr.es (F. L. Gutiérrez-Vela); patricia@ugr.es (P. Paderewski-Rodríguez)
ORCID: 0000-0003-0688-6458 (J. A. Trillo-Manzano); 0000-0002-6048-740X (J. A. Salazar-Cardona);
0000-0001-6629-7597 (F. L. Gutiérrez-Vela); 0000-0001-6626-9633 (P. Paderewski-Rodríguez)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
designed as games, but instead they use game elements in a non-gaming context. This use of game
elements is called «gamification» [4], and it can have a lot of different goals depending on what the
gamified system was designed for.
Another factor that needs to be considered is the type of player the GBS is directed to. Lots of authors
distinguish player profiles based on their expectations for the GBS and the behavior they show while
gaming [5]. Marcewzski [2] also gives a very detailed classification using the motivations the players
have when playing a game. There are some characteristics of the player that influence the player profile
they present, like the age, and the playability and PX evaluation can be analyzed from this perspective
to find the right public the game should be directed to.
There are also differences between how GBS present their game. Some of them have really advanced
graphics while others look very simplistic; some encourage the player to think strategically to solve the
game challenges while others require good reflexes; some use a realistic setting while others delve into
fantasy... This is what is called «game genres», and there is a lot of variety between them, making their
classifications long and complex [6]. There is also to consider the growing presence of pervasive games,
i.e., GBS that extend the concept of game in one or various dimensions [7]. As an example, a
geolocalization game uses spacial pervasivity, and so do the VR systems. These kinds of games have
very unique characteristics that need to be considered in order to evaluate them with reasonable results.
Regarding the existing heuristic evaluations tools for videogames, we consider González Sánchez
tool [8] as the base for the extension to GBS. This author proposes a tool based on his playability and
PX characterization, PHET (Playability Heuristic Evaluation Tool). The tool, implemented with Excel
as a prototype, uses weights to contextualize the results of the evaluation based in things like the player
profile or the genre of the game. This idea can be easily adapted to consider GBS in general, so the
GBS playability and PX evaluation tool can be based on PHET and modified to include all the types of
game-based systems we discussed before.
3. Proposed heuristic evaluation tool (extension of PHET)
As we just said, PHET is a tool which can be applied to GBS with the right modifications. In this
section, we will study what changes are necessary to give enough context to the evaluation given a
specific GBS, considering its goals, the player profiles it is directed to, and its genre as a game.
3.1. Playability and PX characterization
Before considering how the GBS goals, the player profile and the game genre are involved in the
analysis, we need to establish the playability and PX concepts and characterization we are working
with. In González Sánchez thesis, the PX is considered as attributes of playability, and there is another
classification with «facets» of playability that differentiates between the game elements. This work will
characterize playability using game design elements and PX with player experiences, similarly to the
González Sánchez classifications, but with a clearer distinction between them. An early version of this
characterization can be seen in Figure 1.
The goal with this classification is that it must be general enough to be able to describe all the game
design elements and player experiences that can be present in a GBS, but also it must be specific enough
to be able to include weights depending on the GBS that it is being analyzed. Not all the characterization
elements need to be included in a GBS: for example, pervasivity can only be found in pervasive games,
but it must be present in the analysis if that is the case.
Figure 1: Playability and PX characterization
3.2. Type of GBS
As we said earlier, the player profile the GBS is directed to, the goals of the GBS and its genre are
some of the aspects that can influence the analysis of the system quality. They are also mentioned as
variables to consider in the final model of his PHET, so it is reasonable to include them in its extension.
Starting with player profiles, the principal types of player Marczewski [2] considers are the
following: philanthropist, achiever, socializer, free spirit, self-seeker, consumer, networker and
exploiter. The first four are player profiles with intrinsic motivations to play the GBS, and the last four
are the respective counterpart for player profiles with extrinsic motivations.
We can reason some of the relationship between the playability and PX characterization presented
and those profiles based on their names. For example, multiplayer aspects will be more relevant to
socializer players, as well as group experiences, but the challenge and entertainment experiences like
motivation and skill may be the focus of the achievers. These preferences would be carried over to the
evaluation using bigger weights with the corresponding game design elements and player experiences,
but player profile is only one of the variables considered, so the GBS objectives and game genre can
make those weights lose value or get even more relevance.
GBS goals are also discussed in Marczewski’s work. He addresses them as “gameful experiences”,
but his classification is based on the goals each of those experiences have. Apart from fun, which has
been the principal goal of games since they are an entertainment product, we can extract the following
game objectives from Marczewski’s classification: learning, meaning, purpose and simulation. It is
interesting to also consider therapy as a possible GBS goal, as studied by Horne [3].
Like with player profiles, the GBS goals influence the relevance of the game design elements and
player experiences the system offers. A meaningful game will rely more on the sentimental experiences,
whereas a simulation may give more importance to structural experiences like immersion or coherence.
As a last way to give different weights to certain attributes of the GBS in the playability and PX
evaluation, we can classify the GBS based on their game genre. The genre of a game can describe
aspects of the game like the style or the theme, as the classification by Lee suggests [5], but the main
genre tag that makes an impact in the evaluation of playability and PX is the gameplay genre. This tag
includes the following genres: action, action/adventure, driving/racing, fighting, puzzle, RPG,
shooter, simulation, sports and strategy.
The aspects of the game that certain genres focus on more are often evident. For example, fighting
games will give more relevance to the challenge of the system, while RPGs usually have the freedom
they offer as their principal asset.
3.3. Extension of the heuristic evaluation tool
To be able to use PHET to analyze the quality of GBS other than classic videogames, the first thing
we need to do is to add a way of selecting the goals of the system, the player profiles it is directed to
and the genre of the game. These variables can be determined by the developers of the GBS or can be
inferred by a questionnaire about the characteristics that define each element of the classifications.
Once these factors are established, the heuristic evaluation should be adjusted to match them. For
example, if the game has not multiplayer functionalities, it is useless to have heuristics for the
multiplayer attributes and facets of playability.
The next thing to do is considering weights for each of those variables. After the questionnaire is
completed, the values of each heuristic would be altered depending on the impact of said variables in
that specific heuristic, so that the final results of the analysis for each attribute and facet of the
playability reflects the GBS goals, the player profiles and the genre that were selected earlier.
4. Conclusions and future work
In this paper, we have proposed a way to extend a playability and PX heuristic evaluation tool so
that we can analyze the quality of any GBS. This extension is made based on the characterization of a
GBS by its goals, the player profiles the system is directed to, and the genre of the game. These variables
help to contextualize the relevance of game design elements and player experiences, of which a new
classification has also been proposed.
This context is implemented in the evaluation tool as weights that are applied to each game design
element and player experience. To establish these weights, we can use logical reasoning and the
experience of experts in the matter to get an initial approximation, but as future work we should refine
the weights by comparing the results obtained with this tool to the results of evaluations designed for
specific games, getting more accurate weights with each new comparison.
PHET is based on an Excel prototype, and it is natural that its extension will be initially implemented
in the same way. However, it would be interesting to bring this tool to a wider audience, so we will
consider implementing the extension in a web or an app to be used not only by experts that need to
evaluate a game, but also by players that want to submit their perspective on the quality.
5. References
[1] J. Salazar Cardona, F. L. Vela, J. Arango, J. Gallardo, Game-based systems: Towards a new
proposal for playability analysis 3082 (2021) 47–56.
[2] A. Marczewski, Even Ninja Monkeys Like to Play: Gamification, Game Thinking and
Motivational Design, 1st ed., CreateSpace Independent Publishing Platform, 2015. URL:
https://www.gamified.uk/gamification-framework/differences-between-gamification-and-
games/amp/.
[3] L. Horne-Moyer, B. Moyer, D. Messer, E. Messer, The use of electronic games in therapy: a
review with clinical implications, Current psychiatry reports 16 (2014) 520. doi:10.1007/
s11920-014-0520-6.
[4] K. Seaborn, D. I. Fels, Gamification in theory and action: A survey, International Journal of
Human-Computer Studies 74 (2015) 14–31. doi:10.1016/j.ijhcs.2014.09.006, pT: J; UT:
WOS:000347765000002.
[5] R. A. Bartle, Player types, Jeannie Novak: Game Development Essentials (2008) 39–40.
[6] J. Lee, N. Karlova, R. Clarke, K. Thornton, A. Perti, Facet analysis of video game genres (2014).
doi:10.9776/14057.
[7] J. Arango, F. L. Vela, C. Collazos, F. Moreira, Modeling and Defining the Pervasive Games and
Its Components from a Perspective of the Player Experience, 2018, pp. 625–635.
doi:10.1007/978-3-319-77712-2_58.
[8] J. L. González Sánchez, Jugabilidad. caracterización de la experiencia del jugador en video-
juegos, 2010. URL: http://hdl.handle.net/10481/5671.