Heuristic evaluations are one of the most used tools to analyze the quality of videogames. However, the general approach to this evaluation doesn't take into account the type of gamebased system it pretends to evaluate, often leading to not applicable heuristics or results lacking context. Considering evaluation variables such as the player profile the game is made for, the goals of the game-based system, the game genre, etc. can be used to improve the displayed heuristics for a game, as well as giving some heuristics more importance than others when calculating the results of the analysis. We propose the extension of an existing heuristic evaluation tool implementing this functionality and making it open to more variables that may be able to help to contextualize the analysis even more.
Game-based systems (GBS) variety make them hard to study and analyze in a general way, because some aspects that may be core to a game can be non-existent in another one. This can be a problem when trying to evaluate the playability and player experience (PX) of a game, as the methods used are usually designed after a specific GBS, making it hard to extend these evaluations to other systems.
The solution to this problem is to design a heuristic evaluation that can adapt to any kind of GBS that can be designed. However, in order to do so, we first need a classification of the GBS to be able to distinguish the differences between them. We also need to establish a relation between those characteristics and how the results of the evaluation are obtained.
We have used the term game-based systems, but we haven’t presented a definition of them. GBS
have been used in previous instances [
GBS can be classified according to various criteria. One of them is the main objectives of the GBS
as a game experience. The general concept of videogame is the standard game experience, and its
primary goal is to be fun to its players. With the evolution of videogames, some other objectives have
been included in the game experience they offer, and nowadays there are GBS that share their main
objective of fun with other primary goals. These are the GBS considered as «serious games», like the
learning games or purposeful games mentioned in Marczewski’s classification, but there are more, like
therapy games [
Another factor that needs to be considered is the type of player the GBS is directed to. Lots of authors
distinguish player profiles based on their expectations for the GBS and the behavior they show while
gaming [
There are also differences between how GBS present their game. Some of them have really advanced
graphics while others look very simplistic; some encourage the player to think strategically to solve the
game challenges while others require good reflexes; some use a realistic setting while others delve into
fantasy... This is what is called «game genres», and there is a lot of variety between them, making their
classifications long and complex [
Regarding the existing heuristic evaluations tools for videogames, we consider González Sánchez
tool [
As we just said, PHET is a tool which can be applied to GBS with the right modifications. In this section, we will study what changes are necessary to give enough context to the evaluation given a specific GBS, considering its goals, the player profiles it is directed to, and its genre as a game. 3.1.
Before considering how the GBS goals, the player profile and the game genre are involved in the analysis, we need to establish the playability and PX concepts and characterization we are working with. In González Sánchez thesis, the PX is considered as attributes of playability, and there is another classification with «facets» of playability that differentiates between the game elements. This work will characterize playability using game design elements and PX with player experiences, similarly to the González Sánchez classifications, but with a clearer distinction between them. An early version of this characterization can be seen in Figure 1.
The goal with this classification is that it must be general enough to be able to describe all the game design elements and player experiences that can be present in a GBS, but also it must be specific enough to be able to include weights depending on the GBS that it is being analyzed. Not all the characterization elements need to be included in a GBS: for example, pervasivity can only be found in pervasive games, but it must be present in the analysis if that is the case.
As we said earlier, the player profile the GBS is directed to, the goals of the GBS and its genre are some of the aspects that can influence the analysis of the system quality. They are also mentioned as variables to consider in the final model of his PHET, so it is reasonable to include them in its extension.
Starting with player profiles, the principal types of player Marczewski [
We can reason some of the relationship between the playability and PX characterization presented and those profiles based on their names. For example, multiplayer aspects will be more relevant to socializer players, as well as group experiences, but the challenge and entertainment experiences like motivation and skill may be the focus of the achievers. These preferences would be carried over to the evaluation using bigger weights with the corresponding game design elements and player experiences, but player profile is only one of the variables considered, so the GBS objectives and game genre can make those weights lose value or get even more relevance.
GBS goals are also discussed in Marczewski’s work. He addresses them as “gameful experiences”,
but his classification is based on the goals each of those experiences have. Apart from fun, which has
been the principal goal of games since they are an entertainment product, we can extract the following
game objectives from Marczewski’s classification: learning, meaning, purpose and simulation. It is
interesting to also consider therapy as a possible GBS goal, as studied by Horne [
Like with player profiles, the GBS goals influence the relevance of the game design elements and player experiences the system offers. A meaningful game will rely more on the sentimental experiences, whereas a simulation may give more importance to structural experiences like immersion or coherence.
As a last way to give different weights to certain attributes of the GBS in the playability and PX
evaluation, we can classify the GBS based on their game genre. The genre of a game can describe
aspects of the game like the style or the theme, as the classification by Lee suggests [
The aspects of the game that certain genres focus on more are often evident. For example, fighting games will give more relevance to the challenge of the system, while RPGs usually have the freedom they offer as their principal asset.
To be able to use PHET to analyze the quality of GBS other than classic videogames, the first thing we need to do is to add a way of selecting the goals of the system, the player profiles it is directed to and the genre of the game. These variables can be determined by the developers of the GBS or can be inferred by a questionnaire about the characteristics that define each element of the classifications.
Once these factors are established, the heuristic evaluation should be adjusted to match them. For example, if the game has not multiplayer functionalities, it is useless to have heuristics for the multiplayer attributes and facets of playability.
The next thing to do is considering weights for each of those variables. After the questionnaire is completed, the values of each heuristic would be altered depending on the impact of said variables in that specific heuristic, so that the final results of the analysis for each attribute and facet of the playability reflects the GBS goals, the player profiles and the genre that were selected earlier.
In this paper, we have proposed a way to extend a playability and PX heuristic evaluation tool so that we can analyze the quality of any GBS. This extension is made based on the characterization of a GBS by its goals, the player profiles the system is directed to, and the genre of the game. These variables help to contextualize the relevance of game design elements and player experiences, of which a new classification has also been proposed.
This context is implemented in the evaluation tool as weights that are applied to each game design element and player experience. To establish these weights, we can use logical reasoning and the experience of experts in the matter to get an initial approximation, but as future work we should refine the weights by comparing the results obtained with this tool to the results of evaluations designed for specific games, getting more accurate weights with each new comparison.
PHET is based on an Excel prototype, and it is natural that its extension will be initially implemented in the same way. However, it would be interesting to bring this tool to a wider audience, so we will consider implementing the extension in a web or an app to be used not only by experts that need to evaluate a game, but also by players that want to submit their perspective on the quality.