=Paper= {{Paper |id=Vol-2719/paper13 |storemode=property |title=An exploration on automating player personality identification in role playing games |pdfUrl=https://ceur-ws.org/Vol-2719/paper13.pdf |volume=Vol-2719 |authors=Alejandro Villar,Leonor Cuesta,Rodrigo M. Pérez,Carlos León |dblpUrl=https://dblp.org/rec/conf/cosecivi/VillarCPL20 }} ==An exploration on automating player personality identification in role playing games== https://ceur-ws.org/Vol-2719/paper13.pdf

An Exploration on Automating Player
Personality Identification in Role Playing
Games?

Alejandro Villar1 , Leonor Cuesta2 , Rodrigo M. Pérez3 , and Carlos León4

Computer Science Faculty and Institute of Knowledge Technology, Universidad
Complutense de Madrid, Spain
{1 alvill04,2 leonorcu,3 rodrip01,4 cleon}@ucm.es

Abstract. When creating videogames, designers adjust the game char-
acteristics to optimize the experience of the target players. This design
process is usually manual, and the amount of insight that videogame
designers have over the player is limited. Identifying the psychological
profile of a videogame user can lead to specific adaptation of videogame
aspects like narrative, length or challenge. Additionally, performing this
identification automatically can both leverage the game experience and
reduce the amount of work needed for customizing the players’ experi-
ence. This research describes an emergent methodology to automatically
create the personality profile of a player, and a prototype implemen-
tation. Additionally, a pilot study has been run, with preliminary but
promising results.

Keywords: Videogame · Personality · Profiling · Non-intrusive · Inter-
activity

1 Introduction
Nowadays, videogames include several adaptive features that react to the player
actions and strategies, and this produces more adapted experiences in which
users can craft unique and comfortable experiences. These adaptations are usu-
ally based on ad-hoc approaches focusing on the task at hand, and react to
the player performance and in-game decisions. Designers are constrained by the
accessible data that a specific game can provide, usually restricted to variables
related to gaming performance instead of what the player expects from the ex-
perience. This limits the designer’s options when aiming at enabling a closer
player-focused design.
Player-specific game adaptation has been tested in several games, but the
implementations mostly rely on difficulty changes. This usually imply translating
adaptability into adding or reducing features like the number of enemies, speed
?
This work has been supported by the CANTOR project (PID2019-108927RB-I00)
funded by the Spanish Ministry of Science and Innovation; and by the project FEI
INVITAR-IA (FEI-EU-17-23) of the University Complutense of Madrid.

Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
2 Villar et al.

of events or time allowed until the user manages to complete the level. This
approach can lead to successful gaming experiences. However, more complex
games like roleplaying games, which have more complex dynamics, cannot rely
on this simplification for a rich customized experience.
In particular, one of the most important aspects in roleplaying games is
the player personality profile. Roleplaying games have a very important com-
ponent of decision making, and the different players can take different options
according to both their role and their own personality traits. Adapting roleplay-
ing videogames to the particularities of these different types of personality can
leverage the players’ experience.
Against this background, this paper sets out as an exploratory effort to pro-
vide a method to automatically identify specific aspects of the players personal-
ity. The research is based on the hypothesis that it is possible to carry out auto-
matic personality profiling in a non-intrusive way in a roleplaying videogame. In
order to deploy the methodology and test the validity of the hypothesis, a study
where the player profile was automatically acquired was run. The research used
the Big Five personality model, and the overall identification of the personality
traits was inferred from the player interaction with non-player characters in a
virtual environment. The results were compared against the results of a running
a validated Big Five questionnaire with the subjects.
The results (detailed in section 4) show that, while the exploratory character
of this work does not yet permit to draw strong conclusions, the initial hypothesis
of being able to identify user personality in an automatic non-intrusive way
through videogames is plausible.

2 Previous Work

While not exclusively related to psychological models, the idea of automated
user profiling has already been explored in several videogames. For instance,
Wii Fit [16] seeks to profile the user physical condition split on speed, balance
and stamina through daily tests, tracking the exercises realized in-game; Dr.
Kawashima’s Brain Training [14] tries to keep the brain healthy and young
based on memory or concentration exercises, and adapts the difficulty to the
predicted mental age as well as the current performance in the selected exercise,
changing dynamically [17]. Pokémon Mystery Dungeon [15] assigns a Pokémon
to the player according to a small personality test where answers add to a set
of 14 personality types, assigning the Pokemon linked to the highest trait at
the end. In all these cases, the system tries to be non-intrusive. In most cases,
however, the level of profiling is relatively shallow and does not provide valuable
data beyond that specific for the particular game.
Regarding psychological aspects, the particularities, types and features of
personality have been studied from several perspectives. For instance, Raymond
Cattell proposed a theory in which personality was divided into 16 factors [4].
Briggs and Myers [13] studied the innate characteristics of a person and how
they establish basic personality characteristics. In particular, this project has
Automating Player Personality Identification 3

applied the model proposed by Lewis Goldberg [7], the Big Five model. The Big
Five model is broadly accepted and it is one of the most commonly used models
of personality. The model divides human personality in five categories, namely
openness to experience (O), conscientiousness (C), extraversion (E), agreeable-
ness (A) and neuroticism (N). It is also known as the OCEAN model [8]. These
five factors are briefly explained below:
Openness to experience describes how creative, adventurous and curious the
person is. Those with high scores are more curious and open to new experiences.
Conscientiousness refers to the efficiency and organization of a person. A low
score indicates greater spontaneity. Extraversion is connected to how outgoing
and energetic a person is. The higher the score, the more enthusiastic and out-
going a person is. Agreeableness describes how friendly and empathetic a person
is. Neuroticism describes how sensitive and nervous people are. A low score
indicates that the person is emotionally stable.
Based on the Big Five model, there exist several Big Five Inventories (BFI)
with different amount of elements, but always with the same approach and ob-
jective. One of the most widely used and currently accepted questionnaire is the
44-item BFI created by Oliver P. John and Sanjay Srivastava [9]. The English
version of the questionnaire and the necessary calculations can be found in the
work cited above. A Spanish version is provided in another study where Oliver
P. John himself was one of the researchers [3].
These models have been previously used in videogames. Chris Bateman and
Richard Boon profiled four types of videogame players [1]. The results were
used for a related research in which player performance in Fallout New Vegas
was used to identify the category of the player after the gameplay [11]. More
recent research has studied how to create an automatic player profile from a
purely conversational adventure [10], using the dispositional approach psycho-
logical model [19]. Additionally, there are studies on the relation between play
style in World of Warcraft and the results from the Big Five questionnaire [2].
To our best knowledge, no research has applied the Big Five model to a general
RPG setting without any intermediate assumption on the game type.

3 A RPG-based Tutorial for Profile Acquisition

In order to test the hypothesis presented in section 1, a simple prototype of
a roleplaying game has been implemented. The objective is to have subjects
playing freely and create a user personality profile based on the initial stages of
the gameplay and the decisions they make.
The videogame has been created with Minecraft [12].The use of Minecraft
for research on videogames has been previously explored [6]. It offers a complete
and finished work environment [20], so it is possible to bypass a relatively large
proportion of the development time that would have been required for build-
ing an experience from scratch.It gives the possibility to play in different game
modes, each with different functionalities, and to be able to choose the difficulty
level. This allows to adapt the experience in a very simple way to the research’s
4 Villar et al.

needs. Choosing Minecraft as development environment came with several up-
sides, with plenty of customization from the game itself, since it allows for game
modes to just add content to a map (creative mode) or to experience a finished
one without changing it (adventure mode), combined with difficulty settings that
do not spawn enemies (pacific) creates the perfect medium to both create and
conduct experiments where even newcomers can interact with the world freely.
In the prototype, a simple town with several characters and a few simple quests
were implemented. A screenshot of the prototype can be seen in Figure 1.

Fig. 1. Screenshot of the game prototype.

Among others, the prototype used two mods: JourneyMap, a customizable
mini-map so that any tester can find their way through the level, and Custom
NPCs, a tool designed to create RPGs based on answering dialogues and com-
pleting quests.

3.1 First Pilot Study
A complex, multiple-branch story was written for deploying the system. The
story was composed as a set of dialogs. The Non-Player Characters (NPCs) were
assigned dialogue trees that follow a common narrative (composed by 4 short
stories). The different branches were written to cover the Big Five categories,
adding or subtracting points from them as the user chooses an option, their per-
sonality mirroring their answer to specific conversational stimulus. These points
have been assigned to each of the dialog interventions based on the research
team criteria.
This first prototype sought to observe the initial response to the interactive
experience. For the same reason the experiment was conducted in person and the
researcher had to observe the player while he or she played without interfering
with his decisions, and write down the options he chose for later analysis. 6
Automating Player Personality Identification 5

samples were obtained. Once the user had finished playing, a questionnaire with
the original Spanish version of the Big Five Inventory was handed over [3]. The
answers were recorded and the results were processed to calculate the values
of each of the Big Five factors. At the end of the experiment, player dialogue
decisions were processed to calculate the result of each factor according to the
values assigned for each interaction. Since the values of each factor of the Big
Five were obtained as a value between 0 and 100 according to the answers of
the questionnaire, it was necessary to normalize the results of the gameplay to
the same range. The maximum and minimum values that could be obtained by
the user along the virtual experience can be seen in the Table 1. These values
correspond to the points assigned to the dialog options. It must be noted that the
neuroticism category has not been included since at this point in the experiment
it was not intended to obtain its value through user interactions.

Openess Conscientiousness Extraversion Agreeablenes Neuroticism
Max 14 7 21 12 -
Min -10 -10 -5 -12 -
Table 1. Gameplay’s maximum and minimum of each factor that could be obtained
(First Prototype)

Once the results from the questionnaire and the calculation of the game
interactions are obtained, a comparison was made to relate both values and
get the error rates for each factor. The results were obtained by applying the
arithmetic mean of the absolute values of each subject’s error. The results can
be observed in the Table 2.

Openness Conscientiousness Extraversion Agreeableness Neuroticism
Mean 16.94 21.79 23.06 14.08 -
Std. Dev. 8.73 21.3 12.25 13.25 -
Table 2. Average differences and standard deviation between BFI Questionnaire and
Gameplay.

It can be seen that the category with the highest error is the Extraversion.
Agreeableness is the category where the results are most related and the data
is concentrated around the average. Following these first results, it was hypoth-
esized that the source of error for each category is due to the value ranges of
the user’s decision range are not equidistant. This is based on the fact that the
best results obtained correspond to Agreeableness, this being the only factor that
6 Villar et al.

fulfills this equidistance. This means that the user start with 0 points and add
or subtract with the chosen dialog option, where 12 points in the gameplay is
equal to 100 in the questionnaire and -12 is 0. Section 3.2 describes the changes
applied in the second prototype of the research.

3.2 Second Pilot Study

Based on the results and conclusions of the first iteration, a number of changes
were applied to address the limitations. For this purpose a modification of the
gameplay was made.
The dialogues interventions were revalued to offset category’s maximums and
minimums, as shown in Table 3. In addition, it was decided to include values
to Neuroticism category. Additionally, only those branches of dialogues that the
player accessed were taken into account for the total sum. Otherwise, if if all the
possible lines were taken into account, the results would be normalized against
non-valid data points.

Openness Conscientiousness Extraversion Agreeableness Neuroticism
Max 47 18 45 54 34
Min -44 -17 -45 -53 -36
Table 3. Maximum and minimum obtainable values in the gameplay (Second Proto-
type).

In addition, and in order to increase the access to potential players, for this
second iteration, a telemetry system for automatic capture and storage of user
data was implemented. Screen sharing systems were used for this purpose. Steam
Remote [5] and Parsec [18] were used. In the sessions, the user had full control
of the researcher’s computer.

4 Overall Results and Analysis

In the second pilot study, a total of 42 experiments were carried out. Figure 2
shows the values obtained in this second pilot by each player for each one of the
categories in the questionnaire and gameplay.
As it can be seen in Figure 2, the results yield a very high dispersion along
the Y-axis in some factors such as Conscientiousness. However, other factors
concentrate their results around smaller aspects such as Openness. For Consci-
entiousness, the dispersion cover almost all the possible range. While this is in
line with the usual dispersion of the personality traits in the Big Five model,
it contrast with the less dispersed factors. It is assumed that this is because of
the characteristics of the videogame: most of the evaluation points have strong
Automating Player Personality Identification 7

Conscientiousness differences, and they define the type of game that the player
chooses. As such, the stories are better suited for a diverse range. On the con-
trary, aspects less involved in the narrative decisions (like Openness) tend to
show a more condensed set of answers.

In parallel with the analysis of the responses, the accuracy of the model (i.e.
the similarity between the results of the gameplay and the results of the Big
Five questionnaire) is shown in Figure 3, which shows the mean squared error
(MSE) of the Big Five dimensions. In all cases except for Conscientiousness
in story 4, the MSE value is below 20%. This leads to support the idea that,
while the accuracy of the player profiling process is not perfect, the gameplay
helps to approximate the Big Five values for players. The higher error that
can be observed for Conscientiousness is assumed to be caused by the same
characteristic discussed earlier, namely the wider range of captured values for
Conscientiousness in the gameplay due to the narrative particularities of the
environment.

Fig. 2. Players’ results in the Questionnaire and the Gameplay.

Figure 3 shows relatively positive results. However, it is concluded that a
larger volume of samples of each of the factors is needed for more statistical
validation. It can be seen that Conscientiousness is the factor with the worst
results due to its scarce presence in the experience as opposed to Openness as
indicated in the previous table.
8 Villar et al.

Fig. 3. Big Five category average error for each story.

5 Discussion
The current state of the project is still in its early stages and the general appli-
cability of the results is yet to be fully determined. One of the most noteworthy
points is the decision of a prospective assignment of personality values for the
dialogue options, that is, why it was decided to establish some previous values
for their subsequent balancing. In the presented prototype, the values were set
by the authors, but this implies a certain level of noise in the results. Even with
this limitation, the results are positive, but further refinements of the system re-
quire an additional methodology of expert-based assignment of the personality
values.
Based on the hypothesis that personality can be analyzed in a non-intrusive
way through videogames, an underlying hypothesis can be extracted, that there
is a relationship between a player personality outside the videogame and the way
they play within it. To validate it, it is possible to refer to the data extracted
during the second iteration where a clear relationship between the results can
be seen, with an error attributable to this lack of psychological basis in the
assignment.
One of the most direct uses of the proposed method is, in parallel with the
assignment of question values by experts, the use of machine learning algorithms
to, based on the results of the Big Five Inventory questionnaire, assign automat-
ically computed values. The relevance of this approach was tried in this research.
In particular, the consistency of the results were tested by carrying out a Pearson
correlation study between the results from the gameplay and the results from
the Big Five Inventory questionnaire. The results were ρ ≈ 0.17, p ≈ 0.35. These
obtained values lack statistical significance, but it is assumed that this is due
to the relatively low amount of data for the correlation to be significant. The
study was completed with Support Vector Machines which was trained with the
Automating Player Personality Identification 9

options selected by the users in the gameplay session and their results in the Big
Five questionnaire. SVM (precision = 0.15) along with Random Forest Regres-
sors (R2 < 0), confirm that the statistical significance of the data set is still too
low due to the reduced amount of data points.
In short, it has been concluded that, at least while having such a small data
set, it is not yet possible to automate the identification of the personality or
to study the relationship between the two sets of responses through machine
learning.

6 Conclusions and Future Work
The paper has reported on a emergent study on the possibility of automated
player personality profiling in RPG videogames. A plausible methodology for
automatically identification user personality in a non-invasive way through an
interactive experience has also been proposed The results have been reasonably
positive (but yet inconclusive) and they show that there is a possibility of find-
ing a methodology with which to identify, with a reasonable error, the player’s
personality where the use of dialogues and their interactions seem to be of great
importance. Still, the exploratory nature of the research leaves several ways of
improvement as part of the future work.
The most straightforward improvement to the proposed methodology would
be to gather more data and study the effect of applying machine learning tech-
niques to compute the best fit value, for all the dimensions in the Big Five
model, to the questions in the story (as introduced in Section 5). In any case,
regarding the influence of the specific questions in the story, it is clear that the
assigned values in the dialogue interactions must be validated by an expert be-
cause the actual values have been assigned by the researchers and there has not
been any expert involved in the process. Related to this, checking the robustness
of the results and the methodology requires to increase the amount of obtained
samples.
Further research includes the need to test this methodology against other
videogames and thus be able to verify that the hypothesis applies to different
genres. It is believed that this kind of approach is not of application in general,
since it would be hard to apply dialogue-based interaction in a car-racing game,
for instance. However, we believe that the approach is not necessarily restricted
to classic, adventure based roleplaying games. For instance, decisions based dy-
namics not specifically rendered as dialogues could be a source of information as
well.

References
1. Bateman, C., Boon, R.: 21st Century Game Design (2006)
2. Bean, A., Groth-Marnat, G.: Video gamers and personality: A five-factor model
to understand game playing style. Psychology of Popular Media Culture 5(1), 27–
38 (2016). https://doi.org/10.1037/ppm0000025, place: US Publisher: Educational
Publishing Foundation
10 Villar et al.

3. Benet-Martı́nez, V., John, O.P.: Los Cinco Grandes Across Cultures and
Ethnic Groups: Multitrait Multimethod Analyses of the Big Five in
Spanish and English (1998). https://doi.org/10.1037/0022-3514.75.3.729,
https://psycnet.apa.org/record/1998-12057-011
4. Cattell, R.: Confirmation and clarification of primary person-
ality factors (Sep 1947). https://doi.org/10.1007/BF02289253,
https://link.springer.com/article/10.1007/BF02289253
5. Corporation, V.: Steam. https://store.steampowered.com/ (2003)
6. Frazier, S., Riedl, M.: Improving Deep Reinforcement Learning in
Minecraft with Action Advice. arXiv:1908.01007 [cs, stat] (Aug 2019),
http://arxiv.org/abs/1908.01007, arXiv: 1908.01007
7. Goldberg, L.R.: The Structure of Phenotypic Personality Traits (1993).
https://doi.org/10.1037/0003-066X.48.1.26, https://psycnet.apa.org/record/1993-
17546-001
8. Hodent, C.: The Gamer’s Brain (2017). https://doi.org/10.1201/9781315154725
9. John, O.P., Srivastava, S.: Big Five Inventory (Bfi) (1999).
https://doi.org/10.1525/fq.1998.51.4.04a00260
10. Karami, A.B., Encelle, B., Sehaba, K.: A Pilot Study on Estimating Players
Dispositional Profiles from Game Traces Analysis, pp. 1101–1120 (jan 2020).
https://doi.org/10.1007/978-3-030-29516-5 82
11. Mcmahon, N., Wyeth, P., Johnson, D.: Personality and player types in fallout new
vegas (09 2012). https://doi.org/10.1145/2367616.2367632
12. Mojang: Minecraft. https://www.minecraft.net/en-us/ (2009)
13. Myers, I.B.: The Myers-Briggs Type Indicator: Manual (1962) (Apr 2014).
https://doi.org/10.1037/14404-000, https://psycnet.apa.org/record/2013-29682-
000
14. Nintendo: Brain Training. https://www.nintendo.co.uk/Games/Nintendo-
Switch/Dr-Kawashima-s-Brain-Training-for-Nintendo-Switch-1656777.html (2005)
15. Nintendo: Pokémon Mystery Dungeon. https://mysterydungeon.pokemon.com/en-
us/ (2005)
16. Nintendo: Wii Fit. https://www.nintendo.com/wiifit/launch/ (2007)
17. Nouchi, R., Taki, Y., Takeuchi, H., Hashizume, H., Akitsuki, Y., Shige-
mune, Y., Sekiguchi, A., Kotozaki, Y., Tsukiura, T., Yomogida, Y.,
Kawashima, R.: Brain Training Game Improves Executive Functions and
Processing Speed in the Elderly: A Randomized Controlled Trial. PLOS
ONE 7(1), e29676 (Jan 2012). https://doi.org/10.1371/journal.pone.0029676,
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0029676, pub-
lisher: Public Library of Science
18. Parsec: Parsec
19. Staw, B.M., Bell, N.E., Clausen, J.A.: The Dispositional Approach
To Job Attitudes: A Lifetime Longitudinal Test. Administrative Sci-
ence Quarterly 31(1), 56–77 (1986). https://doi.org/10.2307/2392766,
https://www.jstor.org/stable/2392766, publisher: [Sage Publications, Inc.,
Johnson Graduate School of Management, Cornell University]
20. Steve Nebel, S.S., Rey, G.D.: Mining learning and crafting scientific experiments:
A literature review on the use of Minecraft in education and research (apr 2016),
https://psycnet.apa.org/record/2016-19735-025