<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Workshops,
Los Angeles, USA, March</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Using a Serious Game to Teach User-Centered Design</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Domina R. Kiunsi</string-name>
          <email>domina.kiunsi@ju.se</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bruce Ferwerda</string-name>
          <email>bruce.ferwerda@ju.se</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science and Informatics, School of Engineering, Jönköping University</institution>
          ,
          <addr-line>Jönköping</addr-line>
          ,
          <country country="SE">Sweden</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Computer Science and Informatics, School of Engineering, Jönköping University</institution>
          ,
          <addr-line>Jönköping</addr-line>
          ,
          <country country="SE">Sweden</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <volume>20</volume>
      <issue>2019</issue>
      <abstract>
        <p>The development of products with a good user experience requires a thorough understanding of the prospective users' behaviors, preferences, and needs. One of the design approaches that places emphasis on the needs of users is the user-centered design process. However, there is a general resistance in organizations to incorporate user-centered design practices in product development. One influencing factor is that user-centered design is multidisciplinary. Hence, creates challenges to get a mutual understanding and collaboration across diferent stakeholders throughout the development. In this paper we present the results of a serious game prototype that was created to describe user-centered design across stakeholders. Results of the prototype evaluation reveal that it has potential to impart knowledge on concepts related to user-centered design. Additionally, we propose further development of the game by personalizing game elements to increase the efectiveness of learning and player experience.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>CCS CONCEPTS</title>
      <p>• Applied computing → Computer games; • Human-centered
computing → User models.</p>
    </sec>
    <sec id="sec-2">
      <title>INTRODUCTION</title>
      <p>
        Traditional product development has been focusing mainly on the
technical specifications of their products. However, especially with
the advancement of technologies, we may have reached a point
for the average consumer that the perceived advantages of better
specifications (e.g., more computational power) is not significantly
noticeable anymore. For example, the advancement of
semiconductors has already surpassed Moore’s law 1 predictions in 2016 [
        <xref ref-type="bibr" rid="ref42">42</xref>
        ].
1Starting from the 1960’s, Moore’s law predicted a yearly doubling of the number
of transistors producible on a microchip, causing an exponential growth over the
years. The number of transistors is closely connected to the improvement of the chip’s
performance.
      </p>
      <p>IUI Workshops’19, March 20, 2019, Los Angeles, USA
Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted
for private and academic purposes. This volume is published and copyrighted by its
editors.
In conjunction with the increased possibilities of current
technologies, product development is starting to shift from a specification
perspective to a more consumer perspective in which the focus
is on creating intelligent systems that can adapt to and serve the
preferences and needs of its user.</p>
      <p>
        User experience design (UX) as a discipline is concerned with
all the elements that afect a user’s perception with a company’s
product or service [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ]. The International Organization for
Standardization (ISO) has defined UX as a person’s perceptions and
responses that result from the use and/or anticipated use of a
product [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. A product with a good UX has shown to be fundamental
to its success [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ], and has shown to contribute to increase a
company’s competitiveness [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Products with a poor UX can easily
cause a user to seek alternative options [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ]. However, the majority
of the organizations still focus on the technical specifications of
their products instead [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        In order to design a good UX the process needs to be
humancentered, which means that a product should suit the user rather
than making the user suit the product [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. The human-centered
development approach is called user-centered design (UCD) [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ].
UCD is an iterative process during the design and development that
focuses on understanding users and their context to achieve a good
UX [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ] by enhancing factors such as usability and usefulness [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ].
      </p>
      <p>
        Despite the importance of UX, the integration of the UCD
process in the product development is not as widespread as it should be.
One of the core principles of UCD is the close cooperation between
diferent stakeholders. This means a shift from a traditional
linearsequential life cycle (e.g., waterfall method) to involvement of all
stakeholders during all stages of the development (i.e., UCD). One
of the challenges to face is the diverse nature of the stakeholder’s
backgrounds, goals and standpoints, which may not always benefit
the user [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ]. For example, for some stakeholders a complete shift
in perspective is needed to create awareness of the importance
to understand the users (e.g., programmers are traditionally more
inclined to focus on code rather than users). Hence, incorporating
UCD is challenging because the degree of understanding and
adoption of the UCD process is often uneven among stakeholders [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ].
Moreover, it is important that stakeholders take action to maintain
and improve their knowledge about UCD skills, knowledge, and
culture [
        <xref ref-type="bibr" rid="ref41">41</xref>
        ]. To achieve this, there is a need for stakeholders to
learn about UCD and the value it brings to product development.
      </p>
      <p>
        There has been an increased interest in game-based learning.
Games that have an educational or training goal aside of mere
entertainment are known as serious games [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ]. With the wide variety of
education and training needs, there has been an increased interest
in using serious games in the workplace [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ]. Prior studies indicate
that gaming during learning improves eficiency and positively
stimulates achievement of the intended learning outcome [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. As a result,
games are increasingly used in various learning environments. For
example, learning coding through solving puzzles [
        <xref ref-type="bibr" rid="ref27 ref29">27, 29</xref>
        ]. Hence,
a serious game solution may be able to efectively educate about
the UCD process as well.
      </p>
      <p>To test the potential of using a serious game to educate about the
UCD process, we conducted a preliminary study with a prototype
of a serious game. The prototype consisted of a card game that
describes the UCD process and roles of relevant stakeholders in the
UCD process. The goal of the card game was to create awareness
of UCD practices among various stakeholders, enabling them to
experience the process of product development from the perspectives
of the diferent roles involved.</p>
      <p>Our preliminary results indicated that the participants gained
new knowledge relating to UCD through the card game. The
knowledge areas were divided into categories, which include the UCD
design process, UX techniques and the purpose of the organization
roles (i.e., stakeholders) involved. Additionally, this paper proposes
further development of the prototype by creating a personalized
serious game in which game elements can be adjusted based on
organizational and player characteristics to improve learning and
the player experience.
2
2.1</p>
    </sec>
    <sec id="sec-3">
      <title>RELATED WORK</title>
    </sec>
    <sec id="sec-4">
      <title>Serious Games</title>
      <p>
        Serious games are games in which education rather than
entertainment is the main goal [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ]. Although serious games are often
intended for learning, their application extends to other goals such as
acquisition of specific skills [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Serious games are categorized
depending on their characterizing goal, that is the additional goal (e.g.,
learning, training health) apart from entertainment [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Hence,
serious games have been applied in various domains such as health,
military, corporate and education [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        The use of game-based learning and training in organizations
is not new, games have been used to teach concepts and processes
such as marketing, project management and risk management [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
In corporate environments, serious games motivate communication
and collaboration, and are used to get people more involved and
interactive in work related situations [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. An investigation on the
use of serious games in organizations revealed that employees
prefer playing games as a means of learning because they enjoy the
engagement [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. The increased engagement allows serious games to
quickly produce learning results, are cost efective, and can target
multiple people at once [
        <xref ref-type="bibr" rid="ref39">39</xref>
        ].
      </p>
      <p>
        One example of a serious game used for corporate training is
named "COSIGA." The game simulates the collaborative process of
product development among engineers in which the players have
to work together to make the final product [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. COSIGA enables
the players to experience the process of new product development
from the perspectives of the diferent disciplines involved in the
design process and the interactions between these disciplines [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].
Another example of a serious game is "ERPsim." ERPsim is a game
used to teach enterprise recourse planning concepts and ideas. The
main goal of the game was to develop a hands-on understanding of
the concepts underlying enterprise systems [
        <xref ref-type="bibr" rid="ref30">30</xref>
        ].
2.2
      </p>
    </sec>
    <sec id="sec-5">
      <title>Adaptation &amp; Personalization</title>
      <p>
        Tailoring system environments to a user’s behavior, preferences,
and needs are a common practice in fields as recommender systems.
These tailoring strategies usually adhere to a data-driven approach
(e.g., using historical data to predict future behaviors [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]). The
disadvantage of a data-driven approach is that it dependents on
historical data. Hence, fails to facilitate a tailored experience when
lacking historical user data (e.g., for new users). Theory-driven
approaches (e.g., using psychological theory to create user models)
provide opportunities to counteract on the lack of historical data
by using questionnaires or infer user models from external data
sources (e.g., [
        <xref ref-type="bibr" rid="ref12 ref13">12, 13</xref>
        ]). By doing so, theory-driven approaches are
able to facilitate a tailored experience from the start of use.
      </p>
      <p>
        Especially for gaming purposes, tailoring experiences from the
start of use can have a significant impact on players as the first
impression can play an important role in whether players continue
playing the game. Furthermore, individual players within a player
group often exhibit varying attributes such as skills, knowledge
and backgrounds. Hence, tailoring can also contribute to an
increased player acceptance, engagement, and motivation among
heterogeneous player groups [
        <xref ref-type="bibr" rid="ref40">40</xref>
        ].
      </p>
      <p>
        Games in which entertainment is the main purpose rely on player
attributes to tailor the game experience [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Player attributes that are
often adhered to are "gamer types" to indicate the type of player a
user is (see for an overview of gamer type models [
        <xref ref-type="bibr" rid="ref35">35</xref>
        ]). Gamer types
can then be used to adjust the gameplay of the game accordingly.
For example, Orji et al. [
        <xref ref-type="bibr" rid="ref37">37</xref>
        ] increased the eficacy of persuasive
messages of a game by adapting them to diferent gamer types.
      </p>
      <p>
        Serious games on the other hand have the main purpose of
learning and/or training instead of mere entertainment. Hence, the
underlying intention of serious games is to support personal
development instead of gameplay. Therefore, tailoring the gameplay on
gamer types may forgo the intended efect. As the efectiveness of
a serious game is indicated by reaching its learning and/or training
goals [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], adjusting the game elements based on personal
characteristics instead of player attributes may be more efective for learning
and training purposes. For example, Lee &amp; Ferwerda [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ] proposed
to personalize online educational tools by using personality types.
2.3
      </p>
    </sec>
    <sec id="sec-6">
      <title>Challenges of Adapting the UCD process</title>
      <p>
        Because stakeholders may have diferent goals with a product,
there is a general resistance to incorporate UCD practices. There are
several challenges underlying this resistance. UCD practices require
regular communication and collaboration among stakeholders. The
collaboration between diferent stakeholders is a challenge due to
factors as: diference in training, responsibilities, and motivation
in the work place [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. For example, one of the challenges is the
developer mindset. This refers to that some developers are too
focused on the code and thereby loosing sight of other aspects of
the product (e.g., the user’s perspective) [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. One way to alleviate
this problem is by educating developers on the value of UCD.
      </p>
      <p>
        Another challenge is the lack of knowledge and awareness of
the available research methods that could be employed to extract
requirements from end users [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The lack of knowledge and
awareness of available research methods diminishes the efects on the
importance of incorporating UCD.
To increase the willingness to incorporate UCD in the
development process, it is important that the stakeholders understand the
UCD process and be aware of the impact that it can have on the
success of the product [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ]. To overcome the above stated challenges,
we designed a prototype of a serious game to assess the possible
efects it can have in raising awareness of UCD and the efects of it
when incorporating it in the development process.
3
      </p>
    </sec>
    <sec id="sec-7">
      <title>PROTOTYPE DESCRIPTION</title>
      <p>A prototype was created to assess the possible efects of a serious
game for learning UCD in a development process.2 The prototype
consist of a serious card game. The serious card game is a
threeplayer collaborative game which requires the participants to work
together to finish a software product development project while
following the UCD process. The card game consists of a total of 97
cards (example of the cards can be found in Figure 1):
• 4 "project" cards that are describing the type of project the
team will be solving
• 3 "role" cards that are describing the roles of the players
• 4 "phase" cards that are representing the diferent phases in
a UCD process
• 28 "technique" cards that are representing the diferent
methods that the players can apply to complete a UCD phase
• 20 "modifier" cards that are describing the actions the players
need to take during the game
• 30 "capital" cards that provide players buying power to
purchase techniques
• 8 "drawback" cards to add engagement to the game by
warning players of potential failures of the project
3.1</p>
    </sec>
    <sec id="sec-8">
      <title>Game Overview</title>
      <p>The prototype game consists of three common roles in an
organization’s product team: a UX designer, programmer, and business
oficer. Through the project cards the team needs to complete the
given project. In order to complete the project, the team must follow
and complete a UCD process, which is made up of four phases:
research, design, prototyping, and testing. A phase can be completed
by applying the appropriate techniques such as interviews, surveys,
and/or card sorting.</p>
      <p>The team needs to minimally reach 10 points in order to
complete the phase by collecting a corresponding number of technique
points. Techniques are collected by using capital cards. Capital
cards are assigned to each player at the beginning of the game. The
players can use this capital to purchase techniques. The players
play in consecutive order. Hence, only one player can finish a phase
at once. To mimic UCD iterations, the team can encounter modifier
cards during the game. These cards require players to take
particular actions that can change the state of the game. For example, a
modifier can require a player to trade resources with another player.
Modifiers are used once and immediately when picked up.</p>
      <p>
        The games is designed to allow the players to collaborate and
utilize each other’s abilities. Hence, winning is achieved as a team
and not as individual players. To win, the team needs to complete
all the phases of the UCD process, which indicates that they have
2See for a more detailed description of the prototype [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ].
successfully completed their project. The team can loose after
encountering a total of eight drawback cards which can be drawn at
random during the game.
3.2
      </p>
    </sec>
    <sec id="sec-9">
      <title>Game Play</title>
      <p>Each player selects a role upon the start. The team then randomly
selects a project card and places it face up so that it is visible to all
players. Under the project card, the four phase cards are also placed
face up. Each player gets ten capital cards to make a personal deck.
On the table there will be two other decks placed (shufled and
faced down): 1) technique and 2) modifier (+drawback) cards.</p>
      <p>Five technique cards are placed up to start the game. The team
decides which player goes first. Five cards from a personal deck is
drawn on a player’s turn to make a hand. The player then draws a
card from the top from the modifier deck and fulfills the instructed
action. Finally, the player uses the capital cards on their hand to
acquire techniques from the table; each picked up technique needs
to be replaced with a new one from the deck to ensure there are
always five technique cards on the table. The acquired new cards
then add up to a player’s personal deck. If on a player’s turn their
hand contains techniques that are applicable to the current phase,
then they have to be placed faced up in front of the player. This will
enable the team to form a playing strategy, such as trading cards to
complete the phase.
3.3</p>
    </sec>
    <sec id="sec-10">
      <title>Results of the Card Game Assessment</title>
      <p>In order to evaluate the educational goal of the game, three types
of tests were conducted: 1) assessment of the content validity of
the card game to determine whether the constructs represented are
representative for UCD, 2) playtest to assess the functionality of
the game, and 3) subjective player assessment on the educational
goal of the game.</p>
      <p>The content validity assessment was conducted by presenting
the game and administering an open-ended questionnaire to a
practitioner of UCD. The questionnaire contained four open questions
relating to four card types: role, project, phase, and technique cards.
These cards were presented for assessment because they contain
UCD and UX concepts that players are intended to learn during
gameplay. The results showed that the cards contain valid
information relating the UCD process, UX techniques, roles involved, and
types of projects.</p>
      <p>The functionality assessment was conducted through a playtest,
which involved a total of six participants divided in two groups.
All participants indicated that they had no prior knowledge and
awareness of the UCD process. The participants were observed
while playing the game in order to capture the game experience.
The results of the functionality assessment gave indication that the
game objective was well understood by the players. Players showed
to form strategies and engaged in communication and collaboration
to complete the game.</p>
      <p>The learning outcome assessment was conducted by providing
an open-ended questionnaire to the participants at the end of game.
The result of the questionnaire showed that participants gained
new knowledge and awareness about UCD process. One of the
knowledge areas identified was learning the design process; players
indicated having a better idea of the phases needed to develop a new
product, and learned about UX techniques that could be applied
during product development. Finally players specified that they
had a new perspective on the roles involved in UCD.
4</p>
    </sec>
    <sec id="sec-11">
      <title>CONCLUSION</title>
      <p>The results reveal that the game is functional and a good
representation of the UCD process. Players indicated that they were
able to acquire new knowledge and became more aware about the
UCD process. Hence, results indicate that the goal of the serious
game is fulfilled. However, the results presented are only
preliminary. Further research can be carried out with participants from
organizations to assess the impact and learning potential. This
would provide more insight on how the card game performs from
a practitioners’ perspective. Furthermore, during evaluation the
participants only played the game once, it would be interesting
to see whether playing the game over an extended period of time
helps the participants learn and retain more/diferent information.</p>
      <p>Although the prototype reached its goal, it in general lacks
aesthetic appeal. Aesthetic appeal can play an important role for games.
Hence, the graphical elements of the cards could be improved to add
components such as images which would further improve player
immersion and engagement. In the following section we propose
means of personalization and adaptation to further improve the
gameplay, player experience, and learning outcomes.
5
5.1</p>
    </sec>
    <sec id="sec-12">
      <title>PROPOSAL</title>
    </sec>
    <sec id="sec-13">
      <title>Adaptation &amp; Personalization Dimensions</title>
      <p>
        Adaptation and personalization of diferent game elements can have
an influence on the efectiveness of the game. Since the game is
intended for use by diferent organizations, it is important that the
game can adapt to diferent organizational structures. By being able
to adapt the game to diferent organizational structures, the game
is able to increase awareness through the direct applicability of the
respective organization, and thereby also lowering the threshold of
ultimately adapting the UCD process in the organization.
Furthermore, the current prototype supports only three basic roles. The
game can be expanded to include more roles that are involved in
the UCD process (depending on the organizational context). On a
player-level, mapping the positions that players have in the
organization to the roles within the game can have positive efects on the
gameplay as well. Research has shown that self-representation (i.e.,
the ability for a player to identify themselves to the context of the
game) is important to engage and encourage players to continue
playing [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Self-representation can consequently lead to increased
attention and learning of the concepts during the game [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ].
      </p>
      <p>
        The game is not only intended to educate its players about the
UCD process itself, but also to train players on what their position
entails if such a UCD process would be adapted by the organization.
Hence, the game’s intention is also provide opportunities for players
to work on personal development within their roles. Prior studies
have often used personality types to characterize an individual
and to model their behaviors, preferences, and needs (e.g., [
        <xref ref-type="bibr" rid="ref17 ref18">17,
18</xref>
        ]). Hence, personality can conversely also indicate undesirable
characteristics for certain position within the UCD process. The
game can adapt certain gameplay elements to improve on these
characteristics. For example, when a player is shown to be introvert
by nature and the position that they are in requires to be more
outspoken, the game can adapt the gameplay in such a way that
more emphasis will be put on training the player to speak out more.
Personality can also be used to adapt the strategy to teach. For
example, Chen et al. [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] analyzed usage data of online learning
environments and found relationships between users’ personality
traits and diferent strategies users adopt for learning.
      </p>
      <p>
        The acquisition of personality traits can be done in several ways:
explicitly and implicitly. Explicitly is done through the use of
questionnaires. For example, a commonly used questionnaire is the
44-item Big Five Inventory (BFI; [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]) or the Ten Item Personality
Inventory (TIPI; [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]). However, obtrusiveness is a common
problem for explicit acquisition methods by requiring time from the user
to fill in the questionnaire and thereby interrupting the interaction
between the user and the system. Alternatively, the implicit method
can be exploited by using data from external source (e.g., through
the connectedness with social networking services, such as single
sign-on mechanisms). For example, research has shown to be able
infer personality from platforms as Facebook [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], Twitter [
        <xref ref-type="bibr" rid="ref38">38</xref>
        ], and
Instagram [
        <xref ref-type="bibr" rid="ref15 ref16">15, 16</xref>
        ]. The main disadvantage of using the implicit
acquisition method is that it may lack prediction accuracy.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>ISO</surname>
          </string-name>
          9241-
          <fpage>110</fpage>
          :
          <year>2010</year>
          .
          <year>2010</year>
          .
          <article-title>Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems</article-title>
          . Standard. International Organization for Standardization, Geneva, CH.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Carmelo</given-names>
            <surname>Ardito</surname>
          </string-name>
          , Paolo Buono, Danilo Caivano, Maria Francesca Costabile, and
          <string-name>
            <given-names>Rosa</given-names>
            <surname>Lanzilotti</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Investigating and promoting UX practice in industry: An experimental study</article-title>
          .
          <source>International Journal of Human-Computer Studies 72</source>
          ,
          <issue>6</issue>
          (
          <year>2014</year>
          ),
          <fpage>542</fpage>
          -
          <lpage>551</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Aida</given-names>
            <surname>Azadegan</surname>
          </string-name>
          and Johann ckh Riedel.
          <year>2012</year>
          .
          <article-title>Serious games integration in companies: a research and application framework</article-title>
          .
          <source>In Advanced Learning Technologies (ICALT)</source>
          ,
          <source>2012 IEEE 12th International Conference on. IEEE</source>
          ,
          <fpage>485</fpage>
          -
          <lpage>487</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Sander</given-names>
            <surname>Bakkes</surname>
          </string-name>
          , Chek Tien Tan, and
          <string-name>
            <given-names>Yusuf</given-names>
            <surname>Pisan</surname>
          </string-name>
          .
          <year>2012</year>
          .
          <article-title>Personalised gaming: a motivation and overview of literature</article-title>
          .
          <source>In Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the System. ACM</source>
          ,
          <volume>4</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Dirk</given-names>
            <surname>Basten</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Gamification</article-title>
          .
          <source>IEEE Software 34</source>
          ,
          <issue>5</issue>
          (
          <year>2017</year>
          ),
          <fpage>76</fpage>
          -
          <lpage>81</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Kathy</given-names>
            <surname>Baxter</surname>
          </string-name>
          , Catherine Courage, and
          <string-name>
            <given-names>Kelly</given-names>
            <surname>Caine</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Understanding your users: A practical guide to user research methods</article-title>
          . Morgan Kaufmann.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Andrej</given-names>
            <surname>Jerman</surname>
          </string-name>
          <string-name>
            <surname>Blažič</surname>
          </string-name>
          , Primož Cigoj, and Borka Jerman Blažič.
          <year>2016</year>
          .
          <article-title>Serious game design for digital forensics training</article-title>
          .
          <source>In Digital Information Processing, Data Mining, and Wireless Communications (DIPDMWC)</source>
          ,
          <source>2016 Third International Conference on. IEEE</source>
          ,
          <fpage>211</fpage>
          -
          <lpage>215</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Fran</surname>
            <given-names>C Blumberg</given-names>
          </string-name>
          , Debby E Almonte, Jared S Anthony, and
          <string-name>
            <given-names>Naoko</given-names>
            <surname>Hashimoto</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Serious games: What are they? What do they do? Why should we play them</article-title>
          .
          <source>The Oxford handbook of media psychology</source>
          (
          <year>2013</year>
          ),
          <fpage>334</fpage>
          -
          <lpage>351</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Marc</given-names>
            <surname>Busch</surname>
          </string-name>
          , Elke Mattheiss, Rita Orji, Andrzej Marczewski, Wolfgang Hochleitner, Michael Lankes, Lennart E Nacke, and
          <string-name>
            <given-names>Manfred</given-names>
            <surname>Tscheligi</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Personalization in serious and persuasive games and gamified interactions</article-title>
          .
          <source>In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. ACM</source>
          ,
          <volume>811</volume>
          -
          <fpage>816</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Guanliang</surname>
            <given-names>Chen</given-names>
          </string-name>
          , Dan Davis,
          <string-name>
            <given-names>Claudia</given-names>
            <surname>Hauf</surname>
          </string-name>
          , and
          <string-name>
            <surname>Geert-Jan Houben</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>On the impact of personality in massive open online learning</article-title>
          .
          <source>In Conference on user modeling adaptation and personalization. ACM</source>
          ,
          <volume>121</volume>
          -
          <fpage>130</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Ralf</surname>
            <given-names>Dörner</given-names>
          </string-name>
          , Stefan Göbel, Wolfgang Efelsberg, and
          <string-name>
            <given-names>Josef</given-names>
            <surname>Wiemeyer</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Serious games: foundations, concepts and practice</article-title>
          . Springer.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>Bruce</given-names>
            <surname>Ferwerda</surname>
          </string-name>
          and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Schedl</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Enhancing Music Recommender Systems with Personality Information and Emotional States: A Proposal.</article-title>
          .
          <source>In UMAP Workshops.</source>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>Bruce</given-names>
            <surname>Ferwerda</surname>
          </string-name>
          and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Schedl</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Personality-based user modeling for music recommender systems</article-title>
          .
          <source>In Joint European Conference on Machine Learning and Knowledge Discovery in Databases</source>
          . Springer,
          <fpage>254</fpage>
          -
          <lpage>257</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Bruce</surname>
            <given-names>Ferwerda</given-names>
          </string-name>
          , Markus Schedl, and
          <string-name>
            <given-names>Marko</given-names>
            <surname>Tkalcic</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Personality traits and the relationship with (non-) disclosure behavior on facebook</article-title>
          .
          <source>In Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee</source>
          ,
          <fpage>565</fpage>
          -
          <lpage>568</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>Bruce</given-names>
            <surname>Ferwerda</surname>
          </string-name>
          and
          <string-name>
            <given-names>Marko</given-names>
            <surname>Tkalcic</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>Predicting Users' Personality from Instagram Pictures: Using Visual and/or Content Features?</article-title>
          .
          <source>In The 26th Conference on User Modeling, Adaptation and Personalization</source>
          ,
          <string-name>
            <surname>Singapore</surname>
          </string-name>
          (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Bruce</given-names>
            <surname>Ferwerda</surname>
          </string-name>
          and
          <string-name>
            <given-names>Marko</given-names>
            <surname>Tkalcic</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>You Are What You Post: What the Content of Instagram Pictures Tells About Users' Personality. In The 23rd International on Intelligent User Interfaces</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Bruce</surname>
            <given-names>Ferwerda</given-names>
          </string-name>
          , Marko Tkalcic, and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Schedl</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Personality Traits and Music Genre Preferences: How Music Taste Varies Over Age Groups</article-title>
          .
          <source>In Proceedings of the 1st Workshop on Temporal Reasoning in Recommender Systems (RecTemp) at the 11th ACM Conference on Recommender Systems, Como, August</source>
          <volume>31</volume>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Bruce</surname>
            <given-names>Ferwerda</given-names>
          </string-name>
          , Marko Tkalcic, and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Schedl</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Personality Traits and Music Genres: What Do People Prefer to Listen To?</article-title>
          .
          <source>In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization. ACM</source>
          ,
          <volume>285</volume>
          -
          <fpage>288</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Jesse</surname>
            <given-names>James</given-names>
          </string-name>
          <string-name>
            <surname>Garrett</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Elements of user experience, the: user-centered design for the web and beyond</article-title>
          .
          <source>Pearson Education.</source>
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Samuel</surname>
            <given-names>D</given-names>
          </string-name>
          <string-name>
            <surname>Gosling</surname>
          </string-name>
          ,
          <string-name>
            <surname>Peter J Rentfrow</surname>
          </string-name>
          , and
          <string-name>
            <surname>William</surname>
            <given-names>B Swann</given-names>
          </string-name>
          <string-name>
            <surname>Jr</surname>
          </string-name>
          .
          <year>2003</year>
          .
          <article-title>A very brief measure of the Big-Five personality domains</article-title>
          .
          <source>Journal of Research in personality 37</source>
          ,
          <issue>6</issue>
          (
          <year>2003</year>
          ),
          <fpage>504</fpage>
          -
          <lpage>528</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <source>[21] Jannicke Baalsrud Hauge and Johann CKH Riedel</source>
          .
          <year>2012</year>
          .
          <article-title>Evaluation of simulation games for teaching engineering and manufacturing</article-title>
          .
          <source>Procedia Computer Science</source>
          <volume>15</volume>
          (
          <year>2012</year>
          ),
          <fpage>210</fpage>
          -
          <lpage>220</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Oliver</surname>
            <given-names>P John</given-names>
          </string-name>
          , Eileen M Donahue, and Robert L Kentle.
          <year>1991</year>
          .
          <article-title>The big five inventoryâĂŤversions 4a and</article-title>
          54.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Pariya</surname>
            <given-names>Kashfi</given-names>
          </string-name>
          , Agneta Nilsson, and
          <string-name>
            <given-names>Robert</given-names>
            <surname>Feldt</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Integrating User eXperience practices into software development processes: implications of the UX characteristics</article-title>
          .
          <source>PeerJ Computer Science</source>
          <volume>3</volume>
          (
          <year>2017</year>
          ),
          <year>e130</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>Domina</given-names>
            <surname>Kiunsi</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>Serious gaming as a tool to describe a user-centred design process</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Yehuda</surname>
            <given-names>Koren</given-names>
          </string-name>
          , Robert Bell, and
          <string-name>
            <given-names>Chris</given-names>
            <surname>Volinsky</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Matrix Factorization Techniques for Recommender Systems</article-title>
          . IEEE Computer (
          <year>2009</year>
          ),
          <fpage>42</fpage>
          -
          <lpage>49</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>Mike</given-names>
            <surname>Kuniavsky</surname>
          </string-name>
          .
          <year>2003</year>
          .
          <article-title>Observing the user experience: a practitioner's guide to user research</article-title>
          . Elsevier.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Michael</surname>
            <given-names>J</given-names>
          </string-name>
          <string-name>
            <surname>Lee</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>How can a social debugging game efectively teach computer programming concepts?</article-title>
          .
          <source>In Proceedings of the ninth annual international ACM conference on International computing education research. ACM</source>
          ,
          <volume>181</volume>
          -
          <fpage>182</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Michael</surname>
            <given-names>J</given-names>
          </string-name>
          <string-name>
            <surname>Lee</surname>
            and
            <given-names>Bruce</given-names>
          </string-name>
          <string-name>
            <surname>Ferwerda</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Personalizing online educational tools</article-title>
          .
          <source>In Proceedings of the 2017 ACM Workshop on Theory-Informed User Modeling for Tailoring and Personalizing Interfaces. ACM</source>
          ,
          <volume>27</volume>
          -
          <fpage>30</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Michael</surname>
            <given-names>J</given-names>
          </string-name>
          <string-name>
            <surname>Lee and Andrew J Ko</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Comparing the efectiveness of online learning approaches on CS1 learning outcomes</article-title>
          .
          <source>In Proceedings of the eleventh annual international conference on international computing education research. ACM</source>
          ,
          <volume>237</volume>
          -
          <fpage>246</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Pierre-Majorique Léger</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>Using a simulation game approach to teach ERP concepts</article-title>
          .
          <source>HEC Montréal</source>
          ,
          <article-title>Groupe de recherche en systèmes d'information.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>Sohye</given-names>
            <surname>Lim</surname>
          </string-name>
          and
          <string-name>
            <given-names>Byron</given-names>
            <surname>Reeves</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player</article-title>
          .
          <source>International Journal of Human-Computer Studies 68</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>2</lpage>
          (
          <year>2010</year>
          ),
          <fpage>57</fpage>
          -
          <lpage>68</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Ji-Ye</surname>
            <given-names>Mao</given-names>
          </string-name>
          , Karel Vredenburg, Paul W Smith,
          <string-name>
            <given-names>and Tom</given-names>
            <surname>Carey</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>The state of user-centered design practice</article-title>
          .
          <source>Commun. ACM 48</source>
          ,
          <issue>3</issue>
          (
          <year>2005</year>
          ),
          <fpage>105</fpage>
          -
          <lpage>109</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Amir</surname>
            <given-names>Matallaoui</given-names>
          </string-name>
          , Nicolai Hanner, and
          <string-name>
            <given-names>Rüdiger</given-names>
            <surname>Zarnekow</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Introduction to gamification: Foundation and underlying theories</article-title>
          .
          <source>In Gamification</source>
          . Springer,
          <fpage>3</fpage>
          -
          <lpage>18</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>David R Michael and Sandra L Chen</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>Serious games: Games that educate, train, and inform</article-title>
          . Muska &amp; Lipman/Premier-Trade.
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Lennart</surname>
            <given-names>E Nacke</given-names>
          </string-name>
          , Chris Bateman, and Regan L Mandryk.
          <year>2014</year>
          .
          <article-title>BrainHex: A neurobiological gamer typology survey</article-title>
          .
          <source>Entertainment computing 5</source>
          ,
          <issue>1</issue>
          (
          <year>2014</year>
          ),
          <fpage>55</fpage>
          -
          <lpage>62</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>Don</given-names>
            <surname>Norman</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>The design of everyday things: Revised and expanded edition</article-title>
          . Constellation.
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <surname>Rita</surname>
            <given-names>Orji</given-names>
          </string-name>
          , Julita Vassileva, and Regan L Mandryk.
          <year>2014</year>
          .
          <article-title>Modeling the eficacy of persuasive strategies for diferent gamer types in serious games for health</article-title>
          .
          <source>User Modeling and User-Adapted Interaction 24</source>
          ,
          <issue>5</issue>
          (
          <year>2014</year>
          ),
          <fpage>453</fpage>
          -
          <lpage>498</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Daniele</surname>
            <given-names>Quercia</given-names>
          </string-name>
          , Michal Kosinski, David Stillwell,
          <string-name>
            <given-names>and Jon</given-names>
            <surname>Crowcroft</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Our twitter profiles, our selves: Predicting personality with twitter</article-title>
          .
          <source>In Privacy, Security, Risk and Trust (PASSAT)</source>
          and
          <source>2011 IEEE Third Inernational Conference on Social Computing (SocialCom)</source>
          ,
          <source>2011 IEEE Third International Conference on. IEEE</source>
          ,
          <fpage>180</fpage>
          -
          <lpage>185</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Johann</surname>
            <given-names>CKH Riedel</given-names>
          </string-name>
          , Yanan Feng, and
          <string-name>
            <given-names>Aida</given-names>
            <surname>Azadegan</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Serious Games Adoption in Organizations-An Exploratory Analysis</article-title>
          .
          <source>In European Conference on Technology Enhanced Learning</source>
          . Springer,
          <fpage>508</fpage>
          -
          <lpage>513</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <given-names>Alexander</given-names>
            <surname>Streicher</surname>
          </string-name>
          and
          <string-name>
            <given-names>Jan D</given-names>
            <surname>Smeddinck</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Personalized and adaptive serious games</article-title>
          .
          <source>In Entertainment Computing and Serious Games</source>
          . Springer,
          <fpage>332</fpage>
          -
          <lpage>377</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Giorgio</surname>
            <given-names>Venturi</given-names>
          </string-name>
          , Jimmy Troost, and
          <string-name>
            <given-names>TImo</given-names>
            <surname>Jokela</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>People, organizations, and processes: An inquiry into the adoption of user-centered design in industry</article-title>
          .
          <source>International Journal of Human-Computer Interaction 21</source>
          ,
          <issue>2</issue>
          (
          <year>2006</year>
          ),
          <fpage>219</fpage>
          -
          <lpage>238</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          [42]
          <string-name>
            <given-names>M</given-names>
            <surname>Mitchell</surname>
          </string-name>
          <article-title>Waldrop</article-title>
          .
          <year>2016</year>
          .
          <article-title>The chips are down for Moore's law</article-title>
          .
          <source>Nature News</source>
          <volume>530</volume>
          ,
          <issue>7589</issue>
          (
          <year>2016</year>
          ),
          <fpage>144</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>