<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Reflection on the Octalysis framework as a design and evaluation tool</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Philip Weber</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Laura Grönewald</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Thomas Ludwig</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Cyber-Physical Systems, University of Siegen</institution>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <fpage>75</fpage>
      <lpage>84</lpage>
      <abstract>
        <p>The Octalysis framework is a gamification framework used for the design and evaluation of “human-focused” systems. Although several practitioners have applied it within their daily work, only a few academic articles have reflected on its applicability. With this study, we present how and where the framework is currently applied based on a large-scale literature study and reflect on the potentials and obstacles of using it within a Human Computer Interaction (HCI) master's class. Our empirical findings show that the use of the Octalysis framework is often simplified and can also be overwhelming. The results further reveal that the framework itself can be helpful in the creation and evaluation of concepts, especially when extensive user research is not possible (e.g., due to time constraints). We contribute to the field of gamification by critically reflecting on the use of the Octalysis framework.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Octalysis framework</kwd>
        <kwd>gamification frameworks</kwd>
        <kwd>human computer interaction</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Even if there is no agreed definition of the term
gamification, the most popular agreed definition
is ”the use of game design elements in non-game
contexts” [
        <xref ref-type="bibr" rid="ref21 ref93">7</xref>
        ]. A great part of the academic work
is aimed at making the effects of this “use”
measurable (e.g.: through increased user
satisfaction, retention rates, productivity,
engagement) and thus informing the selection of
adequate design elements. It is common for
practitioners to use popular gamification
frameworks such as the “Octalysis Framework”
[
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ] or the “Playful Experience Framework”
(PLEX) [22] to decide which design elements are
suitable for which kind of use case [28].
      </p>
      <p>With this paper we would like to focus more
on the design process of gamification using such
frameworks and investigate how one of these
frameworks translates into the “real world” as
well as which impact such a gamification
framework has on the design practices. Since we
6th International GamiFIN Conference 2022 (GamiFIN 2022),
April 26-29 2022, Finland
EMAIL: philip.weber@uni-siegen.de (A. 1);
laura.groenewald@uni-siegen.de (A. 2);
thomas.ludwig@unisiegen.de (A. 3)
ORCID: 0000-0003-4020-6321 (A. 3)
️© 2022 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <p>CEUR Workshop Proceedings (CEUR-WS.org)
see that this particular discourse currently plays a
rather underrepresented role in gamification
research, we would like to pave the way for
fostering this discussion. We therefore examine
the Octalysis Framework as it gains a lot of
attention among practitioners [28]. We ourselves
have already used the framework a few times in
projects and thus gained hands-on experience
with it.</p>
      <p>Although the Octalysis framework is attracting
attention and adoption among practitioners
[27,40], we are not aware of any academic study
that has reflected on its usage and on its real-world
applicability as well as impact on the design of
gamified systems. We consider this an important
research gap that we would like to address with
this paper.
2.</p>
    </sec>
    <sec id="sec-2">
      <title>Octalysis framework</title>
      <p>
        The Octalysis framework is a gamification
framework developed by Yu-kai Chou [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ]. The
framework is used to design gamified systems and
evaluate applications in terms of their
motivational drivers, referred to as core drives
(CD) in the framework [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ]. The framework
consists of eight core drives: (1) Epic Meaning
and Calling, (2) Development and
Accomplishment, (3) Empowerment of Creativity
and Feedback, (4) Ownership and Possession, (5)
Social Influence and Relatedness, (6) Scarcity and
Impatience, (7) Unpredictability and Curiosity,
and (8) Loss and Avoidance.
      </p>
      <p>Chou identified and collected more than 100
game design elements that he mapped to one or
more core drives to increase the motivational
affordances of specific core drives. As an
example, the use of “Easter Eggs/Sudden
Rewards” addresses and increases the core drive
of “Unpredictability and Curiosity” (CD7). The
strength of the motivation based on the individual
core is measured by the Octalysis Score ranging
from 0 to 10, where 0 indicates that the core drive
is not addressed within a system and 10 means
that there is no potential for further improving the
core drive. This results in differently shaped
octagons as the height of the Octalysis Scores can
be illustrated for each core drive (Figure 1). There
are many other nuances that go beyond the scope
of this paper (e.g., a distinction between
extrinsic/intrinsic motivation within the
individual core drives, black and white hat
gamification, implicit and explicit gamification,
and a very briefly covered ninth “hidden” core
drive called “sensation”).</p>
      <p>
        The examination of the eight core drives can
be broken down into four phases of a user journey
(Discovery, Onboarding, Scaffolding, Endgame)
which are labeled as Octalysis Level 2. In
addition, if different needs of diverse target
groups are considered, Octalysis Level 3 can also
be used, in which the four phases are linked to an
individual user group [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ].
3.
      </p>
    </sec>
    <sec id="sec-3">
      <title>Methodology</title>
      <p>To investigate the Octalysis framework from a
scientific point of view, we have chosen a
twostep methodology. First, we present a large-scale
literature review on the applicability of the
Octalysis framework within different domains.
Here, we will outline why the framework was
used (reasons for using the model) and summarize
the scattered articles on its usage. To better reflect
on the use of the framework, we then introduced
the Octalysis framework to a Human Computer
Interaction (HCI) master’s course at our home
university. The ten students of this course
evaluated existing apps based on the Octalysis
framework and afterwards designed and justified
their own gamified app experience. Afterwards,
students shared their experiences with the
framework in a focus group before handing in
written reports, where we asked them to reflect on
the role of the Octalysis framework during their
design processes. Based on the students’
experiences and our assessment of their created
designs, we discuss the benefits and drawbacks of
using the Octalysis framework.</p>
    </sec>
    <sec id="sec-4">
      <title>3.1. Literature review</title>
      <p>To obtain a complete picture of the current use
of the Octalysis framework, we conducted a
largescale literature review (Figure 2). For this
purpose, we examined in total 344 results found
in Google Scholar using the search term
“Octalysis framework” on October 1, 2020. After
excluding results that did not met minimum
academic standards (e.g., presentation slides), 280
results remained. We further excluded results that
we either could not understand due to language
barriers or that only mentioned the Octalysis
framework in passing (e.g., a mere explanation of
the framework), resulting in 101 papers. As a final
restriction to increase scientific rigor, we
excluded undergraduate theses, narrowing down
our analysis to 67 academic publications (and an
additional four doctoral theses). We repeated the
same steps again on November 10, 2021 for
recently published papers, increasing the total
number of papers by 22 to 366 and the relevant
papers by seven to a total of 78 publications.</p>
      <p>The 78 papers were evaluated and analyzed by
the group of authors who collectively coded the
first papers, thereby inductively establishing a
coding system by mutual agreement. This was
followed by individual coding of the remaining
papers and the subsequent reconciliation of the
results.</p>
    </sec>
    <sec id="sec-5">
      <title>3.2. Classroom study</title>
      <p>To test the applicability, learnability, and
possible difficulties involved in the use of the
Octalysis framework, we introduced the
framework to a group of 10 master’s students
(seven males, three females) as part of an HCI
master’s course in summer 2020. The course, held
entirely in English, was an elective in the
respective HCI curriculum. Two of the authors
prepared, taught, and supervised the course,
which was held from mid-April of 2020 to the end
of September 2020. Due to the ongoing Covid
19pandemic, the course was held remotely.</p>
      <p>Participating students had varying previous
experiences due to their different academic
backgrounds. Most of the students came from the
fields of psychology, interaction design, and
computer science. While some of the students had
a basic knowledge of gamification as well as
motivation and design models, none knew or had
used the Octalysis framework previously. Most of
them were aware of the difference between
intrinsic and extrinsic motivation and had a
general knowledge of the self-determination
theory of Ryan and Deci [36]. Thus, the students
could be described as “young professionals” who
had already been exposed to related motivational
and HCI topics due to their interest and expertise
but who were not yet familiar with the Octalysis
framework.</p>
      <p>
        The course consisted of several introductory
sessions to explain the Octalysis framework,
particularly the different core drives, levels 2 and
3 of the framework, and most of the major game
techniques [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ]. The students were then given the
task of using the framework as a tool of analysis
to examine various food-related apps and identify
existing design patterns and potential design
spaces. The results of the analyses were discussed
by the entire group and were then made accessible
to all course participants.
      </p>
      <p>
        Based on these discussions, all students had to
develop their own idea, flesh out a concept with a
UX story [33], create a video prototype [23], and
justify the key design decisions using the
Octalysis framework [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ]. Subsequently, the
students presented the results and submitted a
project report, which documented the project
work, outlined further developments of an idea,
and, in particular, reflected on the use of the
Octalysis framework. In the last meeting, the
students discussed their experiences and opinions
about the framework in a focus group moderated
by us.
      </p>
      <p>This exercise was audio-recorded, and key
passages were partially transcribed. The verbatim
quotes used in this paper are attributed to the
students (P01 to P10) and are taken from the
project reports as well as from these
documentations. The resulting material was
iteratively coded and verified in several meetings
among the authors.
4.</p>
    </sec>
    <sec id="sec-6">
      <title>Results of the literature study</title>
      <p>To understand in detail how the Octalysis
framework was previously utilized, we identified
in the literature the different application domains
in which the Octalysis framework was used
(Table 1). The framework is predominantly used
in the context of gamified educational
experiences. Examples are Rohr and Fischer [35],
who used the framework to generate
qualitativeempirical user requirements from students to
better understand the target group and derive
appropriate requirements for a gamified
elearning platform and Mårell-Olsson [25], who
introduced the framework to university students,
enabling them to act as cocreators to design
gamified teaching activities for middle-school
students.</p>
      <p>
        Some of the papers we analyzed did not focus
on a specific domain but rather investigated the
Octalysis framework and gamification from a
rather holistic perspective. Tondello et al. [38]
used the framework as a starting point for the
development of their gameful design heuristics
and compared the Octalysis framework with other
well-known frameworks and methods, such as the
“Hexad” framework by Marczewski [24], the
“Kaleidoscope of effective gamification" by
Kappen and Nacke [
        <xref ref-type="bibr" rid="ref73">19</xref>
        ], the motivational design
lenses by Deterding [
        <xref ref-type="bibr" rid="ref105 ref110">6</xref>
        ], the six different
motivational dimensions for meaningful
gamification by Nicholson [29], or the “Super
Better” design method by McGonigal [27].
Morschheuser et al. [28] interviewed 25
gamification experts, in which four experts stated
that they made use of the Octalysis framework and
considered this and other gamification
frameworks to be important in their ideation
phase.
      </p>
      <p>Seven papers were related to the domain of
healthcare, where the Octalysis framework was
used, for example, to increase the motivation of
patients with physical disabilities to perform
exercises [9,10]. Six studies were in the area of
marketing, e-commerce, and customer loyalty, an
example of which is the use of the Octalysis
framework by Fathian et al. [13] to develop a
model that maps the relationship between game
mechanics and elements of customer loyalty.</p>
      <p>
        Additionally, some papers address topics of
organizational management. For instance,
Ellenberger et al. [12] used the Octalysis
framework to analyze the integration of
gamification in a company and its influence on
company culture. While Korn et al. [21] used the
Octalysis framework as a foundation to develop a
model for the description and analysis of
recrutainment applications, Sanchez-Gordón et al.
[
        <xref ref-type="bibr" rid="ref60">37</xref>
        ] investigated the compatibility of the
Octalysis framework with the human factors of
ISO 10018.
      </p>
      <p>Other specific topic areas were also identified,
including social media [42], conversational agents
[8], and promotion of pro-environmental behavior
(e.g., [32]), where only one to three publications
existed that were grouped under the label of “other
domains”.</p>
    </sec>
    <sec id="sec-7">
      <title>4.1. Reasons for using the Octalysis framework</title>
      <p>We analyzed how the Octalysis framework
was approached and deployed within the
reviewed studies. We discovered four primary
practices (Table 2). Since the practices are not
mutually exclusive, we marked in a few cases
papers with more than one practice. In most cases,
we identified a paper as addressing only one of the
practices.</p>
      <p>
        The most common practice is using the
Octalysis framework for designing new ideas,
systems, and applications and justifying the
selection of game elements by assigning them to
core drives. The ideation process usually takes
place within the author’s working group (e.g.,
[
        <xref ref-type="bibr" rid="ref13 ref83">5,9,15</xref>
        ]); there was only one exception to the
understanding that the design process is explicitly
a codesign process [25].
      </p>
      <p>
        Another common practice is to use the
Octalysis framework to align it with other models,
frameworks, and methods, usually with the intent
of deriving new methods (e.g., “gameful design
heuristics” [38], or “gamification characteristics
measurement scale for mobile application users”
[
        <xref ref-type="bibr" rid="ref44 ref98">1</xref>
        ]), or more context-specific models (e.g., for
recrutainment applications [21], customer loyalty
measures [13], or the smart home context [32]). It
is striking here that either new approaches are
developed based on the framework or that it is
compared with existing models. However, no
reviewed paper dealt with the empirical validation
with (sub)areas of the framework.
      </p>
      <p>
        Furthermore, the framework was used to
analyze existing applications, such as sports apps
[40] or video-sharing platforms [42]. More rarely,
the Octalysis framework was used to obtain
insights about users. Here, the most common
approach was to create questionnaires on the basis
of the framework and use them to conduct
quantitative surveys (e.g., [
        <xref ref-type="bibr" rid="ref44 ref98">1,26</xref>
        ]). Less
frequently, it was used for qualitative studies,
such as focus groups or interviews (e.g., [14]).
      </p>
    </sec>
    <sec id="sec-8">
      <title>4.2. Reflections on usage</title>
      <p>The literature shows that reflections on the use
of the Octalysis framework as a methodology, the
reasons for its selection, or remarks on the
exclusion of the framework are rare. Only 13 of
the 78 publications mentioned those reasons; but
often only with a sentence or short paragraph. One
reason to use the Octalysis framework could be its
good reputation among gamification practitioners
[28], as it bridges psychology and game elements
[3]. Recently, the framework has also been used
in academic publications, which Karać and
Stabauer [20] mention as a reason for their choice.</p>
      <p>
        On the other hand, the framework is
sometimes criticized for not being
contextspecific. Korn et al. [21], for example, see a lack
of “‘serious’ business-related components” that
allow a proper evaluation of existing
recrutainment systems. Yfantis and Tseles [
        <xref ref-type="bibr" rid="ref57">41</xref>
        ]
argue that in the public sector, it might be useful
to consider additional right brain core drives to
further boost intrinsic motivation.
      </p>
      <p>Another criticism is the strong influence of the
subjective perception when assessing core drives
and game elements [3,12,30], which depends on
personal experiences and intuition [3], making
any objective comparisons between gamification
approaches based on the Octalysis Gamification
Score rather difficult [12]. Similarly, it is not
possible to estimate the extent to which a design
element has an impact on the fulfillment of a
single core drive [3]. In addition, the framework
could lead to implementing rather “fashionable”
game elements without a prior understanding of
customer needs through appropriate user research
[20].</p>
      <p>
        Broer [
        <xref ref-type="bibr" rid="ref53">2</xref>
        ] criticizes the Octalysis framework
for its lack of scientific evaluation, and Tondello
[39] views gamification frameworks, including
the Octalysis framework, as unsuitable for
evaluating gameful applications due to the high
level of needed familiarity with the frameworks.
      </p>
    </sec>
    <sec id="sec-9">
      <title>5. Insights from the classroom study</title>
      <p>Within the classroom study, students identified
and communicated several advantages and
disadvantages of using the Octalysis framework.
In addition, we as lecturers made some interesting
observations, which will be subsequently shared.</p>
      <p>The overarching design principle of the
Octalysis framework that puts human needs above
functional aspects was appreciated (P8). Through
analysis of the three different Octalysis levels
(Levels 1, 2, and 3), one gets a new perspective on
human motivation and the user journey (P8). The
ideation phase appeared more structured; thus,
ideas could be brought up more easily (P1, P5).
The many different game techniques were
particularly helpful in generating ideas (P2, P8)
and were easy to understand through real-life
scenarios that seemed to be transferable to many
contexts (P5, P10). In addition, P4 and P10 said
that the Octalysis framework made them aware of
blackhat gamification techniques (such as “doom
scrolling” [31]) and that they want to use them
more consciously in their future designs. P5
describes the framework as a solid guideline that
can help inspire new ideas. In general, most
students saw the usefulness of the framework for
developing new ideas instead of for evaluating
existing concepts and ideas. However, P7
expressed a positive opinion of the framework for
evaluation purposes, emphasizing the objectivity
it provides: “Designing such an experience was
considered to be subjective but with the Octalysis
framework you can somehow evaluate the
application.”</p>
      <p>However, students (P5, P8) criticized the
apparent objectivity of the framework when
evaluating other concepts. P5 considered that
even similar concepts could not be meaningfully
evaluated and compared with the framework. This
is due to the subjective nature of quantifying the
individual design elements in a single
gamification score, which was criticized by P8.</p>
      <p>In relation to the conceptualization of
applications using the Octalysis framework,
students found that it is too general and does not
sufficiently consider the design context (P2, P5).
P2, for example, would still prefer to use the 6D
framework [11] in the business context because it
is specifically designed to be applied in this
domain. Other students (P8, P10) expressed their
fear that that the Octalysis framework could make
them blind to key design challenges. In particular,
CD5, where intrinsic elements (social relatedness)
and extrinsic elements (social pressure) are
combined under one core drive (“Social Influence
and Relatedness”), is a big concern as it can lead
to an oversimplification of human motivations.
“Putting two very different types of motivation
into the same category seems to oversimplify the
nature of motivation and could lead to unintended
user experiences if a distinction is not drawn
during the design process” (P8). Furthermore, P5
notes that the framework did not help her select an
appropriate number of game techniques or to link
them in a meaningful way, nor did it provide much
particular guidance on how to design and
implement these elements.</p>
      <p>Half of the students (P2, P4, P6, P8, P10)
complained that the user perspective is not placed
in a key position within the framework when
designing with it. In particular, a “practice-based
user research study” is missing (P6); this could
have been achieved by conducting interviews
beforehand (P2, P4) or by evaluating the
prototype through usability tests (P6) and
improving it by other types of user feedback (P8,
P10).</p>
      <p>One of the biggest criticisms mentioned by the
students (P5, P8, P10) is the lack of scientific
grounding of the Octalysis framework. This is
expressed by a statement of P5: “Scientific
approaches and references are seldom used
within the framework.” P10 criticized that many
passages in the book are either anecdotal or barely
cited. For example, P10 sees strong similarities
with the different types of “nudges” described by
Thaler and Sunstein [34] and the game techniques
of the Octalysis framework. P8 commented on the
individual core drives and considers core drives 2,
5, and 7 to be particularly problematic for various
reasons, referring to the works of Ryan and Deci
[36] and Hassenzahl et al. [18], which partly
overlap with the Octalysis framework.</p>
      <p>Despite the criticisms mentioned, most
students concluded that the Octalysis framework
helped them better understand human motivation
in their design practice. Even P8, who raised some
concerns, concluded that: “Ultimately, however,
the Octalysis framework can be a helpful tool, if
used with critical reflection and in combination
with motivation frameworks that are scientifically
more grounded.” (P8)</p>
      <p>From the role of the lecturers and with a close
look at the concepts and prototypes that were
created, we made two complementary
observations. It is striking that most of the
students created their concepts only with the game
techniques mentioned in the framework (even
though we deliberately said that the core drives
are more relevant and that new game techniques
and ideas may also be explored and innovated
within the course). In the end, this could lead to a
decrease in creativity and diversity in future
designs, perhaps hindering the development of
novel approaches and thinking out of the box.</p>
      <p>Although we were satisfied with the quality
and elaboration of the work, it was noticeable,
especially in the weaker projects, that students
tried to incorporate as many game techniques as
possible into their own concept, that way hoping
to create an engaging experience. As a result, the
individual elements of the concept sometimes
seemed somewhat detached. One specific
example is the game technique of mentorships,
which was used in many concepts but was rarely
conceived in a context-specific way or
meaningfully integrated into the overall systems.
The idea that more design elements will lead to
more engagement might be a misconception,
especially if the additional elements are poorly
thought out.</p>
    </sec>
    <sec id="sec-10">
      <title>6. Discussion</title>
      <p>Our classroom study confirmed all the
mentioned criticisms from the literature review
and portrayed them in greater detail. Only the
statement that the framework is difficult to use for
the evaluation of gameful applications [39] was
not apparent since all of the students were able to
communicate and justify their concepts through
the lens of the Octalysis Framework very well.
However, this was probably due to the rather
experienced participants and the teaching format,
which entailed several sessions to introduce the
Octalysis framework to the students.</p>
      <p>In addition to the existing literature, we
identified some advantages and disadvantages of
using the framework. We see it as problematic to
become less innovative if designers only follow
the given structure of the Octalysis framework for
a project and does not think about other new game
elements that are not (yet) included in the
Octalysis framework. Here, other creative
techniques could continue to be used and we see
the empirical investigation of user needs and the
unconstrained analysis of the given design space
as highly important. Similar to [28], long-term
research on the design practices of gamification
practitioners and academics would be required to
understand this dynamic.</p>
      <p>
        Equally problematic is the potential
misconception that having as many gamification
elements as possible will necessarily result in
higher engagement and a better user experience.
To be fair, the gamification framework does not
mention it [
        <xref ref-type="bibr" rid="ref25 ref41 ref6">4</xref>
        ], but it was misunderstood by
several students in the classroom study and was
recognizable in the approaches of some academic
publications. We are aware that this is in contrast
to a series of recent experimental studies by
Groening and Binnewies [17], which
demonstrated, in a small controlled setting, that
motivation and performance tend to increase as
more game elements are added. In any case, we
see a need for empirical-based research in this
area, with a particular focus on investigating
practice-oriented implications for the design of
gameful applications and systems.
      </p>
      <p>
        The Octalysis framework sensitizes for
elements of black hat gamification. Therefore, the
Octalysis framework could potentially be used as
a model to educate and raise awareness about
black hat gamification elements and closely
related concepts such as dark patterns [
        <xref ref-type="bibr" rid="ref20 ref3">16</xref>
        ].
      </p>
    </sec>
    <sec id="sec-11">
      <title>7. Limitations and future work</title>
      <p>It is important to point out some limitations of
our work. These are mainly related to the
conducted classroom study, in which we do not
deal with objectively quantifiable outcomes, but
rather with the articulation of subjective
experiences of the participating students and our
own perceptions. The style of didactic
presentation, especially in the case of complex
frameworks such as the Octalysis, definitely has
some influence on the appraisal of this
framework. With respect to this, we tried to
introduce the majority of the framework to the
students and did this in a way that was as
nonjudgmental as possible.</p>
      <p>Furthermore, there was no control group in our
study (e.g., a second course with students who
could have tackled the same challenge without the
Octalysis framework). Also, in terms of
fundamental understanding of gamification and
user experience, it was a rather homogeneous
group of HCI master’s students. It would be
interesting to examine the influence of the
Octalysis framework on the design practice of
more digital-distant target groups and of
gamification experts.</p>
      <p>With regard to other widely used gamification
frameworks, we also consider it useful to subject
these frameworks to methodologically similar
studies, in order to better understand and compare
the design practices that result from the use of the
respective frameworks. Only after extensive
discussion along these lines it seem reasonable to
us to derive practical implications and condense
them into design guidelines for the use of these
frameworks.</p>
    </sec>
    <sec id="sec-12">
      <title>8. Conclusion</title>
      <p>Since no academic studies on the use of the
Octalysis framework as a design and evaluation
tool exist, we have addressed this research gap in
this paper. Based on a comprehensive literature
study and the application of the Octalysis
framework in the context of a master’s program
course, we were able to show the challenges and
the potential of the use of the Octalysis
framework. We believe that the framework
supports the design process, particularly for idea
generation. However, there is an important need
for empirical-based validation of the Octalysis
framework. We also see the need for ongoing
methodological reflections while using the
framework to inform and guide the design
practice of researchers and practitioners; this also
holds true for other gamification frameworks and
methods. We hope that our study has contributed
to making the applicability of the Octalysis
framework academically plausible and, thereby,
providing benefits in the selection of gamification
frameworks.</p>
    </sec>
    <sec id="sec-13">
      <title>9. Acknowledgments</title>
      <p>
        This work was carried out within the scope of
the project Rendezfood (EFRE-0801425), which
is funded by the European Regional Development
Fund. We sincerely thank Lea Michel for her
contribution to this work and the anonymous
reviewers for their constructive feedback, which
has improved our paper. Furthermore, we would
like to thank all participants who took part in the
study.
10. References
[
        <xref ref-type="bibr" rid="ref44 ref98">1</xref>
        ] Richard D’arc da Silva Brito, Luis Hernan
Contreras Pinochet, Evandro Luiz Lopes,
and Mauri Aparecido de Oliveira. 2018.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <article-title>application users</article-title>
          .
          <source>Internext</source>
          <volume>13</volume>
          ,
          <issue>1</issue>
          :
          <fpage>01</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          https://doi.org/10.18568/1980-
          <fpage>4865</fpage>
          .
          <fpage>1311</fpage>
          -
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          16. [2]
          <string-name>
            <given-names>Jan</given-names>
            <surname>Broer</surname>
          </string-name>
          .
          <year>2017</year>
          . The Gamification Inventory
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          of Bremen. [3]
          <string-name>
            <given-names>Philipp</given-names>
            <surname>Busch</surname>
          </string-name>
          .
          <year>2018</year>
          . „Spielerische Ansätze
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>http://books.google.com/books?id=SskI0H</mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <article-title>Mwck4C&amp;pgis=1</article-title>
          . [4]
          <string-name>
            <surname>Yu-kai Chou</surname>
          </string-name>
          .
          <year>2015</year>
          . Actionable
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <given-names>Gamification. Octalysis</given-names>
            <surname>Media</surname>
          </string-name>
          , Fremont. [5]
          <string-name>
            <given-names>Mário</given-names>
            <surname>Cruz</surname>
          </string-name>
          .
          <year>2018</year>
          . “Chicos, Sacad El Móvil
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <source>Educational Technology</source>
          <volume>1</volume>
          :
          <fpage>282</fpage>
          -
          <lpage>298</lpage>
          . [6]
          <string-name>
            <given-names>Sebastian</given-names>
            <surname>Deterding</surname>
          </string-name>
          .
          <year>2015</year>
          . The Lens of
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Design.</surname>
          </string-name>
          Human-Computer Interaction 30,
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          https://doi.org/10.1080/07370024.
          <year>2014</year>
          .993
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          471. [7]
          <string-name>
            <given-names>Sebastian</given-names>
            <surname>Deterding</surname>
          </string-name>
          , Dan Dixon, Rilla
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Khaled</surname>
            ,
            <given-names>and Lennart</given-names>
          </string-name>
          <string-name>
            <surname>Nacke</surname>
          </string-name>
          .
          <year>2011</year>
          . From
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>the 15th International Academic MindTrek</mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Environments - MindTrek</surname>
          </string-name>
          '
          <volume>11</volume>
          ,
          <fpage>9</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          https://doi.org/10.1145/2181037.2181040. [8]
          <string-name>
            <given-names>Ioannis</given-names>
            <surname>Doumanis</surname>
          </string-name>
          and
          <string-name>
            <given-names>Serengul</given-names>
            <surname>Smith</surname>
          </string-name>
          .
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>International Journal of Serious Games</source>
          <volume>2</volume>
          ,
          <fpage>3</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          https://doi.org/10.17083/ijsg.v2i3.
          <fpage>79</fpage>
          . [9]
          <string-name>
            <given-names>Martina</given-names>
            <surname>Eckert</surname>
          </string-name>
          , Ignacio Gomez-Martinho,
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Meneses</surname>
            ,
            <given-names>and Luis</given-names>
          </string-name>
          <string-name>
            <surname>Salgado</surname>
          </string-name>
          .
          <year>2018</year>
          . The
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <source>Transactions on Game-Based Learning 5,</source>
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <volume>16</volume>
          :
          <fpage>155085</fpage>
          . https://doi.org/10.4108/eai.13-
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          7-
          <fpage>2018</fpage>
          .
          <fpage>155085</fpage>
          . [10]
          <string-name>
            <surname>Martina</surname>
            <given-names>Eckert</given-names>
          </string-name>
          , Mónica Jiménez, María-
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Salgado</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>Blexer-med: A Medical Web</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          Springer International Publishing,
          <volume>289</volume>
          -
          <fpage>299</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -76111-
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <volume>4</volume>
          _
          <fpage>29</fpage>
          . [11]
          <string-name>
            <surname>Karla</surname>
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Egan</surname>
            ,
            <given-names>M. Christina</given-names>
          </string-name>
          <string-name>
            <surname>Schneider</surname>
          </string-name>
          , and
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <given-names>Steve</given-names>
            <surname>Ferrara</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>The 6D Framework: A</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>Science+Business Media</surname>
          </string-name>
          , New York,
          <fpage>275</fpage>
          -
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          292. https://doi.org/10.1007/978-1-
          <fpage>4419</fpage>
          -
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          9356-
          <fpage>4</fpage>
          _
          <fpage>16</fpage>
          . [12]
          <string-name>
            <surname>Thomas</surname>
            <given-names>Ellenberger</given-names>
          </string-name>
          , Deane Harder, and
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <string-name>
            <given-names>Brechbühler</given-names>
            <surname>Pešková</surname>
          </string-name>
          .
          <year>2020</year>
          . Gamification in
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>658</fpage>
          -26960-9. [13]
          <string-name>
            <surname>Mohammad</surname>
            <given-names>Fathian</given-names>
          </string-name>
          , Hossein Sharifi, and
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          <string-name>
            <given-names>Faranaksadat</given-names>
            <surname>Solat</surname>
          </string-name>
          .
          <year>2019</year>
          . Investigating the
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          <source>of Information Technology Management</source>
          <volume>11</volume>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          https://doi.org/10.22059/jitm.
          <year>2019</year>
          .
          <volume>287056</volume>
          .2
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          390. [14]
          <string-name>
            <surname>Helge</surname>
            <given-names>Fischer</given-names>
          </string-name>
          , Matthias Heinz, Lars
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          <string-name>
            <surname>Schlenker</surname>
            ,
            <given-names>and Fabiane</given-names>
          </string-name>
          <string-name>
            <surname>Follert</surname>
          </string-name>
          .
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          <source>(GeNeMe)</source>
          <year>2016</year>
          ,
          <fpage>93</fpage>
          -
          <lpage>104</lpage>
          . [15]
          <string-name>
            <surname>Sergio</surname>
            <given-names>A. A.</given-names>
          </string-name>
          <string-name>
            <surname>Freitas</surname>
          </string-name>
          , Edna D. Canedo,
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          <string-name>
            <surname>and Marcia</surname>
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Silva</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Evaluation of an</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          Springer International Publishing,
          <volume>579</volume>
          -
          <fpage>592</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -58700-
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          <volume>4</volume>
          _
          <fpage>18</fpage>
          . [16]
          <string-name>
            <surname>Colin</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Gray</surname>
          </string-name>
          , Yubo Kou, Bryan Battles,
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          <year>2018</year>
          .
          <article-title>The Dark (Patterns) Side of UX</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          <string-name>
            <surname>Design</surname>
          </string-name>
          .
          <source>In Proceedings of the 2018 CHI</source>
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          <string-name>
            <surname>Systems</surname>
          </string-name>
          ,
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          https://doi.org/10.1145/3173574.3174108. [17]
          <string-name>
            <given-names>Christopher</given-names>
            <surname>Groening</surname>
          </string-name>
          and Carmen
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          <string-name>
            <surname>Binnewies</surname>
          </string-name>
          .
          <year>2021</year>
          . The More, the Merrier?
          <article-title>-</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          <source>Interaction</source>
          <volume>37</volume>
          ,
          <issue>12</issue>
          :
          <fpage>1130</fpage>
          -
          <lpage>1150</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          https://doi.org/10.1080/10447318.
          <year>2020</year>
          .187
        </mixed-citation>
      </ref>
      <ref id="ref49">
        <mixed-citation>
          0828. [18]
          <string-name>
            <given-names>Marc</given-names>
            <surname>Hassenzahl</surname>
          </string-name>
          , Sarah Diefenbach, and
        </mixed-citation>
      </ref>
      <ref id="ref50">
        <mixed-citation>
          <string-name>
            <given-names>Anja</given-names>
            <surname>Göritz</surname>
          </string-name>
          .
          <year>2010</year>
          . Needs, affect, and
        </mixed-citation>
      </ref>
      <ref id="ref51">
        <mixed-citation>
          experience.
          <source>Interacting with Computers</source>
          <volume>22</volume>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref52">
        <mixed-citation>
          https://doi.org/10.1016/j.intcom.
          <year>2010</year>
          .
          <volume>04</volume>
          .00
        </mixed-citation>
      </ref>
      <ref id="ref53">
        <mixed-citation>
          2. [19]
          <string-name>
            <surname>Dennis L. Kappen</surname>
            and
            <given-names>Lennart E.</given-names>
          </string-name>
          <string-name>
            <surname>Nacke</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref54">
        <mixed-citation>
          <year>2013</year>
          . The Kaleidoscope of Effective
        </mixed-citation>
      </ref>
      <ref id="ref55">
        <mixed-citation>
          <string-name>
            <surname>Applications</surname>
          </string-name>
          ,
          <fpage>119</fpage>
          -
          <lpage>122</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref56">
        <mixed-citation>
          https://doi.org/10.1145/2583008.2583029. [20]
          <string-name>
            <given-names>Jovana</given-names>
            <surname>Karać</surname>
          </string-name>
          and
          <string-name>
            <given-names>Martin</given-names>
            <surname>Stabauer</surname>
          </string-name>
          .
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref57">
        <mixed-citation>
          41-
          <fpage>54</fpage>
          . https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -
        </mixed-citation>
      </ref>
      <ref id="ref58">
        <mixed-citation>
          58484-
          <fpage>3</fpage>
          _
          <fpage>4</fpage>
          . [21]
          <string-name>
            <surname>Oliver</surname>
            <given-names>Korn</given-names>
          </string-name>
          , Florian Brenner, Julian Börsig,
        </mixed-citation>
      </ref>
      <ref id="ref59">
        <mixed-citation>
          <string-name>
            <given-names>Müller. 2018. Defining</given-names>
            <surname>Recrutainment</surname>
          </string-name>
          : A
        </mixed-citation>
      </ref>
      <ref id="ref60">
        <mixed-citation>
          <source>(eds.)</source>
          .
          <fpage>37</fpage>
          -
          <lpage>49</lpage>
          . https://doi.org/10.1007/978-3-
        </mixed-citation>
      </ref>
      <ref id="ref61">
        <mixed-citation>
          319-
          <fpage>60486</fpage>
          -
          <issue>2</issue>
          _
          <fpage>4</fpage>
          . [22]
          <string-name>
            <surname>Andrés</surname>
            <given-names>Lucero</given-names>
          </string-name>
          , Jussi Holopainen, Elina
        </mixed-citation>
      </ref>
      <ref id="ref62">
        <mixed-citation>
          <string-name>
            <surname>Karapanos</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>The playful experiences</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref63">
        <mixed-citation>
          evaluation.
          <source>In Proceedings of the 6th</source>
        </mixed-citation>
      </ref>
      <ref id="ref64">
        <mixed-citation>
          https://doi.org/10.1145/2513506.2513530. [23]
          <string-name>
            <surname>Wendy</surname>
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Mackay</surname>
          </string-name>
          and Anne Laure Fayard.
        </mixed-citation>
      </ref>
      <ref id="ref65">
        <mixed-citation>
          <string-name>
            <given-names>1999. Video</given-names>
            <surname>Brainstorming</surname>
          </string-name>
          and Prototyping:
        </mixed-citation>
      </ref>
      <ref id="ref66">
        <mixed-citation>
          <article-title>'99 extended abstracts on Human factors in</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref67">
        <mixed-citation>
          <source>computer systems - CHI '99</source>
          ,
          <fpage>118</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref68">
        <mixed-citation>
          https://doi.org/10.1145/632780.632790. [24]
          <string-name>
            <given-names>Andrzej</given-names>
            <surname>Marczewski</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>User Types</article-title>
          . In
        </mixed-citation>
      </ref>
      <ref id="ref69">
        <mixed-citation>
          <string-name>
            <given-names>Independent</given-names>
            <surname>Publishing Platform</surname>
          </string-name>
          ,
          <fpage>65</fpage>
          -
          <lpage>80</lpage>
          . [25]
          <string-name>
            <given-names>Eva</given-names>
            <surname>Mårell-Olsson</surname>
          </string-name>
          .
          <year>2019</year>
          . University
        </mixed-citation>
      </ref>
      <ref id="ref70">
        <mixed-citation>
          <string-name>
            <given-names>Emergent</given-names>
            <surname>Technologies in Swedish</surname>
          </string-name>
          K-
          <volume>12</volume>
        </mixed-citation>
      </ref>
      <ref id="ref71">
        <mixed-citation>
          <string-name>
            <surname>Architecture</surname>
          </string-name>
          (s)
          <volume>42</volume>
          :
          <fpage>47</fpage>
          -
          <lpage>69</lpage>
          . [26]
          <string-name>
            <surname>Fitri</surname>
            <given-names>Marisa</given-names>
          </string-name>
          , Sharifah Sakinah Syed Ahmad,
        </mixed-citation>
      </ref>
      <ref id="ref72">
        <mixed-citation>
          <string-name>
            <given-names>Aris</given-names>
            <surname>Widodo</surname>
          </string-name>
          .
          <year>2020</year>
          . Evaluation of Student
        </mixed-citation>
      </ref>
      <ref id="ref73">
        <mixed-citation>
          <article-title>19 with Octalysis Gamification Framework</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref74">
        <mixed-citation>
          <source>Science and Applications</source>
          <volume>11</volume>
          , 11:
          <fpage>104</fpage>
          -
          <lpage>116</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref75">
        <mixed-citation>
          https://doi.org/10.14569/IJACSA.
          <year>2020</year>
          .0111
        </mixed-citation>
      </ref>
      <ref id="ref76">
        <mixed-citation>
          114. [27]
          <string-name>
            <given-names>Jane</given-names>
            <surname>McGonigal</surname>
          </string-name>
          .
          <year>2015</year>
          . SuperBetter: A
        </mixed-citation>
      </ref>
      <ref id="ref77">
        <mixed-citation>
          <string-name>
            <surname>Books</surname>
          </string-name>
          , New York. [28]
          <string-name>
            <surname>Benedikt</surname>
            <given-names>Morschheuser</given-names>
          </string-name>
          , Lobna Hassan, Karl
        </mixed-citation>
      </ref>
      <ref id="ref78">
        <mixed-citation>
          <string-name>
            <surname>Werder</surname>
            ,
            <given-names>and Juho</given-names>
          </string-name>
          <string-name>
            <surname>Hamari</surname>
          </string-name>
          .
          <year>2018</year>
          . How to
        </mixed-citation>
      </ref>
      <ref id="ref79">
        <mixed-citation>
          <source>and Software Technology</source>
          <volume>95</volume>
          :
          <fpage>219</fpage>
          -
          <lpage>237</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref80">
        <mixed-citation>
          https://doi.org/10.1016/j.infsof.
          <year>2017</year>
          .
          <volume>10</volume>
          .015. [29]
          <string-name>
            <given-names>Scott</given-names>
            <surname>Nicholson</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>A RECIPE for</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref81">
        <mixed-citation>
          <string-name>
            <given-names>International</given-names>
            <surname>Publishing</surname>
          </string-name>
          , Cham,
          <fpage>1</fpage>
          -
          <lpage>20</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref82">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -10208-
        </mixed-citation>
      </ref>
      <ref id="ref83">
        <mixed-citation>
          <volume>5</volume>
          _
          <fpage>1</fpage>
          . [30]
          <string-name>
            <surname>Tania</surname>
            <given-names>Ouariachi</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chih-Yen Li</surname>
            , and
            <given-names>Wim J. L.</given-names>
          </string-name>
        </mixed-citation>
      </ref>
      <ref id="ref84">
        <mixed-citation>
          <string-name>
            <surname>Elving</surname>
          </string-name>
          .
          <year>2020</year>
          . Gamification Approaches for
        </mixed-citation>
      </ref>
      <ref id="ref85">
        <mixed-citation>
          <string-name>
            <given-names>Best</given-names>
            <surname>Practices</surname>
          </string-name>
          .
          <source>Sustainability</source>
          <volume>12</volume>
          ,
          <issue>11</issue>
          :
          <fpage>4565</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref86">
        <mixed-citation>
          https://doi.org/10.3390/su12114565. [31]
          <string-name>
            <given-names>Jay</given-names>
            <surname>Peters</surname>
          </string-name>
          .
          <year>2021</year>
          .
          <article-title>Google is making it easier</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref87">
        <mixed-citation>
          <string-name>
            <surname>Verge</surname>
          </string-name>
          .
          <source>Retrieved December 1</source>
          ,
          <year>2021</year>
          from
        </mixed-citation>
      </ref>
      <ref id="ref88">
        <mixed-citation>
          https://www.theverge.com/
          <year>2021</year>
          /10/14/2272
        </mixed-citation>
      </ref>
      <ref id="ref89">
        <mixed-citation>
          <article-title>6625/google-search-results-continuous-</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref90">
        <mixed-citation>
          <article-title>scrolling-mobile-doomscroll</article-title>
          . [32]
          <string-name>
            <surname>Pedro</surname>
            <given-names>Ponce</given-names>
          </string-name>
          , Alan Meier, Juana Isabel
        </mixed-citation>
      </ref>
      <ref id="ref91">
        <mixed-citation>
          <string-name>
            <given-names>Omar</given-names>
            <surname>Mata</surname>
          </string-name>
          .
          <year>2020</year>
          .
          <article-title>Tailored gamification and</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref92">
        <mixed-citation>
          https://doi.org/10.1016/j.jclepro.
          <year>2020</year>
          .12116
        </mixed-citation>
      </ref>
      <ref id="ref93">
        <mixed-citation>
          7. [33]
          <string-name>
            <given-names>Whitney</given-names>
            <surname>Quesenbery</surname>
          </string-name>
          and
          <string-name>
            <given-names>Kevin</given-names>
            <surname>Brooks</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref94">
        <mixed-citation>
          2010.
          <article-title>Storytelling for User experience:</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref95">
        <mixed-citation>
          <string-name>
            <surname>Media</surname>
          </string-name>
          , New York. [34]
          <string-name>
            <surname>Richard</surname>
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Thaler</surname>
          </string-name>
          and
          <string-name>
            <surname>Cass R. Sunstein</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref96">
        <mixed-citation>
          2008.
          <article-title>Nudge: Improving decisions about</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref97">
        <mixed-citation>
          University Press. [35]
          <string-name>
            <given-names>Fabiane</given-names>
            <surname>Rohr</surname>
          </string-name>
          and
          <string-name>
            <given-names>Helge</given-names>
            <surname>Fischer</surname>
          </string-name>
          .
          <year>2014</year>
          . Mehr
        </mixed-citation>
      </ref>
      <ref id="ref98">
        <mixed-citation>
          1-
          <fpage>9</fpage>
          . [36]
          <string-name>
            <surname>Richard</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Ryan</surname>
          </string-name>
          and
          <string-name>
            <surname>Edward L. Deci</surname>
          </string-name>
          .
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref99">
        <mixed-citation>
          <source>Psychologist</source>
          <volume>55</volume>
          ,
          <issue>1</issue>
          :
          <fpage>68</fpage>
          -
          <lpage>78</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref100">
        <mixed-citation>
          https://doi.org/10.1037/
          <fpage>0003</fpage>
          -
          <lpage>066X</lpage>
          .
          <year>55</year>
          .1.
          <fpage>68</fpage>
          . [37]
          <string-name>
            <surname>Mary-Luz</surname>
          </string-name>
          Sanchez-Gordón, Ricardo
        </mixed-citation>
      </ref>
      <ref id="ref101">
        <mixed-citation>
          2016.
          <article-title>Gamification and Human Factors in</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref102">
        <mixed-citation>
          <string-name>
            <surname>from Octalysis</surname>
          </string-name>
          <article-title>Framework to ISO 10018</article-title>
          . In
        </mixed-citation>
      </ref>
      <ref id="ref103">
        <mixed-citation>
          <string-name>
            <surname>Publishing</surname>
          </string-name>
          ,
          <fpage>234</fpage>
          -
          <lpage>241</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref104">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -44817-
        </mixed-citation>
      </ref>
      <ref id="ref105">
        <mixed-citation>
          <volume>6</volume>
          _
          <fpage>19</fpage>
          . [38]
          <string-name>
            <surname>Gustavo F Tondello</surname>
          </string-name>
          , Dennis L Kappen,
        </mixed-citation>
      </ref>
      <ref id="ref106">
        <mixed-citation>
          <string-name>
            <given-names>Marim</given-names>
            <surname>Ganaba</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Lennart E</given-names>
            <surname>Nacke</surname>
          </string-name>
          .
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref107">
        <mixed-citation>
          <year>2019</year>
          ., Masaaki Kurosu (ed.). Springer
        </mixed-citation>
      </ref>
      <ref id="ref108">
        <mixed-citation>
          <string-name>
            <given-names>International</given-names>
            <surname>Publishing</surname>
          </string-name>
          , Cham,
          <fpage>224</fpage>
          -
          <lpage>244</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref109">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>030</fpage>
          -22646-
        </mixed-citation>
      </ref>
      <ref id="ref110">
        <mixed-citation>
          <volume>6</volume>
          _
          <fpage>16</fpage>
          . [39]
          <string-name>
            <surname>Gustavo</surname>
            <given-names>Fortes</given-names>
          </string-name>
          <string-name>
            <surname>Tondello</surname>
          </string-name>
          .
          <year>2019</year>
          . Dynamic
        </mixed-citation>
      </ref>
      <ref id="ref111">
        <mixed-citation>
          from http://hdl.handle.net/10012/14807. [40]
          <string-name>
            <given-names>Áron</given-names>
            <surname>Tóth</surname>
          </string-name>
          and
          <string-name>
            <given-names>Bálint</given-names>
            <surname>Szabó</surname>
          </string-name>
          .
          <year>2018</year>
          . A Pilot
        </mixed-citation>
      </ref>
      <ref id="ref112">
        <mixed-citation>
          <string-name>
            <given-names>and Feedback</given-names>
            <surname>Mechanics</surname>
          </string-name>
          .
          <source>In</source>
          <year>2018</year>
          9th IEEE
        </mixed-citation>
      </ref>
      <ref id="ref113">
        <mixed-citation>
          https://doi.org/10.1109/CogInfoCom.
          <year>2018</year>
          .8
        </mixed-citation>
      </ref>
      <ref id="ref114">
        <mixed-citation>
          639870. [41]
          <string-name>
            <given-names>V.</given-names>
            <surname>Yfantis</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Tseles</surname>
          </string-name>
          .
          <year>2013</year>
          . Exploring
        </mixed-citation>
      </ref>
      <ref id="ref115">
        <mixed-citation>
          <string-name>
            <given-names>The</given-names>
            <surname>Octalysis Conceptual Model</surname>
          </string-name>
          . [42]
          <string-name>
            <given-names>Yuxiang</given-names>
            <surname>Zhao</surname>
          </string-name>
          and
          <string-name>
            <given-names>Jian</given-names>
            <surname>Tang</surname>
          </string-name>
          .
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref116">
        <mixed-citation>
          <string-name>
            <surname>Publishing</surname>
          </string-name>
          , Cham,
          <fpage>467</fpage>
          -
          <lpage>479</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref117">
        <mixed-citation>
          https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -39513-
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>