=Paper=
{{Paper
|id=Vol-2883/paper9
|storemode=property
|title=The role of gameful perception as a mediator for intrinsically motivating gamification.
|pdfUrl=https://ceur-ws.org/Vol-2883/paper9.pdf
|volume=Vol-2883
|authors=Henrik Wesseloh,Niklas Buddensiek,Tim Pantel,Felix M. Stein,Phillip Szelat,Matthias Schumann
|dblpUrl=https://dblp.org/rec/conf/gamifin/WesselohBPSSS21
}}
==The role of gameful perception as a mediator for intrinsically motivating gamification.==
https://ceur-ws.org/Vol-2883/paper9.pdf
Henrik Wesseloha, Niklas Buddensieka, Tim Pantela, Felix M. Steina, Phillip Szelata and
Matthias Schumanna
a
University of Göttingen, Platz der Göttinger Sieben 5, Göttingen, 37073, Germany
Scientific studies have shown that the use of gamification is effective in promoting motivation
and behavior. However, reasons for the effectiveness of the implemented game elements are
analyzed insufficiently. The following article therefor addresses the issue of gameful perception
induced through gamification by using an experimental, single factor between-subjects design.
First, we draw on Self-Determination Theory to derive a grounded research model. Then, we
investigate two groups by using a learning application with a gamified and non-gamified
version to test our hypotheses. Our results show that the users gameful perception of an activity
mediates the motivational outcomes. We discuss our findings and relate these to
design to outline pitfalls and the importance of evoking a conscious gameful experience.
Gamification, Intrinsic Motivation, SDT, Gameful Perception, SEM
unknown to what extent certain factors besides
gameful affordances influence the motivational
As an important instrument for fostering or impact. These aspects are however needed to
maintaining human motivation and behavior, explain the different outcomes among similar
gamification, i.e. the use of game design game design elements [13, 18]. For this study
elements in a non-game context [6] to evoke we therefor analyze whether the motivational
gameful experiences [11], has gained more effect of gamification depends on a gameful
attention in recent years. Gamification is used perception. We assume that in order to benefit
across different domains, e.g. education, health, from gamification, it is mandatory that users
or crowdsourcing, to support psychological and also perceive the activity as gameful. Thus, we
behavioral outcomes, like e.g. engagement or address the following research question:
participation [13]. Previous research has shown
that gamification can also help to satisfy RQ: How does a gameful perception influence
intrinsic psychological needs like autonomy, the motivational effect of gamification?
competence or relatedness and thus support
intrinsic motivation [4, 15, 17, 23]. To address this question, we briefly want to
However, the knowledge about how exactly give an overview of related research and derive
gamification works is still incomplete [19]. a research model based on Self-Determination
Systematic reviews on gamification research Theory (SDT) [4] as theoretical foundation. We
mostly report mixed results for the then present our single factor between-subjects
effectiveness [13]. Even though the number of research design by describing the experiment
studies analyzing gamifications effectiveness is procedure, the measures for data collection and
growing, academia still misses insights: It is yet provide a descriptive analysis of participants.
5th International GamiFIN Conference 2021 (GamiFIN 2021),
April 7-10, 2021, Finland.
EMAIL: henrik.wesseloh@uni-goettingen.de
2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
To validate the mediation model, we follow a perceived as informative, it would promote the
structural equation modeling approach [2, 12]. experience of competence [4, 5].
Moreover, we do a mean comparison, to find Mekler et al. (2017) analyzed motivational
differences in motivational effect of a gamified effects of individual game elements and found
and a non-gamified application. Finally, we that, despite their supposed external incentive,
discuss our findings and conclude this paper to points, level and the leaderboard did not lead to
provide implications for future research. a negative (overjustification) effect on intrinsic
Our findings contribute to current research motivation [15]. They attributed this result to a
analyzing the effect of gamification on intrinsic rather low gameful experience during the task.
motivation and need satisfaction, because they Sailer et al. (2017) did a comparable study
provide an understanding of why gamification and found positive effects of gamification on
studies might fail measuring positive outcomes competence and relatedness. Furthermore, they
[13]: When analyzing the effect of (individual) noticed that some participants did not recognize
game elements on psychological outcomes, we the game elements. The authors attributed this
often miss on the actual emersion of a conscious result to potential flaws in their gamification
gameful perception within the participants. design and assumed that it had resulted in a
lower motivational effect [17].
Furthermore, the contributions from Suh et
al. (2018) and Xi & Hamari (2019) both provide
positive empirical insights on how gamification
satisfies the three intrinsic needs [20, 23]. These
Recent empirical studies on the effects of authors point out that the need for further
gamification on intrinsic motivation are mostly research on potential moderators or mediators
based on SDT and its subtheories and therefore remains.
also examine the basic needs for autonomy,
competence or relatedness [cf. 15, 17, 20, 23].
Autonomy refers to a psychological freedom
to perform an activity at own will, while This section depicts the research model, which
competence describes the effectiveness of an we briefly outline in the following.
individual doing this activity. Relatedness For our model, we consider gamification as
covers the social aspect of an activity. In a manifest, exogenous variable that represents
combination, these factors build the concept of the source of a gameful perception. To simplify,
intrinsic motivation, which refers to an inherent gamification is modelled as binary variable, so
interest or enjoyment towards an activity itself an application or service will only vary between
and not for its instrumental value [3, 4]. the gamified (1) and the non-gamified (0) state.
While an older study from Hanus & Fox [8] Perception of gamefulness describes to what
showed that gamification had a negative effect extent individuals consciously notice a game-
on intrinsic motivation recent studies presented like situation when interacting with a system or
positive effects. We outline a few contributions service. This perception is supported e.g. by
which stressed the importance of feelings of challenge, competition, immersion
perception for intrinsic need satisfaction in a and social experiences, as well as playfulness
chronological order. [9]. As gameful designers, we can address these
Lieberoth (2015) hypothesized that framing different experiences when implementing game
an activity as a game can lead to behavioral mechanics in a utilitarian system or service
changes and subjective experiences of intrinsic [10]. Therefore, we state our first hypothesis:
motivation [14]. The study showed that the
perception of a playful frame for an mundane H1: Gamification has a positive impact on the
task has a significant impact on enjoyment. perception of gamefulness.
Deterding (2015), suggested gamification to
support experiences of competence or inhibit Furthermore, Deci & ly
autonomy, depending on perception. Cognitive Evaluation Theory, CET) postulates
He draws on SDT to argue that, if gamification that psychological need satisfaction consists of
is perceived by users as controlling, this would the fulfillment of autonomy, competence and
have a negative influence on the experience of relatedness [4].
autonomy. If the gameful feedback in turn is influences the satisfaction of intrinsic needs [4],
gamification can be used to create beneficial
conditions that support feelings of autonomy,
competence and relatedness [5, 15, 17, 20, 23]. We used a single factor between-subjects study
For example, points, badges, or rankings are design to answer the research question and test
common feedback elements and should thus our model hypotheses. Hence, the participants
support the need of competence [4]. These were divided into two groups: experimental and
elements, however, do not have to inevitably control. Both groups performed the same
originate from games and thus might not be experiment, in which they answered general
perceived per se as gameful affordance. We knowledge questions with our prototypical
take this instant as a reason why gameful application. We treated the experimental group
approaches led to different results despite using with a gamified version, while the control
comparable game elements within similar cases group used a non-gamified version. After the
[13, 18]. So, to benefit from the motivational experiment, both groups were asked to fill out
pull [16] of games, we assume that users need an online survey to determine their experiences.
to perceive the use of a gamified application as These survey results were then analyzed using
gameful. Consequently, we state our second statistical methods.
research hypothesis:
H2a-c: Perception of gamefulness mediates
needs of autonomy (a), The laboratory experiment is based on the use
competence (b) and relatedness (c). of a mobile web application for knowledge
retrieval that uses collaborative and competitive
Moreover, we want to address the support of game design elements to motivate learning. We
intrinsic motivation as a desirable motivational developed this learning app for lectures [22], so
effect. In theory, the three psychological needs mainly groups of 10 or more students use it.
of autonomy, competence and relatedness are During a session, participants answer randomly
required determinants for intrinsic motivation assigned, multiple-choice questions prepared
[4]. To by lecturer on their mobile device, in their own
SDT suggests to measure enjoyment, as they pace, with a maximum duration of 30 seconds
are closely associated [3]. Recent studies on per question. These question sessions are set to
gamification have used this operationalization last between three to five minutes. After a
to prove its motivational effects and concluded session, the participants receive a summary of
that supporting need satisfaction may improve their performance and solutions for their
enjoyment and, thus, intrinsic motivation [20]. individual questions, while lecturers get access
For that reason, we hypothesize: to aggregated performance data.
The gamified version additionally includes
H3a-c: Satisfying the need of autonomy (a), different game design elements to provide a
competence (b), and relatedness (c), has a gameful experience for its users and thus
positive effect on intrinsic motivation. support the feeling of autonomy, competence,
and relatedness. We integrated feedback
Figure 1 visualizes our derived research model elements like, points (for correct answers incl.
with the different hypotheses to test. streaks), badges, and a leaderboard to support
experiences of achievement and competition. the participants via projector [#3,4], the start
To support immersion, users find a narrative screen for participants [#5,6], the gameful
setting and can choose their respective gameful feedback for participants [#7] and the gameful
avatar. Furthermore, the session resembles a post-game screen with the leaderboard [#8].
challenging quest, in which the students team Our plan was to use the application in two
up to encounter a virtual boss enemy, which distinct lectures with live attendees, to set up
resembles the question pool. During this boss the two groups for the experiment. Due to the
battle, each student contributes individually to ongoing corona pandemic, however, no live
overcome the boss by answering questions lectures were held, so we decided to adapt the
correctly but can be eliminated after a defined experiment to take place online virtually. To
number of mistakes (virtual lives). The students simulate other participants during the question
win the boss event collaboratively by solving a session, we integrated bots in our application.
defined number of questions during the set This way, we were able to allow individuals or
time. Overall, we assume, these achievement- smaller groups to participate in the online
related, immersive and social features are able experiment and still maintain the social aspect
to of our gamification concept. Additionally, we
in the educational context [23]. setup a question pool of 45 general knowledge
In contrast, the non-gamified version only questions in the field of sports, politics, history,
shows a timer to indicate the remaining time and science to reach out for a broader audience.
during a question session. All gamification We shared our invitation to participate in the
features like the narrative or the feedback experiment via social media. The individuals
elements are omitted. Figure 2 shows the visual and small groups that joined our experiment via
differences between the gamified and non- a video conference platform that allows screen
gamified application. The screenshots display sharing (of the lecturer screen) were randomly
the quiz question design on a mobile device assigned to the experimental or control group.
(e.g., smartphone) [#1,2], the lecturer screen Before each question session, we informed the
during a question session, which is presented to participants about the procedure of the
experiment and briefly presented the respective all items are self-reported experience measures,
application. Then, the question session started we carefully translated the items to German for
with a duration of 4 minutes (experimental participants to comprehend.
group) or 3 minutes (control group)2. To measure the perception of gamefulness,
After participants had seen their individual we selected items from the GAMEFULQUEST
(gameful) results, the app provided a button scale for playfulness [9]. We did not consider
linked to the online survey. We used this link to other game-specific items to prevent confusion
add a parameter of each ID to be in the control group. To operationalize the SDT
able to connect the self-reported survey data perspective of intrinsic motivation, we chose
with actual performance and session data items from the Intrinsic Motivation Inventory
provided be the app. (IMI) and respective contributions [1, 3, 21].
For the construct of autonomy the Ubisoft
Perceived Experience Questionnaire (UPEQ)
was used, which adapts the autonomy measures
The data collection following the experiment to video games [1]. The construct of relatedness
was carried out using an online survey. All was measured using an adapted questionnaire
items were measured with a seven-point Likert of the IMI for group work
scale (completely disagree (1) to completely [21]. The third SDT construct, competence, was
agree (7)). We used five items per latent measured using the perceived competence
construct to ensure an appropriate length of the scale from the IMI [3]. Additionally, we used
questionnaire but also to leave additional room the IMI scale for enjoyment / interest to
for the removal of weak or unsuitable items. As operationalize intrinsic motivation [3].
2
We chose this distinction, as the gamified session can end early
due to its winning condition. In average, this resulted in the same
playing time.
In addition to these measures, we asked six they played several times a week and 26%
control questions to differentiate between the several times a month . The largest proportion
respondents. These controls include gender, age of respondents (33%) said they played games
and highest educational level. Furthermore, the less often or never . Some participants (34%)
general affinity towards play was queried, with also had used our application before, but for the
users specifying how often they play games majority (64%) the app was new.
during the week and whether participants had
used our application before.
Due to cross-loadings (cf. 4.1 Measurement
Model) and/or poor factor loadings (< 0.70) we To test the postulated hypotheses from our
had to remove certain items during data research model, we used a structural equation
analysis to ensure high construct reliability. modelling (SEM) approach [2, 12]. The data
firms internal consistency analysis was done in Stata 16 and SPSS 26. To
for each construct 0.828). assess the quality of the measurement model,
Table 1 shows the items used as reflective we performed an exploratory factor analysis in
measures for the latent constructs (italic items SPSS. Furthermore, we compared the survey
had to be removed during factor analysis). results between the two groups with a mean
comparison. Subsequently, the structural model
was estimated using the maximum likelihood
estimation (MLE) as the default method [12] in
Overall, the distribution of participants between Stata. Accordingly, the SEM requirements of a
the experimental and comparison group was large sample size, multivariate normality and
balanced with 105 and 93 respondents. The correct model specification [12] were met, as
experiment was taken by 43 individuals, 23 the next section shows.
groups of two, 15 groups of three and 12 groups
with four or more participants. The largest
group in our sample held twelve participants.
The age of the participants was highly The SEM approach requires, the constructs to
concentrated in the range of 20 to 30 years, the be tested for their content, convergent, and
average age was 30.6 years (SD = 13.57), the discriminant validity in order to assess the fit of
median was 25 years. Whereas 134 participants the model with the collected data [2].
were between the age of 20 and 30 years, only First, we assume content validity, because
48 participants were older than 30. Males were of our item selection from validated, existing
slightly overrepresented (60%) compared with scales like IMI [3] and GAMEFULQUEST [9].
females (39%), while two participants replied Second, we tested empirically our sample
their gender as diverse (1%). for convergent validity by determining the
The participants mainly have an academic individual item reliability, composite reliability
background: 43% had a bachelor's degree, 22% (CR) and average variance extracted (AVE) by
a master's or diploma. 21% of the respondents executing a factor analysis. The sample data
indicated that their highest educational level was suitable for factor analysis, as the Kaiser-
was the high school graduation. The most Meyer-Olkin Measure of Sampling Adequacy
common field of study was economics (51%), (KMO) is greater than 0.60 (KMO = 0.834). In
while other fields of study were mainly in the case of individual item reliability, we follow
humanities and social sciences as well as the criteria of high factor loadings (> 0.70) [12],
teacher training and . which is why a few items (POG3; A3; A4; A5;
During the question sessions, the participants C5; IM3; IM4) had to be removed from the
answered 12.8 (SD = 4.81) questions after 14.6 initial model. Afterwards we determined values
(SD = 5.39) seconds on average. The relative for CR that ranged from 0.764 to 0.932, which
success rate, i.e., the ratio of correct responses is above the acceptable limit (> 0.70) [12].
compared to the total number of answered Additionally, we calculated the AVE for each
questions, was 54%. construct, which also exceeded the required
About 22% of the participants had a high threshold (> 0.50) in all cases, except for the
affinity towards play and stated that they played perception of gamefulness (AVE = 0.462),
games such as board games or video games which is slightly below the limit. Still, this
every day , whereas 19% of participants said constructs convergent validity can be regarded
adequate [7], due to its higher composite any squared correlation with another construct,
reliability (CR = 0.764) . which serves as a prove of discriminant validity
Third, we determined discriminant validity at construct level.
by checking for possible cross-loadings via the Table 2 and Table 3 present the results from
pattern matrix. For the adjusted model, all items our factor analysis, showing the relevant scores
showed the highest loadings for their respective for CR, AVE, Alpha ( , as well as
construct. Additionally, we draw upon the cross-loadings and inter-construct correlations
criterion from Fornell & Larcker [7] to show for the Fornell-Larcker-criterion [7].
that the AVE of each construct is higher than
As last data analysis in the measurement model, competence ( = 0.034; p = 0.647). In case of
we did an independent-samples t-test with a relatedness, however, gamification showed a
95% confidence interval to compare the means significant direct effect = 0.163; p = 0.013).
of the survey results between the gamified, In addition, we determined significant indirect
experimental group (n = 105) and the non- effects of gamification on autonomy ( = 0.114;
gamified, control group (n = 93). In advance, p = 0.004), as well as on relatedness = 0.166;
the requirement p = 0.002), but not on competence ( = 0.038;
of equal variances. p = 0.107). According to this result, POG fully
On the one hand, the results of the mean mediates the relationship between gamification
comparison identified differences between the and autonomy but only partially mediates the
gamified and non-gamified group considering a effect of gamification on relatedness. Thus, our
gameful perception and intrinsic motivation hypotheses (H2a-c) are just partly supported.
(both p 0.05). Especially for the construct of As postulated by SDT [4], we are also able
relatedness, the item means differ significantly to prove the relationship of intrinsic motivation
(p 0.01). On the other hand, we were, not able and needs of = 0.163; p = 0.021),
to measure any differences for the constructs of social
autonomy or competence. .395; p = 0.000). Thus, the
Table 4 shows the results of the mean hypotheses derived from SDT (H3a-c) are also
comparison with sample means for the control supported. Moreover, neither of the measured
group ( 0) and experimental group ( 1) and control variables has a significant positive
their respective confidence intervals (CI). association (p 0.05) with POG or any other
latent construct.
In terms of model fit, our structural model
shows good to adequate values for different
We then estimated the revised model with MLE quality criteria [12]: RMSEA = 0.073 (< 0.08),
and determined the model fit indices. The CFI = 0.912 (> 0.9), TLI = 0.896 (~ 0.9), and
results for our structual model indicate effects SRMR = 0.077 (< 0.08). Figure 3 shows the
of gamification on gameful perception (POG) estimated model with standardized coefficients.
( = 0.254; p = 0.001), which was expectable.
Thus, hypothesis H1 is supported. Furthermore,
we found that gameful perception affected the
experience of some basic psychological needs. Overall, we found an evidence for a mediating
The influence of POG on relatedness showed
the strongest effect ( 54; p = 0.000), gamification and intrinsic need satisfaction. We
whereas the impact on experience of autonomy interprete that the reason for the full mediation
seemed moderate ( ). In in case of autonomy lies in the characteristic of
contrast, the relationship between a gameful play itself, which is regarded as a voluntary
perception and experiencing competence could activity (and thus autonomous) [16]. Therefore,
not be confirmed ( = 0.149; p = 0.062). When only if users perceive their activity as gameful,
analyzing for mediating effects, we did not find it likely that they actually feel self-determined
significant direct effects between gamification [4]. The reason for partial mediation in case of
and either autonomy ( = 0.102; p = 0.150) or relatedness can be explained by the design of
our gamified app. The leaderboard or avatars, was statistically not significant could lie in the
e.g., reveal that other users are also involved in group composition, as the affinity to play was
a session. Therefore, regardless of a gameful about 15% lower in the gamified group
perception, users can still feel connected [4]. (M = 2.486) compared to the non-gamified
Additionally, the relatedness is reinforced by a group (M = 2.925).
collaborative competition to beat the boss and
win the quiz, which likely fuels their gameful
perception [9]. This seems reasonable for us,
but, some unexpected findings need discussion. The objective of this study was to analyze how
First, the mean comparison did not show
differences for autonomy and competence motivational effect. Our results indicate the
between the groups. In the case of competence, importance of a gameful perception in context
the bots in the gamified version could be a of a gamified learning app to foster intrinsic
possible cause for the missing change, as they motivation. The results, however, underly a few
limitations which we briefly want to address to
average response time was 5.7 seconds lower, provide some ideas for future research.
while having a success rate of 75%. In During the experiment, both applications for
comparison to the 52% success rate of the knowledge retrieval represented a quiz, which
participants, the integrated bots seemed a bit can be considered per se as a game. In context
of education, however, we want to point out the
of the leaderboard, they may have inhibited the ulitarian (and not hedonic) purpose to
determine learning outcomes. That is why we
expected to emerge from the gameful feedback refer to the control group as non-gamified in
[22]. This flaw in our app design might have led this study. Still, in other scenarios it might be
to the low (non-significant) mediating effect. reasonable to compare low and high degrees of
One reason for the lack of change in autonomy gamification to determine gameful perception.
could also lie in the gameful design of the Furthermore, we want to stress out, that we
application: The gamified app might not have selected and translated items from different,
provided enough choice to support a strong validated scales. To
feeling of autonomy [17]. For this aspect, the perception, we only considered playfulness as
two groups were only treated differently due to an indicator. Still, there are also other possible
the gamified character selection. dimensions that induce a gameful experience
Second, the experience of relatedness which researchers can use to employ a fuller
differed strongly between the groups. In this measure of gamefulness as a mediator [9].
case, the effect of gamification might be Finally, we want to encourage researchers to
skewed, because the group sizes were slightly consider gameful perception when analyzing
larger in the experimental group with 4.22 motivational effects of gamification in future
participants than in the comparison group with (replication) studies. Studies that examine the
2.40 participants in average. In general, this circumstances where individual game elements
small difference in the social environment evoke a gameful perception and thus support
during the experiment might have moderated intrinsic needs are still scarce. We also suggest
the effect. It therefore remains uncertain to what to compare gamification on gamers
extent gamification was the underlying factor and non-gamers need satisfaction, as both target
for the change in social relatedness. groups seem to have a different perspective on
Third, the control variable of affinity to gamification.
play was slightly below the borderline to show
a significant effect on gameful perception
( = 0.154; p = 0.054). We think that the affinity
to play, however, can have an impact on [1] A. Azadvar, A. Canossa, UPEQ: Ubisoft
gameful perception, as gamers have a wider perceived experience questionnaire: A
experience of what an actual game situation self-determination evaluation tool for
feels like compared to non-gamers. Therefore, video games, ACM Internat. Conference
gamers might perceive a gameful situation Proceeding Series (2018) 1 8.
easier as they can relate them to other game [2] K. A. Bollen. Structural Equations with
experiences. A possible reason, why this effect Latent Variables, Wiley, New York, 1989.
[3] E. L. Deci, H. Eghrari, B. C. Patrick, D. R. an Activity as a Game, Games and Culture
Leone, Facilitating Internalization: The 10 (2015) 229 248.
Self-Determination Theory Perspective, [15] E. D. Mekler, F. Brühlmann, A. N. Tuch,
Journal of Personality 62 (1994) 119 K. Opwis, Towards understanding the
142. effects of individual gamification
[4] E. L. Deci, R. M. Ryan, The "What" and elements on intrinsic motivation and
"Why" of Goal Pursuits: Human Needs performance, Computers in Human
and the Self-Determination of Behavior, Behavior 71 (2017) 525 534.
Psychological Inquiry 11 (2000) 227 268. [16] R. M. Ryan, C. S. Rigby, A. Przybylski,
[5] S. Deterding, The Lens of Intrinsic Skill The motivational pull of video games: A
Atoms: A Method for Gameful Design, self-determination theory approach,
Human-Computer Interaction 30 (2015) Motivation and Emotion 30 (2006) 347
294 335. 363.
[6] S. Deterding, D. Dixon, R. Khaled, L. E. [17] M. Sailer, J. U. Hense, S. K. Mayr, H.
Nacke, From game design elements to Mandl, How gamification motivates: An
gamefulness: Defining "Gamification", experimental study of the effects of
Proceedings of the 15th International specific game design elements on
Academic MindTrek Conference on psychological need satisfaction,
Envisioning Future Media Environments Computers in Human Behavior 69 (2017)
(2011) 1 7. 371 380.
[7] C. Fornell, D. F. Larcker, Evaluating [18] M. Sailer, L. Homner, The Gamification
Structural Equation Models with of Learning: a Meta-analysis, Educational
Unobservable Variables and Psychology Review 32 (2019) 77 112.
Measurement Error, Journal of Marketing [19] S. Schöbel, A. Janson, K. Jahn, B.
Research 18 (1981) 39 50. Kordyaka, O. Turetken, N. Djafarova, M.
[8] M. D. Hanus, J. Fox, Assessing the effects Saqr, D. Wu, M. Söllner, M. Adam, P.
of gamification in the classroom: A Heiberg Gad, H. Wesseloh, J. Leimeister,
longitudinal study on intrinsic motivation, A Research Agenda for the Why, What,
social comparison, satisfaction, effort, and and How of Gamification Designs:
academic performance, Computers and Outcomes of an ECIS 2019 Panel,
Education 80 (2015) 152 161. Communications of the Association for
[9] J. Högberg, J. Hamari, E. Wästlund, Information Systems 46 (2020) 706 721.
Gameful Experience Questionnaire [20] A. Suh, C. Wagner, L. Liu, Enhancing
(GAMEFULQUEST): an instrument for User Engagement through Gamification,
measuring the perceived gamefulness of Journal of Computer Information Systems
system use, User Modeling and User- 58 (2018) 204 213.
Adapted Interaction (2019) 619 660. [21] P. Tiago, M. J. Gouveia, C. Capinha, M.
[10] R. Hunicke, M. LeBlanc, R. Zubek, MDA: Santos-Reis, H. M. Pereira, The influence
A Formal Approach to Game Design and of motivational factors on the frequency of
Game Research, Workshop on Challenges participation in citizen science activities,
in Game AI (2004) 1 4. Nature Conservation 18 (2017) 61 78.
[11] K. Huotari, J. Hamari, A definition for [22] H. Wesseloh, F. M. Stein, P. Szelat, M.
gamification: Anchoring gamification in Schumann, Boss Fights in Lectures! - A
the service marketing literature, Longitudinal Study on a Gamified
Electronic Markets 27 (2017) 21 31. Application for Testing Factual
[12] R. B. Kline. Principles and Practice of Knowledge, Proceedings of the 4th
Structural Equation Modeling, The International GamiFIN Conference
Guilford Press, New York, 2011. (2020) 31 40.
[13] J. Koivisto, J. Hamari, The rise of [23] N. Xi, J. Hamari, Does gamification
motivational information systems: A satisfy needs? A study on the relationship
review of gamification research, between gamification features and
International Journal of Information intrinsic need satisfaction, International
Management 45 (2019) 191 210. Journal of Information Management 46
[14] A. Lieberoth, Shallow Gamification (2019) 210 221.
Testing Psychological Effects of Framing