Henrik Wesseloha, Niklas Buddensieka, Tim Pantela, Felix M. Steina, Phillip Szelata and Matthias Schumanna a University of Göttingen, Platz der Göttinger Sieben 5, Göttingen, 37073, Germany Scientific studies have shown that the use of gamification is effective in promoting motivation and behavior. However, reasons for the effectiveness of the implemented game elements are analyzed insufficiently. The following article therefor addresses the issue of gameful perception induced through gamification by using an experimental, single factor between-subjects design. First, we draw on Self-Determination Theory to derive a grounded research model. Then, we investigate two groups by using a learning application with a gamified and non-gamified version to test our hypotheses. Our results show that the users gameful perception of an activity mediates the motivational outcomes. We discuss our findings and relate these to design to outline pitfalls and the importance of evoking a conscious gameful experience. Gamification, Intrinsic Motivation, SDT, Gameful Perception, SEM unknown to what extent certain factors besides gameful affordances influence the motivational As an important instrument for fostering or impact. These aspects are however needed to maintaining human motivation and behavior, explain the different outcomes among similar gamification, i.e. the use of game design game design elements [13, 18]. For this study elements in a non-game context [6] to evoke we therefor analyze whether the motivational gameful experiences [11], has gained more effect of gamification depends on a gameful attention in recent years. Gamification is used perception. We assume that in order to benefit across different domains, e.g. education, health, from gamification, it is mandatory that users or crowdsourcing, to support psychological and also perceive the activity as gameful. Thus, we behavioral outcomes, like e.g. engagement or address the following research question: participation [13]. Previous research has shown that gamification can also help to satisfy RQ: How does a gameful perception influence intrinsic psychological needs like autonomy, the motivational effect of gamification? competence or relatedness and thus support intrinsic motivation [4, 15, 17, 23]. To address this question, we briefly want to However, the knowledge about how exactly give an overview of related research and derive gamification works is still incomplete [19]. a research model based on Self-Determination Systematic reviews on gamification research Theory (SDT) [4] as theoretical foundation. We mostly report mixed results for the then present our single factor between-subjects effectiveness [13]. Even though the number of research design by describing the experiment studies analyzing gamifications effectiveness is procedure, the measures for data collection and growing, academia still misses insights: It is yet provide a descriptive analysis of participants. 5th International GamiFIN Conference 2021 (GamiFIN 2021), April 7-10, 2021, Finland. EMAIL: henrik.wesseloh@uni-goettingen.de 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) To validate the mediation model, we follow a perceived as informative, it would promote the structural equation modeling approach [2, 12]. experience of competence [4, 5]. Moreover, we do a mean comparison, to find Mekler et al. (2017) analyzed motivational differences in motivational effect of a gamified effects of individual game elements and found and a non-gamified application. Finally, we that, despite their supposed external incentive, discuss our findings and conclude this paper to points, level and the leaderboard did not lead to provide implications for future research. a negative (overjustification) effect on intrinsic Our findings contribute to current research motivation [15]. They attributed this result to a analyzing the effect of gamification on intrinsic rather low gameful experience during the task. motivation and need satisfaction, because they Sailer et al. (2017) did a comparable study provide an understanding of why gamification and found positive effects of gamification on studies might fail measuring positive outcomes competence and relatedness. Furthermore, they [13]: When analyzing the effect of (individual) noticed that some participants did not recognize game elements on psychological outcomes, we the game elements. The authors attributed this often miss on the actual emersion of a conscious result to potential flaws in their gamification gameful perception within the participants. design and assumed that it had resulted in a lower motivational effect [17]. Furthermore, the contributions from Suh et al. (2018) and Xi & Hamari (2019) both provide positive empirical insights on how gamification satisfies the three intrinsic needs [20, 23]. These Recent empirical studies on the effects of authors point out that the need for further gamification on intrinsic motivation are mostly research on potential moderators or mediators based on SDT and its subtheories and therefore remains. also examine the basic needs for autonomy, competence or relatedness [cf. 15, 17, 20, 23]. Autonomy refers to a psychological freedom to perform an activity at own will, while This section depicts the research model, which competence describes the effectiveness of an we briefly outline in the following. individual doing this activity. Relatedness For our model, we consider gamification as covers the social aspect of an activity. In a manifest, exogenous variable that represents combination, these factors build the concept of the source of a gameful perception. To simplify, intrinsic motivation, which refers to an inherent gamification is modelled as binary variable, so interest or enjoyment towards an activity itself an application or service will only vary between and not for its instrumental value [3, 4]. the gamified (1) and the non-gamified (0) state. While an older study from Hanus & Fox [8] Perception of gamefulness describes to what showed that gamification had a negative effect extent individuals consciously notice a game- on intrinsic motivation recent studies presented like situation when interacting with a system or positive effects. We outline a few contributions service. This perception is supported e.g. by which stressed the importance of feelings of challenge, competition, immersion perception for intrinsic need satisfaction in a and social experiences, as well as playfulness chronological order. [9]. As gameful designers, we can address these Lieberoth (2015) hypothesized that framing different experiences when implementing game an activity as a game can lead to behavioral mechanics in a utilitarian system or service changes and subjective experiences of intrinsic [10]. Therefore, we state our first hypothesis: motivation [14]. The study showed that the perception of a playful frame for an mundane H1: Gamification has a positive impact on the task has a significant impact on enjoyment. perception of gamefulness. Deterding (2015), suggested gamification to support experiences of competence or inhibit Furthermore, Deci & ly autonomy, depending on perception. Cognitive Evaluation Theory, CET) postulates He draws on SDT to argue that, if gamification that psychological need satisfaction consists of is perceived by users as controlling, this would the fulfillment of autonomy, competence and have a negative influence on the experience of relatedness [4]. autonomy. If the gameful feedback in turn is influences the satisfaction of intrinsic needs [4], gamification can be used to create beneficial conditions that support feelings of autonomy, competence and relatedness [5, 15, 17, 20, 23]. We used a single factor between-subjects study For example, points, badges, or rankings are design to answer the research question and test common feedback elements and should thus our model hypotheses. Hence, the participants support the need of competence [4]. These were divided into two groups: experimental and elements, however, do not have to inevitably control. Both groups performed the same originate from games and thus might not be experiment, in which they answered general perceived per se as gameful affordance. We knowledge questions with our prototypical take this instant as a reason why gameful application. We treated the experimental group approaches led to different results despite using with a gamified version, while the control comparable game elements within similar cases group used a non-gamified version. After the [13, 18]. So, to benefit from the motivational experiment, both groups were asked to fill out pull [16] of games, we assume that users need an online survey to determine their experiences. to perceive the use of a gamified application as These survey results were then analyzed using gameful. Consequently, we state our second statistical methods. research hypothesis: H2a-c: Perception of gamefulness mediates needs of autonomy (a), The laboratory experiment is based on the use competence (b) and relatedness (c). of a mobile web application for knowledge retrieval that uses collaborative and competitive Moreover, we want to address the support of game design elements to motivate learning. We intrinsic motivation as a desirable motivational developed this learning app for lectures [22], so effect. In theory, the three psychological needs mainly groups of 10 or more students use it. of autonomy, competence and relatedness are During a session, participants answer randomly required determinants for intrinsic motivation assigned, multiple-choice questions prepared [4]. To by lecturer on their mobile device, in their own SDT suggests to measure enjoyment, as they pace, with a maximum duration of 30 seconds are closely associated [3]. Recent studies on per question. These question sessions are set to gamification have used this operationalization last between three to five minutes. After a to prove its motivational effects and concluded session, the participants receive a summary of that supporting need satisfaction may improve their performance and solutions for their enjoyment and, thus, intrinsic motivation [20]. individual questions, while lecturers get access For that reason, we hypothesize: to aggregated performance data. The gamified version additionally includes H3a-c: Satisfying the need of autonomy (a), different game design elements to provide a competence (b), and relatedness (c), has a gameful experience for its users and thus positive effect on intrinsic motivation. support the feeling of autonomy, competence, and relatedness. We integrated feedback Figure 1 visualizes our derived research model elements like, points (for correct answers incl. with the different hypotheses to test. streaks), badges, and a leaderboard to support experiences of achievement and competition. the participants via projector [#3,4], the start To support immersion, users find a narrative screen for participants [#5,6], the gameful setting and can choose their respective gameful feedback for participants [#7] and the gameful avatar. Furthermore, the session resembles a post-game screen with the leaderboard [#8]. challenging quest, in which the students team Our plan was to use the application in two up to encounter a virtual boss enemy, which distinct lectures with live attendees, to set up resembles the question pool. During this boss the two groups for the experiment. Due to the battle, each student contributes individually to ongoing corona pandemic, however, no live overcome the boss by answering questions lectures were held, so we decided to adapt the correctly but can be eliminated after a defined experiment to take place online virtually. To number of mistakes (virtual lives). The students simulate other participants during the question win the boss event collaboratively by solving a session, we integrated bots in our application. defined number of questions during the set This way, we were able to allow individuals or time. Overall, we assume, these achievement- smaller groups to participate in the online related, immersive and social features are able experiment and still maintain the social aspect to of our gamification concept. Additionally, we in the educational context [23]. setup a question pool of 45 general knowledge In contrast, the non-gamified version only questions in the field of sports, politics, history, shows a timer to indicate the remaining time and science to reach out for a broader audience. during a question session. All gamification We shared our invitation to participate in the features like the narrative or the feedback experiment via social media. The individuals elements are omitted. Figure 2 shows the visual and small groups that joined our experiment via differences between the gamified and non- a video conference platform that allows screen gamified application. The screenshots display sharing (of the lecturer screen) were randomly the quiz question design on a mobile device assigned to the experimental or control group. (e.g., smartphone) [#1,2], the lecturer screen Before each question session, we informed the during a question session, which is presented to participants about the procedure of the experiment and briefly presented the respective all items are self-reported experience measures, application. Then, the question session started we carefully translated the items to German for with a duration of 4 minutes (experimental participants to comprehend. group) or 3 minutes (control group)2. To measure the perception of gamefulness, After participants had seen their individual we selected items from the GAMEFULQUEST (gameful) results, the app provided a button scale for playfulness [9]. We did not consider linked to the online survey. We used this link to other game-specific items to prevent confusion add a parameter of each ID to be in the control group. To operationalize the SDT able to connect the self-reported survey data perspective of intrinsic motivation, we chose with actual performance and session data items from the Intrinsic Motivation Inventory provided be the app. (IMI) and respective contributions [1, 3, 21]. For the construct of autonomy the Ubisoft Perceived Experience Questionnaire (UPEQ) was used, which adapts the autonomy measures The data collection following the experiment to video games [1]. The construct of relatedness was carried out using an online survey. All was measured using an adapted questionnaire items were measured with a seven-point Likert of the IMI for group work scale (completely disagree (1) to completely [21]. The third SDT construct, competence, was agree (7)). We used five items per latent measured using the perceived competence construct to ensure an appropriate length of the scale from the IMI [3]. Additionally, we used questionnaire but also to leave additional room the IMI scale for enjoyment / interest to for the removal of weak or unsuitable items. As operationalize intrinsic motivation [3]. 2 We chose this distinction, as the gamified session can end early due to its winning condition. In average, this resulted in the same playing time. In addition to these measures, we asked six they played several times a week and 26% control questions to differentiate between the several times a month . The largest proportion respondents. These controls include gender, age of respondents (33%) said they played games and highest educational level. Furthermore, the less often or never . Some participants (34%) general affinity towards play was queried, with also had used our application before, but for the users specifying how often they play games majority (64%) the app was new. during the week and whether participants had used our application before. Due to cross-loadings (cf. 4.1 Measurement Model) and/or poor factor loadings (< 0.70) we To test the postulated hypotheses from our had to remove certain items during data research model, we used a structural equation analysis to ensure high construct reliability. modelling (SEM) approach [2, 12]. The data firms internal consistency analysis was done in Stata 16 and SPSS 26. To for each construct 0.828). assess the quality of the measurement model, Table 1 shows the items used as reflective we performed an exploratory factor analysis in measures for the latent constructs (italic items SPSS. Furthermore, we compared the survey had to be removed during factor analysis). results between the two groups with a mean comparison. Subsequently, the structural model was estimated using the maximum likelihood estimation (MLE) as the default method [12] in Overall, the distribution of participants between Stata. Accordingly, the SEM requirements of a the experimental and comparison group was large sample size, multivariate normality and balanced with 105 and 93 respondents. The correct model specification [12] were met, as experiment was taken by 43 individuals, 23 the next section shows. groups of two, 15 groups of three and 12 groups with four or more participants. The largest group in our sample held twelve participants. The age of the participants was highly The SEM approach requires, the constructs to concentrated in the range of 20 to 30 years, the be tested for their content, convergent, and average age was 30.6 years (SD = 13.57), the discriminant validity in order to assess the fit of median was 25 years. Whereas 134 participants the model with the collected data [2]. were between the age of 20 and 30 years, only First, we assume content validity, because 48 participants were older than 30. Males were of our item selection from validated, existing slightly overrepresented (60%) compared with scales like IMI [3] and GAMEFULQUEST [9]. females (39%), while two participants replied Second, we tested empirically our sample their gender as diverse (1%). for convergent validity by determining the The participants mainly have an academic individual item reliability, composite reliability background: 43% had a bachelor's degree, 22% (CR) and average variance extracted (AVE) by a master's or diploma. 21% of the respondents executing a factor analysis. The sample data indicated that their highest educational level was suitable for factor analysis, as the Kaiser- was the high school graduation. The most Meyer-Olkin Measure of Sampling Adequacy common field of study was economics (51%), (KMO) is greater than 0.60 (KMO = 0.834). In while other fields of study were mainly in the case of individual item reliability, we follow humanities and social sciences as well as the criteria of high factor loadings (> 0.70) [12], teacher training and . which is why a few items (POG3; A3; A4; A5; During the question sessions, the participants C5; IM3; IM4) had to be removed from the answered 12.8 (SD = 4.81) questions after 14.6 initial model. Afterwards we determined values (SD = 5.39) seconds on average. The relative for CR that ranged from 0.764 to 0.932, which success rate, i.e., the ratio of correct responses is above the acceptable limit (> 0.70) [12]. compared to the total number of answered Additionally, we calculated the AVE for each questions, was 54%. construct, which also exceeded the required About 22% of the participants had a high threshold (> 0.50) in all cases, except for the affinity towards play and stated that they played perception of gamefulness (AVE = 0.462), games such as board games or video games which is slightly below the limit. Still, this every day , whereas 19% of participants said constructs convergent validity can be regarded adequate [7], due to its higher composite any squared correlation with another construct, reliability (CR = 0.764) . which serves as a prove of discriminant validity Third, we determined discriminant validity at construct level. by checking for possible cross-loadings via the Table 2 and Table 3 present the results from pattern matrix. For the adjusted model, all items our factor analysis, showing the relevant scores showed the highest loadings for their respective for CR, AVE, Alpha ( , as well as construct. Additionally, we draw upon the cross-loadings and inter-construct correlations criterion from Fornell & Larcker [7] to show for the Fornell-Larcker-criterion [7]. that the AVE of each construct is higher than As last data analysis in the measurement model, competence ( = 0.034; p = 0.647). In case of we did an independent-samples t-test with a relatedness, however, gamification showed a 95% confidence interval to compare the means significant direct effect = 0.163; p = 0.013). of the survey results between the gamified, In addition, we determined significant indirect experimental group (n = 105) and the non- effects of gamification on autonomy ( = 0.114; gamified, control group (n = 93). In advance, p = 0.004), as well as on relatedness = 0.166; the requirement p = 0.002), but not on competence ( = 0.038; of equal variances. p = 0.107). According to this result, POG fully On the one hand, the results of the mean mediates the relationship between gamification comparison identified differences between the and autonomy but only partially mediates the gamified and non-gamified group considering a effect of gamification on relatedness. Thus, our gameful perception and intrinsic motivation hypotheses (H2a-c) are just partly supported. (both p 0.05). Especially for the construct of As postulated by SDT [4], we are also able relatedness, the item means differ significantly to prove the relationship of intrinsic motivation (p 0.01). On the other hand, we were, not able and needs of = 0.163; p = 0.021), to measure any differences for the constructs of social autonomy or competence. .395; p = 0.000). Thus, the Table 4 shows the results of the mean hypotheses derived from SDT (H3a-c) are also comparison with sample means for the control supported. Moreover, neither of the measured group ( 0) and experimental group ( 1) and control variables has a significant positive their respective confidence intervals (CI). association (p 0.05) with POG or any other latent construct. In terms of model fit, our structural model shows good to adequate values for different We then estimated the revised model with MLE quality criteria [12]: RMSEA = 0.073 (< 0.08), and determined the model fit indices. The CFI = 0.912 (> 0.9), TLI = 0.896 (~ 0.9), and results for our structual model indicate effects SRMR = 0.077 (< 0.08). Figure 3 shows the of gamification on gameful perception (POG) estimated model with standardized coefficients. ( = 0.254; p = 0.001), which was expectable. Thus, hypothesis H1 is supported. Furthermore, we found that gameful perception affected the experience of some basic psychological needs. Overall, we found an evidence for a mediating The influence of POG on relatedness showed the strongest effect ( 54; p = 0.000), gamification and intrinsic need satisfaction. We whereas the impact on experience of autonomy interprete that the reason for the full mediation seemed moderate ( ). In in case of autonomy lies in the characteristic of contrast, the relationship between a gameful play itself, which is regarded as a voluntary perception and experiencing competence could activity (and thus autonomous) [16]. Therefore, not be confirmed ( = 0.149; p = 0.062). When only if users perceive their activity as gameful, analyzing for mediating effects, we did not find it likely that they actually feel self-determined significant direct effects between gamification [4]. The reason for partial mediation in case of and either autonomy ( = 0.102; p = 0.150) or relatedness can be explained by the design of our gamified app. The leaderboard or avatars, was statistically not significant could lie in the e.g., reveal that other users are also involved in group composition, as the affinity to play was a session. Therefore, regardless of a gameful about 15% lower in the gamified group perception, users can still feel connected [4]. (M = 2.486) compared to the non-gamified Additionally, the relatedness is reinforced by a group (M = 2.925). collaborative competition to beat the boss and win the quiz, which likely fuels their gameful perception [9]. This seems reasonable for us, but, some unexpected findings need discussion. The objective of this study was to analyze how First, the mean comparison did not show differences for autonomy and competence motivational effect. Our results indicate the between the groups. In the case of competence, importance of a gameful perception in context the bots in the gamified version could be a of a gamified learning app to foster intrinsic possible cause for the missing change, as they motivation. The results, however, underly a few limitations which we briefly want to address to average response time was 5.7 seconds lower, provide some ideas for future research. while having a success rate of 75%. In During the experiment, both applications for comparison to the 52% success rate of the knowledge retrieval represented a quiz, which participants, the integrated bots seemed a bit can be considered per se as a game. In context of education, however, we want to point out the of the leaderboard, they may have inhibited the ulitarian (and not hedonic) purpose to determine learning outcomes. That is why we expected to emerge from the gameful feedback refer to the control group as non-gamified in [22]. This flaw in our app design might have led this study. Still, in other scenarios it might be to the low (non-significant) mediating effect. reasonable to compare low and high degrees of One reason for the lack of change in autonomy gamification to determine gameful perception. could also lie in the gameful design of the Furthermore, we want to stress out, that we application: The gamified app might not have selected and translated items from different, provided enough choice to support a strong validated scales. To feeling of autonomy [17]. For this aspect, the perception, we only considered playfulness as two groups were only treated differently due to an indicator. Still, there are also other possible the gamified character selection. dimensions that induce a gameful experience Second, the experience of relatedness which researchers can use to employ a fuller differed strongly between the groups. In this measure of gamefulness as a mediator [9]. case, the effect of gamification might be Finally, we want to encourage researchers to skewed, because the group sizes were slightly consider gameful perception when analyzing larger in the experimental group with 4.22 motivational effects of gamification in future participants than in the comparison group with (replication) studies. Studies that examine the 2.40 participants in average. In general, this circumstances where individual game elements small difference in the social environment evoke a gameful perception and thus support during the experiment might have moderated intrinsic needs are still scarce. We also suggest the effect. It therefore remains uncertain to what to compare gamification on gamers extent gamification was the underlying factor and non-gamers need satisfaction, as both target for the change in social relatedness. groups seem to have a different perspective on Third, the control variable of affinity to gamification. play was slightly below the borderline to show a significant effect on gameful perception ( = 0.154; p = 0.054). We think that the affinity to play, however, can have an impact on [1] A. Azadvar, A. Canossa, UPEQ: Ubisoft gameful perception, as gamers have a wider perceived experience questionnaire: A experience of what an actual game situation self-determination evaluation tool for feels like compared to non-gamers. Therefore, video games, ACM Internat. Conference gamers might perceive a gameful situation Proceeding Series (2018) 1 8. easier as they can relate them to other game [2] K. A. Bollen. Structural Equations with experiences. A possible reason, why this effect Latent Variables, Wiley, New York, 1989. [3] E. L. Deci, H. Eghrari, B. C. Patrick, D. R. an Activity as a Game, Games and Culture Leone, Facilitating Internalization: The 10 (2015) 229 248. Self-Determination Theory Perspective, [15] E. D. Mekler, F. Brühlmann, A. N. Tuch, Journal of Personality 62 (1994) 119 K. Opwis, Towards understanding the 142. effects of individual gamification [4] E. L. Deci, R. M. Ryan, The "What" and elements on intrinsic motivation and "Why" of Goal Pursuits: Human Needs performance, Computers in Human and the Self-Determination of Behavior, Behavior 71 (2017) 525 534. Psychological Inquiry 11 (2000) 227 268. [16] R. M. Ryan, C. S. Rigby, A. Przybylski, [5] S. Deterding, The Lens of Intrinsic Skill The motivational pull of video games: A Atoms: A Method for Gameful Design, self-determination theory approach, Human-Computer Interaction 30 (2015) Motivation and Emotion 30 (2006) 347 294 335. 363. [6] S. Deterding, D. Dixon, R. Khaled, L. E. [17] M. Sailer, J. U. Hense, S. K. Mayr, H. Nacke, From game design elements to Mandl, How gamification motivates: An gamefulness: Defining "Gamification", experimental study of the effects of Proceedings of the 15th International specific game design elements on Academic MindTrek Conference on psychological need satisfaction, Envisioning Future Media Environments Computers in Human Behavior 69 (2017) (2011) 1 7. 371 380. [7] C. Fornell, D. F. Larcker, Evaluating [18] M. Sailer, L. Homner, The Gamification Structural Equation Models with of Learning: a Meta-analysis, Educational Unobservable Variables and Psychology Review 32 (2019) 77 112. Measurement Error, Journal of Marketing [19] S. Schöbel, A. Janson, K. Jahn, B. Research 18 (1981) 39 50. Kordyaka, O. Turetken, N. Djafarova, M. [8] M. D. Hanus, J. Fox, Assessing the effects Saqr, D. Wu, M. Söllner, M. Adam, P. of gamification in the classroom: A Heiberg Gad, H. Wesseloh, J. Leimeister, longitudinal study on intrinsic motivation, A Research Agenda for the Why, What, social comparison, satisfaction, effort, and and How of Gamification Designs: academic performance, Computers and Outcomes of an ECIS 2019 Panel, Education 80 (2015) 152 161. Communications of the Association for [9] J. Högberg, J. Hamari, E. Wästlund, Information Systems 46 (2020) 706 721. Gameful Experience Questionnaire [20] A. Suh, C. Wagner, L. Liu, Enhancing (GAMEFULQUEST): an instrument for User Engagement through Gamification, measuring the perceived gamefulness of Journal of Computer Information Systems system use, User Modeling and User- 58 (2018) 204 213. Adapted Interaction (2019) 619 660. [21] P. Tiago, M. J. Gouveia, C. Capinha, M. [10] R. Hunicke, M. LeBlanc, R. Zubek, MDA: Santos-Reis, H. M. Pereira, The influence A Formal Approach to Game Design and of motivational factors on the frequency of Game Research, Workshop on Challenges participation in citizen science activities, in Game AI (2004) 1 4. Nature Conservation 18 (2017) 61 78. [11] K. Huotari, J. Hamari, A definition for [22] H. Wesseloh, F. M. Stein, P. Szelat, M. gamification: Anchoring gamification in Schumann, Boss Fights in Lectures! - A the service marketing literature, Longitudinal Study on a Gamified Electronic Markets 27 (2017) 21 31. Application for Testing Factual [12] R. B. Kline. Principles and Practice of Knowledge, Proceedings of the 4th Structural Equation Modeling, The International GamiFIN Conference Guilford Press, New York, 2011. (2020) 31 40. [13] J. Koivisto, J. Hamari, The rise of [23] N. Xi, J. Hamari, Does gamification motivational information systems: A satisfy needs? A study on the relationship review of gamification research, between gamification features and International Journal of Information intrinsic need satisfaction, International Management 45 (2019) 191 210. Journal of Information Management 46 [14] A. Lieberoth, Shallow Gamification (2019) 210 221. Testing Psychological Effects of Framing