Games and gamification in flipped classrooms: A systematic review

Abstract

The landscape of education is experiencing a shift towards active learning approaches as the need for independent, lifelong learning increases. Traditional lecture-based teaching methodologies are not as effective at keeping students motivated enough to engage with content on a deep level. Therefore, approaches such as student-centred learning, self-directed learning, and flipped classrooms are becoming more popular as educators begin to embrace the idea of giving students more autonomy in the classroom. The popularity of gamification and games in education has led to them being used in conjunction with these active learning methods, however this area lacks a high-level view of present and future work. This study aims to bring clarity to this area of education by presenting a systematic review of the use of games and gamification in flipped classrooms. In general, the results show that current implementations have had positive outcomes, especially in terms of academic performance. The data also shows that the in-class component of flipped classrooms is more commonly gamified compared to the out-of-class component, and that achievement affordances and Kahoot! are popular motivational affordances to use. Further research is proposed concerning social affordances and increased reliance on theoretical foundations.

Keywords

• 1 Gamification
• game-based learning
• flipped classroom
• autonomous education
• systematic review

1. Introduction

The rapid advancement of technology in recent years has resulted in an increasing need for students to develop lifelong learning skills which will allow them to be adaptable in the working world [ 1,2 ]. This is also reflected in the fourth sustainable development goal (SDG #42) which calls for lifelong learning opportunities for all. Some of the core competencies required for lifelong learning include self-management, learning how to learn, and information acquisition skills [ 3 ]. Educators are realising that the best way to impart these kinds of skills is through active learning methodologies which are more capable of encouraging students to engage with content on a deeper level than traditional lecture-based methods [ 2 ].

Three educational approaches related to active learning are self-directed learning (SDL), studentcentred learning (SCL), and flipped classrooms (FC). SDL is an approach to teaching in which students function autonomously, taking a large amount of responsibility for their own learning [ 4 ]. Closely related to this is SCL, an approach to teaching in which the power of the learning process is shifted from the teacher to the student, thus giving the learner more control and responsibility [ 5,6 ]. Finally, the flipped classroom model involves moving the traditional teaching activities which are more passive to outside the classroom so that class time can be used for more active learning methods [ 7 ]. All three of these approaches involve a move away from traditional lecture-based teaching practices towards active learning methods. In the process, the learner becomes more autonomous, which can result in improved lifelong learning skills [ 8 ]. These types of educational approaches have grown in popularity, and the COVID-19 pandemic has served to increase educators’ reliance on nontraditional methods [ 9 ].

While these approaches all have the potential to improve learning outcomes, it remains difficult to motivate learners to take more responsibility for their learning since this requires more effort and it challenges their ideas of how teaching should work (that the teacher should be in charge) [ 10 ]. Games have been shown to be capable of motivating players effectively, according to selfdetermination theory [ 11 ]. As an offshoot of this, gamification and game-based learning (GBL) have been explored as a means by which the motivation of learners in educational contexts might be improved [ 12,13 ].

In order to position this study within the field of gamification, it is necessary to clarify our perspective with regards to the distinction between gamification and game-based learning. Within the literature, the difference is usually described in terms of parts or whole. In other words, game-based learning is considered a full game while gamification consists only of parts of a game [ 14 ]. However, we hold to the definition of gamification provided by [ 15 ] which describes the term as an umbrella concept encompassing any technology or practice which gives rise to experiences akin to games. Therefore, we will refer to both concepts as being part of the category of gamification for the remainder of this paper.

Given the broad definition of gamification, its potential to positively impact learning outcomes and motivation is a common area of research [ 17, 18, 19 ]. However, there is a dearth of studies that document the use of these approaches in conjunction with methods such as flipped classrooms or for making a course more studentcentred [ 20 ]. As a result, it is not yet clear how GBL and gamification are being used in these areas and what the outcomes of these applications are.

Therefore, in the interest of understanding the intersection between games and gamification in autonomous learning contexts, a systematic literature review was conducted. It focused on student-centred learning, self-directed learning, and flipped classrooms, and the ways in which games and gamification have been used in conjunction with these teaching methodologies. This article presents the results of a subsection of articles analysed in the review, focusing specifically on the use of games and gamification in flipped classrooms in order to describe the current state of this specific area of education.

When framed as an approach to active learning, the flipped classroom model is concerned with making better use of the time that students spend in contact with one another and with the teacher. This requires moving information-transmission teaching out of the class. This can serve to increase the autonomy of students by giving them more control over their own learning [ 21 ]. The flipped classroom can also be viewed as a means of democratising the classroom by making students contributors to the learning process [ 22 ]. From this critical pedagogy perspective, it is about more than simply creating independent learners, but about challenging the traditional view of educational settings in which an expert assumes complete ignorance in their student audience and takes it upon themself to remedy this [ 23 ]. As a result of doing away with passive teaching, the progressive teaching principles espoused by Dewey [ 24 ] can be adopted, thus creating the opportunity for students to become open-minded, flexible, and valuable contributors of society.

The flipped classroom model’s success relies upon students being motivated enough to spend significant amounts of time on out-of-class work [ 21,25 ]. Gamification has been used in both the in-class and out-of-class components of a flipped classroom, whether as a means of motivating students to prepare for class [ 20 ] or as a way of actively engaging them during class [ 26 ].

To the best of our knowledge, one other review on gamification in flipped classrooms exists [ 27 ]. Our review seeks to expand upon this contribution by also investigating the use of full games in flipped classrooms as well as conducting the search two years later. In addition, we aim to investigate the component of the flipped classroom which was gamified (in- or out-ofclass) and the types of study designs which were used.

The remainder of this paper is structured as follows: section 2 describes the methodology followed for the review; section 3 presents the results; section 4 provides a discussion of the applicability of these results; and section 5 concludes the paper by outlining future work.

2. Review process

The review was conducted according to the guidelines for an effective review [ 28 ]. The following section describes the steps that were followed during the review process, which is shown in Figure 1.

The literature search was conducted using the Scopus database. It was chosen because it is known to index many publishers, including those most relevant to this field of inquiry (IEEE, ACM, and Springer). The search query was constructed to address the intersection of games and gamification with three main areas of learning considered to be more autonomous than traditional educational experiences – flipped classrooms, SCL, and SDL. The search was conducted using the following search string in April 2021:

TITLE-ABS-KEY (game* OR gamif*) AND (((flipped* OR inverted*) AND (class* OR learning OR education OR instruction OR teaching))

OR ({self-directed learning} OR {self-directed instruction} OR {self-directed education} OR {self-directed teaching} OR {self-managed learning} OR {self-managed instruction} OR {self-managed education} OR {self-managed teaching} OR {independent learning} OR {independent instruction} OR {self-initiated learning} OR {self-initiated education} OR {selfinitiated instruction})

OR ("student centered" OR "student centred" OR "learner centered" OR "learner centred")).

The asterisk (*) was used in the search query to ensure that all variations of a term are included. For example, “gamification” as well as “gamified” would fit the query. 2.1.

Inclusion and exclusion criteria

The search was limited to English papers from journals and conferences, as well as book chapters. The inclusion criteria for papers were: 1. Either games or gamification had to be used in conjunction with flipped learning or for the purpose of making a course more studentcentred or students more self-directed (54 papers excluded). 2. The game or gamification had to be described in enough detail to allow it to be mapped, i.e., the main parts are described in order to allow the motivational affordances to be mapped by the reviewer (10 papers excluded). 3. The study had to include empirical results (descriptions of interventions which did not report results from testing were therefore excluded) (159 papers excluded). 4. If the paper was concerned with SCL or SDL, it had to engage with the concept by describing it in greater detail in the body of the paper instead of only mentioning it in the abstract or keywords (33 papers excluded). In addition, papers were excluded if: 1. They did not describe a formal education setting such as a kindergarten or school (K12) or a tertiary education environment (informal adult education and studies involving students outside of a formal course were excluded) (58 papers excluded). 2. They were concerned with situations where students created their own games to learn (10 papers excluded) 3. They were studies about students’ opinions about teaching methods without any actual changes to a course/classroom (6 papers excluded).

After applying the above-mentioned criteria, 92 papers remained for analysis (58.7% about flipped classroom; 21.7% about SCL; 14.1% about SDL; 4.3% about flipped classroom and SCL, and 1.1% about flipped classroom and SDL). The full list of included papers is available here: https://bit.ly/3lRelj0. This article focuses only on those papers relating to flipped classroom approaches. An analysis of the full sample will be the subject of future work.

Based on the guidelines of [ 28 ], a conceptcentric matrix was used to map the details of each paper. To prevent the introduction of bias, details such as motivational affordances and study outcomes were mapped as described by the authors without further analysis from the reviewer. The review process was conducted by the first author of this paper. Any mapping decisions were discussed amongst the research team.

3. Results

As mentioned before, this article will focus only on papers relating to flipped classroom environments (59 papers; 64.1% of the sample). It should be noted that one paper presented two studies which were mapped separately, hence the total number of studies included in the sample presented in this article is 60.

The presentation of the results will begin with an overview of the demographic data of the studies. Following this are the details of the motivational affordances used, the results of experimental studies and the parts of the flipped classroom that were modified by games and gamification. The terminology relating to motivational affordances and study outcomes is the same as that outlined by [ 29 ]. 3.1.

Demographic details

The popularity of using games and gamification in flipped classrooms has been steadily increasing since 2016, as shown in Figure 2. In 2020, the last full year that was reviewed in this study, the number of publications almost doubled from the previous two years.

Figure 2: The number of papers published per year (N = 59)

Most of the studies were conducted in Asia and Europe, with these contributing 18 papers each (30%), while studies in North America amounted to 8 papers (13.3%) and Africa, the Middle east and South America were each represented by 1 paper (1.7%). 13 studies (21.7%) did not specify where they were conducted.

Games and gamification have been employed in a wide variety of fields, as shown in Table 1, with computing being the most popular at a tertiary level. This could be because teachers of computing content are more likely to be comfortable with adding digital elements to their courses [ 30 ], and this is in line with other reviews on gamification in education [ 17 ]. At a K12 level (kindergarten to twelfth grade), social science and science are the main fields being gamified. Lastly, the use of gamification in flipped classrooms is more common at tertiary level (university/college) than at school level.

Table 2 shows the types of studies which have been employed – whether the game/gamification was combined with flipped classroom and then tested against a control of flipped classroom or a traditional class, or whether the gameful components were simply used as part of a flipped classroom, sometimes along with other educational approaches such as collaboration, peer instruction, and blended learning. The “other settings” study design includes several unique study design types (described below the table).

3.3. Experimental studies, affordances, and outcomes

3.2. Motivational affordances and study designs

Table 2 also shows the motivational affordances used (according to the mapping by [ 29 ]). The most used achievement affordances were points (18 studies; 30%), badges (15 studies; 25%), and leaderboards (13 studies; 21.6%). This also reflected other reviews of the use of gamification in education [ 17,30 ]. Social affordances included teamwork (9 studies; 15%) and competition (9 studies; 15%), while the most common immersion affordances were the use of narrative (8 studies; 13.3%) and role play (6 studies; 10%). The most common non-digital elements were physical dice (6 studies; 10%) and physical playboards and/or tokens (5 studies; 8.3%). These were often employed together in the form of boardgames. Finally, miscellaneous affordances included full commercial games or systems (such as Kahoot!, Socrative or word games). 27 studies (45%) used such games, with 14 (23%) of those using Kahoot!

The class component item in Table 2 describes which part of the flipped classroom was modified to include the motivational affordances. It tended to be more common to modify the in-class component of the course. This could be because motivational affordances could more easily be included in these settings, whereas out-of-class settings would require a digital system to keep track of student activity and provide access to the gameful components. .

23 studies (38.3%) were classified as experimental studies according to the following criteria: 1.

Clearly defined hypothesis or research question. 2. The use of a control group or pre-post-test design 3. The use of inferential statistics.

Figure 3 shows the mapping of these experimental studies according to the motivational outcomes used and the outcome of the study. Positive results imply that the applied motivational affordances resulted in improvements to the specific measured outcomes. Mixed results were mapped according to whether the majority of tests had yielded positive or negative results (negative meaning that the applied affordances had no significant measurable effect on the outcomes). Most of the studies reported positive results, i.e., the applied motivational affordances resulted in improvements to the specific measured outcomes, with achievement affordances and miscellaneous elements being the most popular affordances implemented. Most of the miscellaneous affordances used in experimental studies (8 studies out of 11) were full games.

In terms of the specific types of outcomes that were measured, across all 60 studies the highest measured outcomes were psychological states (25 studies; 41.6%), such as motivation and attitude towards the course content; and performance outcomes (38 studies, 63.3%), such as the level of academic performance, quality of work and level of participation in the course. Figure 4 shows the results of experimental studies organized according to outcome type. The popularity of psychological and performance outcomes is also visible here, and educational performance outcomes showed primarily positive results. Furthermore, there are some cases where studies reported no positive changes where games and gamification were added to a flipped classroom.

4. Discussion

Overall, the results suggest that games and gamification can have a positive effect in flipped classrooms, especially with regards to performance outcomes such as academic achievement.

The analysis revealed that more than one-third of the studies employed the design of combining games/gamification with the flipped classroom without a control group. While the use of a pre/post-test in these contexts made it possible to gauge whether the course was improved by the addition of the gamification, it was impossible to conclusively attribute the outcome to the gamification (compared to studies where a nongamified control group was used). According to [ 31 ], the design of the instructional materials in a flipped classroom have a great influence on the way in which students perceive the learning content. Therefore, it is recommended that future research involves more experimental designs with control groups to make it possible to isolate the effects of the gamification treatment.

Furthermore, the use of points, badges, and leaderboards (PBLs) as the most common affordances remains in line with other reviews on gamification in education. As shown by [ 32 ], the use of PBLs may not always be suitable for the context in which they are employed. For example, leaderboards have contributed to a number of negative effects in gamification studies [ 32 ]. PBLs fall under the category of achievement affordances [ 33 ]. The much lower incidence of immersion- and social-related affordances in the sample analysed indicates a gap in the field which also echoes the state of gamification across other fields [ 29 ]. The flipped classroom model makes it possible to utilise class time for more interactive, social learning activities such as group work and discussions [ 34 ]. In this sense, it may be simpler to implement social affordances in this kind of classroom setting when compared to traditional classes. As such, flipped classrooms are wellplaced to contribute to the field of gamification in this way [ 29 ]. In terms of the use of full games in flipped classrooms, Kahoot! was the primary choice for in-class engagement. The features of this system – free to access and easy to use – make it the ideal companion to in-class activities. 2 2 4 3 1 4 2 1 3 1 1 12 2 3 5 2 2 1 5 1 1 1 1 12

Non-digital elements 1 1 2 2 3 1 4 1 1 2 9

A literature review by [ 35 ] reports that Kahoot! can have a positive effect on learning when compared to other approaches. It also embraces the bring-your-own-device (BYOD) model which is already popular in education, especially at a tertiary level [ 36 ]. The studies that employed Kahoot! in the classroom used it mainly as a type of formative assessment to consolidate knowledge that students would have gained from engaging with the out-of-class material [ 37,38 ].

In general, the in-class component received more attention in terms of the use of games and gamification. Since the flipped classroom model only works if students prepare adequately for class, the use of motivational affordances to encourage this behaviour is an interesting line of inquiry which requires additional investigation [ 20 ].

The studies analysed reported few negative effects from the gamified interventions, although [ 39 ] reported that some students gamed the system in order to earn more badges. This is one of the known negative effects of gamification [ 32 ]. [ 40 ] found that the students only interacted with the gamified elements due to the competition they created, not because they associated it with improved learning. It is because of cases like these that it is important that the motivational affordances be intrinsically tied to the learning content to prevent students from bypassing the learning content while engaging with the game elements [ 16,41 ].

Finally, in terms of methodological approaches, the sample analysed contained no studies which attempted to isolate individual affordances to test their effectiveness. Within the field of gamification, calls have been made for these kinds of studies in order to understand how gamification works [ 29,42 ], and testing the effects of individual affordances is one of the main ways to achieve this. In addition, the current reliance on academic achievement as an outcome measure, while being the simplest way to determine whether an intervention has been effective, misses a deeper understanding of exactly how it has resulted in improved performance. A focus on measures such as psychological outcomes (motivation, attitude towards the content, selfregulation, engagement, confidence etc.) combined with engagement outcomes, such as time spent interacting with the motivational affordances, could shed light on the specific ways in which these affordances function to bring about improved performance from students.

The studies included also showed very little reliance on underlying theoretical work, such as self-determination theory or theories of engagement. In order to support the understanding of how gamification works in different contexts, it is important to base empirical work on firm theoretical assumptions [ 43 ].

5. Limitations

As with any study, this one is not without its limitations. Firstly, the search was limited to the Scopus database. The number of articles retrieved for the final analysis (92 articles in the full sample) is fairly substantial and Scopus is considered to index the most relevant publishers in this field. While this may serve to curtail the effect of this limitation on the findings, it is possible that some publications may have been missed, especially since snowball sampling was not conducted to find additional related studies. Secondly, the search string used may not have been sufficient to retrieve all possible matches to the query. To mitigate this limitation, the search string was revised and refined over several iterations to ensure that, as far as possible, all conceivable variations of the search terms were addressed.

6. Concluding remarks

This study has provided a systematic analysis of studies concerning the use of games or gamification in flipped classrooms within formal education settings. This is an emerging field of inquiry and could benefit from high-level overviews of prior work.

The results show that educators are beginning to embrace the idea of using games and gamification in flipped classrooms, with inclassroom activities being the primary focus. The outcomes of these studies are mostly positive, although mainly achievement affordances were used, and performance-based outcomes were tested. There is much room for variety in these areas.

These results contribute to the current understanding of the state of the field. Future work includes reporting on the full set of studies that were returned in the search query to gain a broader perspective on the use of games and gamification for autonomous learning.

7. References

References

[1] EDUCAUSE , 2020 EDUCAUSE Horizon Report: Teaching and Learning Edition , CO:EDUCAUSE, Boulder, 2020 . https://library.educause.edu/- /media/files/library/2020/3/2020_horizon_r eport_pdf.pdf ( accessed December 8 , 2021 ).

[2] S.L. Boyer , D.R. Edmondson , A.B. Artis , D. Fleming , Self-Directed Learning: A Tool for Lifelong Learning , Journal of Marketing Education . 36 ( 2014 ) 20 - 32 . https://doi.org/10.1177/0273475313494010

[3] A. Kaplan , Lifelong learning: conclusions from a literature review, International Online Journal of Primary Education (IOJPE ) ISSN: 1300 - 915X . 5 ( 2016 ). http://www.iojpe.org/index.php/iojpe/articl e/view/91 (accessed December 8 , 2021 ).

[4] S. Wilcox , Fostering self-directed learning in the university setting , Studies in Higher Education . 21 ( 1996 ) 165 - 176 . https://doi.org/10.1080/0307507961233138 1338.

[5] G. O 'Neill , T. McMahon , Student-centred learning: what does it mean for students and lecturers? , in: G. O'Neill , S. Moore , B. McMullin (Eds.), Emerging Issues in the Practice of University Learning and Teaching , AISHE , Dublin, 2005 : p. 10 .

[6] M. Weimer , Learner-Centered Teaching : Five Key Changes to Practice, 2nd ed., Jossey-Bass , San Francisco, 2013 .

[7] J.L. Bishop , M.A. Verleger , The flipped classroom: A survey of the research , in: ASEE National Cconference Proceedings, Atlanta , 2013 : pp. 1 - 18 .

[8] S. Yan , Teachers' Roles in Autonomous Learning, Journal of Sociological Research . 3 ( 2012 ) 557 - 562 .

[9] S. Pokhrel , R. Chhetri , A Literature Review on Impact of COVID-19 Pandemic on Teaching and Learning, Higher Education for the Future . 8 ( 2021 ) 133 - 141 . https://doi.org/10.1177/2347631120983481 .

[10] V.A. Clifford , The Development of Autonomous Learners in a University Setting, Higher Education Research & Development. 18 ( 1999 ) 115 - 128 . https://doi.org/10.1080/0729436990180109 .

[11] A.K. Przybylski , C.S. Rigby , R.M. Ryan , A motivational model of video game engagement ., Review of General Psychology . 14 ( 2010 ) 154 - 166 .

[12] P. Buckley , E. Doyle, Gamification and student motivation, Interactive Learning Environments . 0 ( 2014 ) 1 - 14 . https://doi.org/10.1080/10494820. 2014 . 964 263.

[13] J.-C. Woo , Digital Game-Based Learning Supports Student Motivation, Cognitive Success, and Performance Outcomes , Journal of Educational Technology & Society. 17 ( 2014 ) 291 - 307 .

[14] S. Deterding , D. Dixon , R. Khaled , L. Nacke , From game design elements to gamefulness: defining gamification , in: A. Lugmayr , H. Franssila , O. Sotamaa , C. Safran , T. Aaltonen (Eds.), Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments , ACM, Tampere, 2011 : pp. 9 - 15 . https://doi.org/10.1145/2181037.2181040.

[15] J. Hamari , Gamification, in: G. Ritzer (Ed.), The Blackwell Encyclopedia of Sociology , John Wiley & Sons, Ltd, Oxford, UK, 2019 : pp. 1 - 3 . https://doi.org/10.1002/9781405165518.wb eos1321.

[16] J.L. Plass , B.D. Homer , C.K. Kinzer , Foundations of Game-Based Learning, Educational Psychologist . 50 ( 2015 ) 258 - 283 . https://doi.org/10.1080/00461520. 2015 . 112 2533.

[17] D. Dicheva , C. Dichev , G. Agre, G. Angelova, Gamification in Education: A Systematic Mapping Study , ( 2015 ). http://www.researchgate.net/profile/Darina _Dicheva/publication/270273830_Gamific ation_in_Education_ A _Systematic_Mappi ng_Study/links/54c95c4b0cf2807dcc262a1 c. pdf (accessed August 17 , 2015 ).

[18] J. Hamari , J. Koivisto , H. Sarsa , Does Gamification Work? - A Literature Review of Empirical Studies on Gamification , in: R. Sprague (Ed.), 2014 47th Hawaii International Conference on System Sciences, IEEE, Waikoloa , 2014 : pp. 3025 - 3034 . https://doi.org/10.1109/HICSS. 2014 . 377 .

[19] D. Dicheva , K. Irwin , C. Dichev , Motivational factors in educational gamification , in: 2018: pp. 408 - 410 . https://doi.org/10.1109/ICALT. 2018 . 00102 .

[20] B. Huang , K.F. Hew , C.K. Lo , Investigating the effects of gamification-enhanced flipped learning on undergraduate students' behavioral and cognitive engagement, Interactive Learning Environments . 27 ( 2019 ) 1106 - 1126 . https://doi.org/10.1080/10494820. 2018 . 149 5653.

[21] L. Abeysekera , P. Dawson , Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research , Higher Education Research & Development. 34 ( 2015 ) 1 - 14 .

[22] J.D. Harden , Learning Without Sages? Reflections on “Flipping” the University Classroom, Alternate Routes: A Journal of Critical Social Research . 26 ( 2015 ).

[23] P. Freire , Pedagogy of the Oppressed, Bloomsbury Publishing USA, New Jersey, 2014 .

[24] E.A. Hopkins , John Dewey and Progressive Education, The Journal of Educational Thought . 50 ( 2017 ) 59 - 68 .

[25] D. Gross , E.S. Pietri , G. Anderson , K. Moyano-Camihort , M.J. Graham , Increased preclass preparation underlies student outcome improvement in the flipped classroom , CBE-Life Sciences Education . 14 ( 2015 ) 8 .

[26] A. Segura-Robles , A. Fuentes-Cabrera , M.E. Parra-González , J. López-Belmonte , Effects on personal factors through flipped learning and gamification as combined methodologies in secondary education , Frontiers in Psychology. 11 ( 2020 ) 1103 .

[27] M. Ekici , A systematic review of the use of gamification in flipped learning , Educ Inf Technol . 26 ( 2021 ) 3327 - 3346 . https://doi.org/10.1007/s10639-020-10394- y.

[28] J. Webster , R.T. Watson, Analyzing the Past to Prepare for the Future: Writing a Literature Review , MIS Quarterly . 26 ( 2002 ) xiii-xxiii.

[29] J. Koivisto , J. Hamari, The rise of motivational information systems: A review of gamification research , International Journal of Information Management . 45 ( 2019 ) 191 - 210 . https://doi.org/10.1016/j.ijinfomgt. 2018 . 10 . 013.

[30] C. Dichev , D. Dicheva , Gamifying education: what is known, what is believed and what remains uncertain: a critical review , International Journal of Educational Technology in Higher Education . 14 ( 2017 ) 9 .

[31] C.K. Lo , K.F. Hew , A critical review of flipped classroom challenges in K-12 education: possible solutions and recommendations for future research , Research and Practice in Technology Enhanced Learning . 12 ( 2017 ) 4 . https://doi.org/10.1186/s41039-016-0044- 2.

[32] A.M. Toda , P.H.D. Valle , S. Isotani, The dark side of gamification: An overview of negative effects of gamification in education , Communications in Computer and Information Science . 832 ( 2018 ) 143 - 156 . https://doi.org/10.1007/978-3- 319 - 97934- 2 _ 9 .

[33] N. Yee , Motivations for Play in Online Games , CyberPsychology & Behavior. 9 ( 2006 ) 54 .

[34] G. Akçayır , M. Akçayır , The flipped classroom: A review of its advantages and challenges , Computers & Education . 126 ( 2018 ) 334 - 345 . https://doi.org/10.1016/j.compedu. 2018 . 07 . 021.

[35] A.I. Wang , R. Tahir , The effect of using Kahoot! for learning - A literature review , Computers & Education . 149 ( 2020 ) 103818 . https://doi.org/10.1016/j.compedu. 2020 . 10 3818.

[36] L. Johnson , S. Adams Becker , V. Estrada , A. Freeman , NMC Horizon Report: 2015 Higher Education Edition , The New Media Consortium, Austin, Texas, 2015 .

[37] U. Durrani , Gamified Flipped Classroom Learning Approach: A Case Study of AJ University , in: 2019 IEEE International Conference on Engineering, Technology and Education (TALE) , IEEE, 2019 : pp. 1 - 5 .

[38] E.T. Ang , J.M. Chan , V. Gopal , N. Li Shia , Gamifying anatomy education , Clinical Anatomy . 31 ( 2018 ) 997 - 1005 . https://doi.org/10.1002/ca.23249.

[39] B. Huang , K.F. Hew , Implementing a theory-driven gamification model in higher education flipped courses: Effects on out-ofclass activity completion and quality of artifacts , Computers & Education . 125 ( 2018 ) 254 - 272 . https://doi.org/10.1016/j.compedu. 2018 . 06 . 018.

[40] A.Y. Gündüz , B. Akkoyunlu , Effectiveness of Gamification in Flipped Learning , SAGE Open . 10 ( 2020 ) 2158244020979837 .

[41] Y.B. Kafai , M.L. Franke , C.C. Ching , J.C. Shih , Game design as an interactive learning environment for fostering students' and teachers' mathematical inquiry , International Journal of Computers for Mathematical Learning . 3 ( 1998 ) 149 - 184 .

[42] L.E. Nacke , S. Deterding , The maturing of gamification research, Computers in Human Behavior . 71 ( 2017 ) 450 - 454 . https://doi.org/10.1016/j.chb. 2016 . 11 .062.

[43] A. Rapp , F. Hopfgartner , J. Hamari , C. Linehan , F. Cena , Strengthening gamification studies: Current trends and future opportunities of gamification research , International Journal of HumanComputer Studies . 127 ( 2019 ) 1 - 6 . https://doi.org/10.1016/j.ijhcs. 2018 . 11 .007.