=Paper=
{{Paper
|id=Vol-3147/paper4
|storemode=property
|title=Games and gamification in flipped classrooms: A systematic review
|pdfUrl=https://ceur-ws.org/Vol-3147/paper4.pdf
|volume=Vol-3147
|authors=Annique Smith,Nikoletta Zampeta Legaki,Juho Hamari
|dblpUrl=https://dblp.org/rec/conf/gamifin/SmithLH22
}}
==Games and gamification in flipped classrooms: A systematic review==
https://ceur-ws.org/Vol-3147/paper4.pdf
Games and gamification in flipped classrooms: A systematic
review
Annique Smith 1,2, Nikoletta Zampeta Legaki 2 and Juho Hamari 2
1
Department of Information Science, Virtual Reality and Interaction Lab, University of Pretoria, 2 Lynnwood
Road, Pretoria, 0181, South Africa
2
Gamification Group, Tampere University, Kalevantie 4, 33100, Tampere, Finland
Abstract
The landscape of education is experiencing a shift towards active learning approaches as the
need for independent, lifelong learning increases. Traditional lecture-based teaching
methodologies are not as effective at keeping students motivated enough to engage with content
on a deep level. Therefore, approaches such as student-centred learning, self-directed learning,
and flipped classrooms are becoming more popular as educators begin to embrace the idea of
giving students more autonomy in the classroom. The popularity of gamification and games in
education has led to them being used in conjunction with these active learning methods,
however this area lacks a high-level view of present and future work. This study aims to bring
clarity to this area of education by presenting a systematic review of the use of games and
gamification in flipped classrooms. In general, the results show that current implementations
have had positive outcomes, especially in terms of academic performance. The data also shows
that the in-class component of flipped classrooms is more commonly gamified compared to the
out-of-class component, and that achievement affordances and Kahoot! are popular
motivational affordances to use. Further research is proposed concerning social affordances and
increased reliance on theoretical foundations.
Keywords1
Gamification, game-based learning, flipped classroom, autonomous education, systematic
review
1. Introduction to impart these kinds of skills is through active
learning methodologies which are more capable
of encouraging students to engage with content on
The rapid advancement of technology in recent
a deeper level than traditional lecture-based
years has resulted in an increasing need for
methods [2].
students to develop lifelong learning skills which
Three educational approaches related to active
will allow them to be adaptable in the working
learning are self-directed learning (SDL), student-
world [1,2]. This is also reflected in the fourth
centred learning (SCL), and flipped classrooms
sustainable development goal (SDG #4 2) which
(FC). SDL is an approach to teaching in which
calls for lifelong learning opportunities for all.
students function autonomously, taking a large
Some of the core competencies required for
amount of responsibility for their own learning
lifelong learning include self-management,
[4]. Closely related to this is SCL, an approach to
learning how to learn, and information acquisition
teaching in which the power of the learning
skills [3]. Educators are realising that the best way
6th International GamiFIN Conference 2022 (GamiFIN 2022),
April 26-29 2022, Finland
EMAIL: annique.smith@up.ac.za (A. Smith);
zampeta.legaki@tuni.fi (N.Z. Legaki); juho.hamari@tuni.fi (J.
Hamari)
ORCID: 0000-0002-7887-9655 (A. Smith); 0000-0002-2707-
8364 (N.Z. Legaki); 0000-0002-6573-588X (J. Hamari)
© 2022 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
2
https://sdgs.un.org/goals/goal4
33
process is shifted from the teacher to the student, Therefore, in the interest of understanding the
thus giving the learner more control and intersection between games and gamification in
responsibility [5,6]. Finally, the flipped classroom autonomous learning contexts, a systematic
model involves moving the traditional teaching literature review was conducted. It focused on
activities which are more passive to outside the student-centred learning, self-directed learning,
classroom so that class time can be used for more and flipped classrooms, and the ways in which
active learning methods [7]. All three of these games and gamification have been used in
approaches involve a move away from traditional conjunction with these teaching methodologies.
lecture-based teaching practices towards active This article presents the results of a subsection of
learning methods. In the process, the learner articles analysed in the review, focusing
becomes more autonomous, which can result in specifically on the use of games and gamification
improved lifelong learning skills [8]. These types in flipped classrooms in order to describe the
of educational approaches have grown in current state of this specific area of education.
popularity, and the COVID-19 pandemic has When framed as an approach to active
served to increase educators’ reliance on non- learning, the flipped classroom model is
traditional methods [9]. concerned with making better use of the time that
While these approaches all have the potential students spend in contact with one another and
to improve learning outcomes, it remains difficult with the teacher. This requires moving
to motivate learners to take more responsibility information-transmission teaching out of the
for their learning since this requires more effort class. This can serve to increase the autonomy of
and it challenges their ideas of how teaching students by giving them more control over their
should work (that the teacher should be in charge) own learning [21]. The flipped classroom can also
[10]. Games have been shown to be capable of be viewed as a means of democratising the
motivating players effectively, according to self- classroom by making students contributors to the
determination theory [11]. As an offshoot of this, learning process [22]. From this critical pedagogy
gamification and game-based learning (GBL) perspective, it is about more than simply creating
have been explored as a means by which the independent learners, but about challenging the
motivation of learners in educational contexts traditional view of educational settings in which
might be improved [12,13]. an expert assumes complete ignorance in their
In order to position this study within the field student audience and takes it upon themself to
of gamification, it is necessary to clarify our remedy this [23]. As a result of doing away with
perspective with regards to the distinction passive teaching, the progressive teaching
between gamification and game-based learning. principles espoused by Dewey [24] can be
Within the literature, the difference is usually adopted, thus creating the opportunity for students
described in terms of parts or whole. In other to become open-minded, flexible, and valuable
words, game-based learning is considered a full contributors of society.
game while gamification consists only of parts of The flipped classroom model’s success relies
a game [14]. However, we hold to the definition upon students being motivated enough to spend
of gamification provided by [15] which describes significant amounts of time on out-of-class work
the term as an umbrella concept encompassing [21,25]. Gamification has been used in both the
any technology or practice which gives rise to in-class and out-of-class components of a flipped
experiences akin to games. Therefore, we will classroom, whether as a means of motivating
refer to both concepts as being part of the category students to prepare for class [20] or as a way of
of gamification for the remainder of this paper. actively engaging them during class [26].
Given the broad definition of gamification, its To the best of our knowledge, one other review
potential to positively impact learning outcomes on gamification in flipped classrooms exists [27].
and motivation is a common area of research [17, Our review seeks to expand upon this contribution
18, 19]. However, there is a dearth of studies that by also investigating the use of full games in
document the use of these approaches in flipped classrooms as well as conducting the
conjunction with methods such as flipped search two years later. In addition, we aim to
classrooms or for making a course more student- investigate the component of the flipped
centred [20]. As a result, it is not yet clear how classroom which was gamified (in- or out-of-
GBL and gamification are being used in these class) and the types of study designs which were
areas and what the outcomes of these applications used.
are.
34
The remainder of this paper is structured as TITLE-ABS-KEY (game* OR gamif*)
follows: section 2 describes the methodology AND
followed for the review; section 3 presents the (((flipped* OR inverted*) AND (class* OR
results; section 4 provides a discussion of the learning OR education OR instruction OR
applicability of these results; and section 5 teaching))
concludes the paper by outlining future work. OR
({self-directed learning} OR {self-directed
2. Review process instruction} OR {self-directed education} OR
{self-directed teaching} OR {self-managed
learning} OR {self-managed instruction} OR
The review was conducted according to the
{self-managed education} OR {self-managed
guidelines for an effective review [28]. The
teaching} OR {independent learning} OR
following section describes the steps that were
{independent instruction} OR {self-initiated
followed during the review process, which is
learning} OR {self-initiated education} OR {self-
shown in Figure 1.
initiated instruction})
OR
("student centered" OR "student centred" OR
"learner centered" OR "learner centred")).
The asterisk (*) was used in the search query
to ensure that all variations of a term are included.
For example, “gamification” as well as
“gamified” would fit the query.
2.1. Inclusion and exclusion criteria
The search was limited to English papers from
journals and conferences, as well as book
chapters. The inclusion criteria for papers were:
1. Either games or gamification had to be
used in conjunction with flipped learning or for
the purpose of making a course more student-
centred or students more self-directed (54
papers excluded).
2. The game or gamification had to be
described in enough detail to allow it to be
mapped, i.e., the main parts are described in
order to allow the motivational affordances to
Figure 1: The flowchart of the systematic review
be mapped by the reviewer (10 papers
process. excluded).
3. The study had to include empirical results
The literature search was conducted using the (descriptions of interventions which did not
Scopus database. It was chosen because it is report results from testing were therefore
known to index many publishers, including those excluded) (159 papers excluded).
most relevant to this field of inquiry (IEEE, ACM, 4. If the paper was concerned with SCL or
and Springer). The search query was constructed SDL, it had to engage with the concept by
to address the intersection of games and describing it in greater detail in the body of the
gamification with three main areas of learning paper instead of only mentioning it in the
considered to be more autonomous than abstract or keywords (33 papers excluded).
traditional educational experiences – flipped
classrooms, SCL, and SDL. The search was In addition, papers were excluded if:
conducted using the following search string in 1. They did not describe a formal education
April 2021: setting such as a kindergarten or school
(K12) or a tertiary education environment
35
(informal adult education and studies 3.1. Demographic details
involving students outside of a formal
course were excluded) (58 papers
The popularity of using games and
excluded).
gamification in flipped classrooms has been
2. They were concerned with situations
steadily increasing since 2016, as shown in Figure
where students created their own games to
2. In 2020, the last full year that was reviewed in
learn (10 papers excluded)
this study, the number of publications almost
3. They were studies about students’
opinions about teaching methods without
any actual changes to a course/classroom
(6 papers excluded).
After applying the above-mentioned criteria,
92 papers remained for analysis (58.7% about
flipped classroom; 21.7% about SCL; 14.1%
about SDL; 4.3% about flipped classroom and
SCL, and 1.1% about flipped classroom and
SDL). The full list of included papers is available
here: https://bit.ly/3lRelj0. This article focuses
only on those papers relating to flipped classroom doubled from the previous two years.
approaches. An analysis of the full sample will be Figure 2: The number of papers published per
the subject of future work. year (N = 59)
Based on the guidelines of [28], a concept-
centric matrix was used to map the details of each Most of the studies were conducted in Asia and
paper. To prevent the introduction of bias, details Europe, with these contributing 18 papers each
such as motivational affordances and study (30%), while studies in North America amounted
outcomes were mapped as described by the to 8 papers (13.3%) and Africa, the Middle east
authors without further analysis from the and South America were each represented by 1
reviewer. The review process was conducted by paper (1.7%). 13 studies (21.7%) did not specify
the first author of this paper. Any mapping where they were conducted.
decisions were discussed amongst the research Games and gamification have been employed
team. in a wide variety of fields, as shown in Table 1,
with computing being the most popular at a
tertiary level. This could be because teachers of
3. Results computing content are more likely to be
comfortable with adding digital elements to their
As mentioned before, this article will focus courses [30], and this is in line with other reviews
only on papers relating to flipped classroom on gamification in education [17]. At a K12 level
environments (59 papers; 64.1% of the sample). It (kindergarten to twelfth grade), social science and
should be noted that one paper presented two science are the main fields being gamified. Lastly,
studies which were mapped separately, hence the the use of gamification in flipped classrooms is
total number of studies included in the sample more common at tertiary level
presented in this article is 60. (university/college) than at school level.
The presentation of the results will begin with Table 2 shows the types of studies which have
an overview of the demographic data of the been employed – whether the game/gamification
studies. Following this are the details of the was combined with flipped classroom and then
motivational affordances used, the results of tested against a control of flipped classroom or a
experimental studies and the parts of the flipped traditional class, or whether the gameful
classroom that were modified by games and components were simply used as part of a flipped
gamification. The terminology relating to classroom, sometimes along with other
motivational affordances and study outcomes is educational approaches such as collaboration,
the same as that outlined by [29]. peer instruction, and blended learning. The “other
settings” study design includes several unique
study design types (described below the table).
36
Table 1 3.3. Experimental studies,
Mapping of studies according to field and level of
education.
affordances, and outcomes
Level of education 23 studies (38.3%) were classified as
Field K12 Tertiary Total experimental studies according to the following
Computing 1 18 19 criteria:
Economics 3 3 1. Clearly defined hypothesis or research
Engineering 1 1 2 question.
Medical 7 7 2. The use of a control group or pre-post-test
Physical education 1 1 design
Science 5 6 11 3. The use of inferential statistics.
Social science 6 5 11
Pre-service teaching 6 6 Figure 3 shows the mapping of these
experimental studies according to the
Total 14 46 60
motivational outcomes used and the outcome of
the study. Positive results imply that the applied
3.2. Motivational affordances and motivational affordances resulted in
study designs improvements to the specific measured outcomes.
Mixed results were mapped according to whether
the majority of tests had yielded positive or
Table 2 also shows the motivational negative results (negative meaning that the
affordances used (according to the mapping by applied affordances had no significant measurable
[29]). The most used achievement affordances effect on the outcomes). Most of the studies
were points (18 studies; 30%), badges (15 studies; reported positive results, i.e., the applied
25%), and leaderboards (13 studies; 21.6%). This motivational affordances resulted in
also reflected other reviews of the use of improvements to the specific measured outcomes,
gamification in education [17,30]. Social with achievement affordances and miscellaneous
affordances included teamwork (9 studies; 15%) elements being the most popular affordances
and competition (9 studies; 15%), while the most implemented. Most of the miscellaneous
common immersion affordances were the use of affordances used in experimental studies (8
narrative (8 studies; 13.3%) and role play (6 studies out of 11) were full games.
studies; 10%). The most common non-digital In terms of the specific types of outcomes that
elements were physical dice (6 studies; 10%) and were measured, across all 60 studies the highest
physical playboards and/or tokens (5 studies; measured outcomes were psychological states (25
8.3%). These were often employed together in the studies; 41.6%), such as motivation and attitude
form of boardgames. Finally, miscellaneous towards the course content; and performance
affordances included full commercial games or outcomes (38 studies, 63.3%), such as the level of
systems (such as Kahoot!, Socrative or word academic performance, quality of work and level
games). 27 studies (45%) used such games, with of participation in the course. Figure 4 shows the
14 (23%) of those using Kahoot! results of experimental studies organized
The class component item in Table 2 describes according to outcome type. The popularity of
which part of the flipped classroom was modified psychological and performance outcomes is also
to include the motivational affordances. It tended visible here, and educational performance
to be more common to modify the in-class outcomes showed primarily positive results.
component of the course. This could be because Furthermore, there are some cases where studies
motivational affordances could more easily be reported no positive changes where games and
included in these settings, whereas out-of-class gamification were added to a flipped classroom.
settings would require a digital system to keep
track of student activity and provide access to the
gameful components.
.
37
Figure 3: The results of experimental studies according to motivational affordances used (N = 23, but
total affordances exceed that because some studies used affordances from multiple categories)
Furthermore, the use of points, badges, and
4. Discussion leaderboards (PBLs) as the most common
affordances remains in line with other reviews on
gamification in education. As shown by [32], the
Overall, the results suggest that games and use of PBLs may not always be suitable for the
gamification can have a positive effect in flipped
context in which they are employed. For example,
classrooms, especially with regards to
leaderboards have contributed to a number of
performance outcomes such as academic
negative effects in gamification studies [32].
achievement.
PBLs fall under the category of achievement
The analysis revealed that more than one-third affordances [33]. The much lower incidence of
of the studies employed the design of combining
immersion- and social-related affordances in the
games/gamification with the flipped classroom
sample analysed indicates a gap in the field which
without a control group. While the use of a pre-
also echoes the state of gamification across other
/post-test in these contexts made it possible to
fields [29]. The flipped classroom model makes it
gauge whether the course was improved by the possible to utilise class time for more interactive,
addition of the gamification, it was impossible to social learning activities such as group work and
conclusively attribute the outcome to the
discussions [34]. In this sense, it may be simpler
gamification (compared to studies where a non-
to implement social affordances in this kind of
gamified control group was used). According to
classroom setting when compared to traditional
[31], the design of the instructional materials in a
classes. As such, flipped classrooms are well-
flipped classroom have a great influence on the placed to contribute to the field of gamification in
way in which students perceive the learning
this way [29]. In terms of the use of full games in
content. Therefore, it is recommended that future
flipped classrooms, Kahoot! was the primary
research involves more experimental designs with
choice for in-class engagement. The features of
control groups to make it possible to isolate the
this system – free to access and easy to use – make
effects of the gamification treatment. it the ideal companion to in-class activities.
38
Table 2
Motivational affordances used based on study design and the component of the flipped classroom that was modified by the affordance.
Study design Class component N Achievement Social affordances Immersion Non-digital Miscellaneous
modified affordances affordances elements elements
vs In-class 2 1 1
traditional 2 1 1
In-class 7 5 2 2 2 4
vs
Out-of-class 9 9 2 3 1
flipped
16 14 4 5 2 5
In- and out-of-class 2 2
Game /
In-class 11 5 3 2 3 9
gamif +
only Out-of-class 6 4 2 2
flipped
Not specified 3 1 1 1 1 3
22 10 4 5 4 16
In- and out-of-class 1 1 1 1
In-class 9 2 2 1 1 7
+ other*
Not specified 4 1 1 3
14 4 3 1 2 11
In-class 2 1 1
Out-of-class 3 2 1 2
Other designs**
Not specified 1 1 1
6 3 1 1 4
60 31 12 12 9 37
Note: total affordances equal more than 60 because some studies employed affordances from more than one category
* Other includes teaching approaches such as collaboration, teamwork, seminars, and project-based learning
** Other designs are: game/gamif + flip vs game/gamif + independant vs game/gamif + traditional;
game/gamif + flipped vs game/gamif;
game/gamif + flipped + other vs game/gamif + flipped + lecture vs traditional;
game/gamif + flipped + other vs flipped vs traditional;
game/gamif + flipped vs gamif other + flipped vs flipped
39
Figure 4: The results of experimental studies according to the types of outcomes measured (N = 23
but total outcomes exceed that because some studies measured outcomes from multiple categories).
A literature review by [35] reports that of the known negative effects of gamification
Kahoot! can have a positive effect on learning [32]. [40] found that the students only interacted
when compared to other approaches. It also with the gamified elements due to the competition
embraces the bring-your-own-device (BYOD) they created, not because they associated it with
model which is already popular in education, improved learning. It is because of cases like these
especially at a tertiary level [36]. The studies that that it is important that the motivational
employed Kahoot! in the classroom used it mainly affordances be intrinsically tied to the learning
as a type of formative assessment to consolidate content to prevent students from bypassing the
knowledge that students would have gained from learning content while engaging with the game
engaging with the out-of-class material [37,38]. elements [16,41].
In general, the in-class component received Finally, in terms of methodological
more attention in terms of the use of games and approaches, the sample analysed contained no
gamification. Since the flipped classroom model studies which attempted to isolate individual
only works if students prepare adequately for affordances to test their effectiveness. Within the
class, the use of motivational affordances to field of gamification, calls have been made for
encourage this behaviour is an interesting line of these kinds of studies in order to understand how
inquiry which requires additional investigation gamification works [29,42], and testing the effects
[20]. of individual affordances is one of the main ways
The studies analysed reported few negative to achieve this. In addition, the current reliance on
effects from the gamified interventions, although academic achievement as an outcome measure,
[39] reported that some students gamed the while being the simplest way to determine
system in order to earn more badges. This is one whether an intervention has been effective, misses
40
a deeper understanding of exactly how it has tested. There is much room for variety in these
resulted in improved performance. A focus on areas.
measures such as psychological outcomes These results contribute to the current
(motivation, attitude towards the content, self- understanding of the state of the field. Future
regulation, engagement, confidence etc.) work includes reporting on the full set of studies
combined with engagement outcomes, such as that were returned in the search query to gain a
time spent interacting with the motivational broader perspective on the use of games and
affordances, could shed light on the specific ways gamification for autonomous learning.
in which these affordances function to bring about
improved performance from students. 7. References
The studies included also showed very little
reliance on underlying theoretical work, such as
[1] EDUCAUSE, 2020 EDUCAUSE Horizon
self-determination theory or theories of
engagement. In order to support the understanding Report: Teaching and Learning Edition,
CO:EDUCAUSE, Boulder, 2020.
of how gamification works in different contexts,
https://library.educause.edu/-
it is important to base empirical work on firm
/media/files/library/2020/3/2020_horizon_r
theoretical assumptions [43].
eport_pdf.pdf (accessed December 8, 2021).
[2] S.L. Boyer, D.R. Edmondson, A.B. Artis, D.
5. Limitations Fleming, Self-Directed Learning: A Tool
for Lifelong Learning, Journal of Marketing
As with any study, this one is not without its Education. 36 (2014) 20–32.
limitations. Firstly, the search was limited to the https://doi.org/10.1177/0273475313494010
Scopus database. The number of articles retrieved [3] A. Kaplan, Lifelong learning: conclusions
for the final analysis (92 articles in the full from a literature review, International
sample) is fairly substantial and Scopus is Online Journal of Primary Education
considered to index the most relevant publishers (IOJPE) ISSN: 1300-915X. 5 (2016).
in this field. While this may serve to curtail the http://www.iojpe.org/index.php/iojpe/articl
effect of this limitation on the findings, it is e/view/91 (accessed December 8, 2021).
possible that some publications may have been [4] S. Wilcox, Fostering self-directed learning
missed, especially since snowball sampling was in the university setting, Studies in Higher
not conducted to find additional related studies. Education. 21 (1996) 165–176.
Secondly, the search string used may not have https://doi.org/10.1080/0307507961233138
been sufficient to retrieve all possible matches to 1338.
the query. To mitigate this limitation, the search [5] G. O’Neill, T. McMahon, Student-centred
string was revised and refined over several learning: what does it mean for students and
iterations to ensure that, as far as possible, all lecturers?, in: G. O’Neill, S. Moore, B.
conceivable variations of the search terms were McMullin (Eds.), Emerging Issues in the
addressed. Practice of University Learning and
Teaching, AISHE, Dublin, 2005: p. 10.
6. Concluding remarks [6] M. Weimer, Learner-Centered Teaching:
Five Key Changes to Practice, 2nd ed.,
Jossey-Bass, San Francisco, 2013.
This study has provided a systematic analysis
[7] J.L. Bishop, M.A. Verleger, The flipped
of studies concerning the use of games or
classroom: A survey of the research, in:
gamification in flipped classrooms within formal
ASEE National Cconference Proceedings,
education settings. This is an emerging field of
Atlanta, 2013: pp. 1–18.
inquiry and could benefit from high-level
[8] S. Yan, Teachers’ Roles in Autonomous
overviews of prior work.
Learning, Journal of Sociological Research.
The results show that educators are beginning
3 (2012) 557–562.
to embrace the idea of using games and
[9] S. Pokhrel, R. Chhetri, A Literature Review
gamification in flipped classrooms, with in-
on Impact of COVID-19 Pandemic on
classroom activities being the primary focus. The
Teaching and Learning, Higher Education
outcomes of these studies are mostly positive,
for the Future. 8 (2021) 133–141.
although mainly achievement affordances were
used, and performance-based outcomes were
41
https://doi.org/10.1177/2347631120983481 Sprague (Ed.), 2014 47th Hawaii
. International Conference on System
[10] V.A. Clifford, The Development of Sciences, IEEE, Waikoloa, 2014: pp. 3025–
Autonomous Learners in a University 3034.
Setting, Higher Education Research & https://doi.org/10.1109/HICSS.2014.377.
Development. 18 (1999) 115–128. [19] D. Dicheva, K. Irwin, C. Dichev,
https://doi.org/10.1080/0729436990180109 Motivational factors in educational
. gamification, in: 2018: pp. 408–410.
[11] A.K. Przybylski, C.S. Rigby, R.M. Ryan, A https://doi.org/10.1109/ICALT.2018.00102
motivational model of video game .
engagement., Review of General [20] B. Huang, K.F. Hew, C.K. Lo, Investigating
Psychology. 14 (2010) 154–166. the effects of gamification-enhanced flipped
[12] P. Buckley, E. Doyle, Gamification and learning on undergraduate students’
student motivation, Interactive Learning behavioral and cognitive engagement,
Environments. 0 (2014) 1–14. Interactive Learning Environments. 27
https://doi.org/10.1080/10494820.2014.964 (2019) 1106–1126.
263. https://doi.org/10.1080/10494820.2018.149
[13] J.-C. Woo, Digital Game-Based Learning 5653.
Supports Student Motivation, Cognitive [21] L. Abeysekera, P. Dawson, Motivation and
Success, and Performance Outcomes, cognitive load in the flipped classroom:
Journal of Educational Technology & definition, rationale and a call for research,
Society. 17 (2014) 291–307. Higher Education Research &
[14] S. Deterding, D. Dixon, R. Khaled, L. Development. 34 (2015) 1–14.
Nacke, From game design elements to [22] J.D. Harden, Learning Without Sages?
gamefulness: defining gamification, in: A. Reflections on “Flipping” the University
Lugmayr, H. Franssila, O. Sotamaa, C. Classroom, Alternate Routes: A Journal of
Safran, T. Aaltonen (Eds.), Proceedings of Critical Social Research. 26 (2015).
the 15th International Academic MindTrek [23] P. Freire, Pedagogy of the Oppressed,
Conference: Envisioning Future Media Bloomsbury Publishing USA, New Jersey,
Environments, ACM, Tampere, 2011: pp. 2014.
9–15. [24] E.A. Hopkins, John Dewey and Progressive
https://doi.org/10.1145/2181037.2181040. Education, The Journal of Educational
[15] J. Hamari, Gamification, in: G. Ritzer (Ed.), Thought. 50 (2017) 59–68.
The Blackwell Encyclopedia of Sociology, [25] D. Gross, E.S. Pietri, G. Anderson, K.
John Wiley & Sons, Ltd, Oxford, UK, 2019: Moyano-Camihort, M.J. Graham, Increased
pp. 1–3. preclass preparation underlies student
https://doi.org/10.1002/9781405165518.wb outcome improvement in the flipped
eos1321. classroom, CBE—Life Sciences Education.
[16] J.L. Plass, B.D. Homer, C.K. Kinzer, 14 (2015) 8.
Foundations of Game-Based Learning, [26] A. Segura-Robles, A. Fuentes-Cabrera,
Educational Psychologist. 50 (2015) 258– M.E. Parra-González, J. López-Belmonte,
283. Effects on personal factors through flipped
https://doi.org/10.1080/00461520.2015.112 learning and gamification as combined
2533. methodologies in secondary education,
[17] D. Dicheva, C. Dichev, G. Agre, G. Frontiers in Psychology. 11 (2020) 1103.
Angelova, Gamification in Education: A [27] M. Ekici, A systematic review of the use of
Systematic Mapping Study, (2015). gamification in flipped learning, Educ Inf
http://www.researchgate.net/profile/Darina Technol. 26 (2021) 3327–3346.
_Dicheva/publication/270273830_Gamific https://doi.org/10.1007/s10639-020-10394-
ation_in_Education_A_Systematic_Mappi y.
ng_Study/links/54c95c4b0cf2807dcc262a1 [28] J. Webster, R.T. Watson, Analyzing the Past
c.pdf (accessed August 17, 2015). to Prepare for the Future: Writing a
[18] J. Hamari, J. Koivisto, H. Sarsa, Does Literature Review, MIS Quarterly. 26
Gamification Work? – A Literature Review (2002) xiii–xxiii.
of Empirical Studies on Gamification, in: R.
42
[29] J. Koivisto, J. Hamari, The rise of Anatomy. 31 (2018) 997–1005.
motivational information systems: A review https://doi.org/10.1002/ca.23249.
of gamification research, International [39] B. Huang, K.F. Hew, Implementing a
Journal of Information Management. 45 theory-driven gamification model in higher
(2019) 191–210. education flipped courses: Effects on out-of-
https://doi.org/10.1016/j.ijinfomgt.2018.10. class activity completion and quality of
013. artifacts, Computers & Education. 125
[30] C. Dichev, D. Dicheva, Gamifying (2018) 254–272.
education: what is known, what is believed https://doi.org/10.1016/j.compedu.2018.06.
and what remains uncertain: a critical 018.
review, International Journal of Educational [40] A.Y. Gündüz, B. Akkoyunlu, Effectiveness
Technology in Higher Education. 14 (2017) of Gamification in Flipped Learning, SAGE
9. Open. 10 (2020) 2158244020979837.
[31] C.K. Lo, K.F. Hew, A critical review of [41] Y.B. Kafai, M.L. Franke, C.C. Ching, J.C.
flipped classroom challenges in K-12 Shih, Game design as an interactive learning
education: possible solutions and environment for fostering students’ and
recommendations for future research, teachers’ mathematical inquiry,
Research and Practice in Technology International Journal of Computers for
Enhanced Learning. 12 (2017) 4. Mathematical Learning. 3 (1998) 149–184.
https://doi.org/10.1186/s41039-016-0044- [42] L.E. Nacke, S. Deterding, The maturing of
2. gamification research, Computers in Human
[32] A.M. Toda, P.H.D. Valle, S. Isotani, The Behavior. 71 (2017) 450–454.
dark side of gamification: An overview of https://doi.org/10.1016/j.chb.2016.11.062.
negative effects of gamification in [43] A. Rapp, F. Hopfgartner, J. Hamari, C.
education, Communications in Computer Linehan, F. Cena, Strengthening
and Information Science. 832 (2018) 143– gamification studies: Current trends and
156. https://doi.org/10.1007/978-3-319- future opportunities of gamification
97934-2_9. research, International Journal of Human-
[33] N. Yee, Motivations for Play in Online Computer Studies. 127 (2019) 1–6.
Games, CyberPsychology & Behavior. 9 https://doi.org/10.1016/j.ijhcs.2018.11.007.
(2006) 54.
[34] G. Akçayır, M. Akçayır, The flipped
classroom: A review of its advantages and
challenges, Computers & Education. 126
(2018) 334–345.
https://doi.org/10.1016/j.compedu.2018.07.
021.
[35] A.I. Wang, R. Tahir, The effect of using
Kahoot! for learning – A literature review,
Computers & Education. 149 (2020)
103818.
https://doi.org/10.1016/j.compedu.2020.10
3818.
[36] L. Johnson, S. Adams Becker, V. Estrada,
A. Freeman, NMC Horizon Report: 2015
Higher Education Edition, The New Media
Consortium, Austin, Texas, 2015.
[37] U. Durrani, Gamified Flipped Classroom
Learning Approach: A Case Study of AJ
University, in: 2019 IEEE International
Conference on Engineering, Technology
and Education (TALE), IEEE, 2019: pp. 1–
5.
[38] E.T. Ang, J.M. Chan, V. Gopal, N. Li Shia,
Gamifying anatomy education, Clinical
43