=Paper=
{{Paper
|id=Vol-3669/paper5
|storemode=property
|title=The use of gamification and virtual reality in higher education: A literature review
|pdfUrl=https://ceur-ws.org/Vol-3669/paper5.pdf
|volume=Vol-3669
|authors=Kristina Nagel,Maria Rauschenberger
|dblpUrl=https://dblp.org/rec/conf/gamifin/NagelR24
}}
==The use of gamification and virtual reality in higher education: A literature review==
https://ceur-ws.org/Vol-3669/paper5.pdf
The use of gamification and virtual reality in higher
education: A literature review
Kristina Nagel1,2, Maria Rauschenberger3
1 University of applied sciences Emden/Leer, Constantiaplatz 4, 26723 Emden, Germany
2 University Sevilla, C. San Fernando 4, 41004 Seville, Spain
Abstract
This paper summarizes findings from a systematic literature review about how gamified virtual reality
(VR) is used in the context of higher education. We analyzed 12 unique studies to understand their
context, how they applied gamification, and the lessons learned. Our results reveal a research gap
especially outside STEM subjects. Additionally, we found the main number of studies do not follow
gamification standards, making it tough to compare and reproduce their results. Also, we noticed a shift
from the conventional “points-badges-leaderboard” approach that was popular in earlier research, to a
more diverse use of game elements in gamified VR applications for higher education. This paper
provides a clear overview of the current state of gamified VR in higher education, providing practical
insights for researchers and practitioners.
Keywords
Gamification, game elements, virtual reality, higher education, literature review, slr 1
education VR applications, and identify research gaps
1. Introduction in the implementation of gamification and VR in higher
education. By synthesizing current research, we aim to
Integrating gamification into education or utilizing present an understanding of the state-of-the-art
virtual reality (VR) for educational purposes are not practices and their implications in higher educational
recent innovations. The application of gamification settings.
principles in educational settings is an established
practice [1][2] and the use of VR for educational
settings has been prevalent since the introduction of 2. Related work
consumer VR products at the latest [3][4].
Gamification entails the application of game Several literature reviews surrounding the topics
elements and principles in non-game contexts to, e.g., gamification, e-learning, or VR were conducted in
motivate and engage learners [5]. Game elements are recent years: For example, [7] analyzed research on
defined as “elements that are characteristic to games — the topic of eLearning in a higher education context.
elements that are found in most (but not necessarily all) They found, that most current studies use a
games” [5]. In education, VR immerses users in quantitative research approach and that the focus of e-
simulated environments, offering unique and learning research has shifted over the years from
interactive learning experiences [3]. This approach “integration of e-learning into higher education is
yields great advantages for learning, including demanding issue” in 2011 to “development of
enhanced enjoyment and communication, as well as customized e-learning environments according to
improved collaboration [6]. learners’ needs” in 2019.
Nevertheless, the simultaneous utilization of both The maturity of Virtual Reality (VR) has made it a
concepts for learning in higher education holds reliable option for educational approaches that are
promise but appears to be less common. Conducting a now being increasingly utilized. Hence, [3] reviewed
literature review is valuable for pinpointing both 38 studies using VR in higher education. Their results
existing pitfalls and potential solutions in the criticize the focus on usability instead of learning
development of new VR applications that integrate outcome, emphasizing that VR is predominantly in an
gamification for educational purposes. experimental stage, and has not been effectively
In this paper, we provide an overview of current incorporated in the daily teaching routine. They did
research concerning the integration of gamification not analyze the details of gamification or game-based
and VR in higher educational contexts, investigate the learning in the studies but focused more on the applied
game elements which are utilized in gamified higher
8th International GamiFIN Conference 2024 (GamiFIN 2024), April 2-
5, 2024, Ruka, Finland
Kristina.nagel@ux-researchgroup.com (K. Nagel);
maria.rauschenberger@hs-emden-leer.de (M. Rauschenberger);
0000-0002-8625-4903 (K. Nagel); 0000-000-5722-576X (M.
Rauschenberger);
© 2024 Copyright for this paper by its authors. The use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
47
learning theories and research methodologies in applied the same search string for title, keywords and
general. abstract of all identified papers (𝑁𝑃0 = 4246) and
However, the use of gamification for educational ended up with 𝑁𝑃1 = 20 papers for the manual scanning
purposes is still a popular research topic. For example, process. We applied the following inclusion and
the literature review of [8] about gamification for exclusion criteria at the screening process:
learning purposes in general found 128 research In1 The paper describes a practical application of
papers regarding the topic. They analyzed, e.g., gamification and virtual reality in a higher
affordances, psychological, and behavioral outcomes education learning context
of empirical studies. 5 of 128 studies used some kind In2 In2 Paper containing a literature review that
of virtual world as a gamification element. However, it aligns with inclusion criteria 1 is retained for
is not specified, whether these virtual worlds are part the snowballing process.
of VR or just any kind of virtual environment. In Ex1 The paper does not specify the target group as
another review, [1] analyzed studies about higher education
gamification frameworks in different learning Ex2 The paper does not actually apply gamification
environments. They had a closer look on the study or serious games concepts
contexts and the game design elements that were used Ex3 The paper does not actually use VR
to enhance learning. [2] reviewed studies about Ex4 The paper does not describe the use of the
gamification more specifically in Massive Open Online gamification strategy in detail
Courses and found that gamification can be a possible Ex5 The paper is not about a practical project
solution to the problem of a high no-show and a low Ex6 The paper focuses on exergaming/sports or
retention rate. To sum up, to the best of our knowledge therapy/rehabilitation/behavior change
there is no review about the use of gamification and VR
in higher education contexts that provides an We identified 𝑁𝑃4 = 6 relevant papers for
overview of the current situation. snowballing. Snowballing in a SLR involves recursively
exploring references of identified papers to discover
additional relevant studies. We applied backward
3. Methodology snowballing (references used in the already identified
papers) and forward snowballing (papers, that
We followed an established process for systematic reference the already identified papers). This process
literature reviews [9] to answer the following research led us to 𝑛 = 7 additional papers. Finally, we considered
questions: 𝑁𝑃5 = 13 papers for the analysis.
RQ1 In which use cases are gamification and VR for
higher education applied?
RQ2 What kind of gamification is used in
combination with VR for higher education?
RQ3 What are the learnings and recommendations
from the use of gamification and VR for higher
education?
To answer these research questions, we analyzed
peer reviewed research papers that provide example
applications of a combination of gamification and
virtual reality in a higher education context. We
identified relevant papers by applying the following
search-string:
“Gamif*” AND (“Virtual Reality” OR “VR”
OR “Virtuality” OR “Mixed Reality”) AND
(“higher education” OR “university” OR
“college” OR “students”)
The search string has been adjusted to the search
engines of the databases IEEE Xplore, ACM,
SpringerLink and Science Direct. For IEEE Xplore, ACM
and Springer link, we applied the search as a full text Figure 1: The selection process to identify the
search, for Science Direct we used the option to restrict literature for the analysis
the search to title, keywords and abstract. We only
considered publications from 2016 to 2022, to ensure A recent study showed that gamification project
current relevance. Figure 1 shows the selection reports are not standardized [11]. For example, the
process of relevant papers from the search results. used game elements are often ambiguously defined
We used the SLR tool from Hinderks et al. [10] to among various studies. Which means, two papers may
manage the search results and to support the selection name a game element similarly but have different
process. The SLR Tool offers the functionality to apply meanings (e.g., one might use points as a form of
an extended search to all papers within a project. currency, while another uses them in a reward system)
Scientific databases have a variety of differing search or two papers may name a game element differently
and filter algorithms. With the extended search, we but mean the same (e.g., points or score for the
48
Table 1
All papers included for analysis
Paper Title Subject Use case goal Use case scope Sourc
ID area e
S1 Serious Game for Medical Imaging medical general teaching, increase Serious game [13]
in Fully Immersive Virtual Reality education student engagement,
increase student retention
S2 Immersive Virtual Reality Training chemical provide equal access to Virtual lab [14]
of Bioreactor Operations engineering expensive equipment, environment
practical learning
S3 Merging 360°-videos and Game- biomedical support training Full laboratory [15]
Based Virtual Environments for exploration and
Phlebotomy Training: Teachers and training
Students View
S4 VR for Education in Information and computer Increase Learnability Single task [16]
Tehnology: application for Bubble science (Gamification) application
Sort
S5 Virtual reality instructional modules computer Support Learning VR instructional [17]
for introductory programming science modules
courses
S6 Thinkercise: An educational VR computer Increase Motivation, Exergame/Serio [18]
game for Python programming science Increase Learning, do us game
physical exercise
S7 VR Medical Gamification for medical increase interactivity Mobile Android [19]
Training and Education education platform
application
S8 Formative evaluation of immersive computer understanding users’ VR exploration [20]
virtual reality expedition mini- science learning processes, application +
games to facilitate computational facilitating students’ mini games
thinking computational thinking
skills
S9 Educational Game-Theme Based computer teach linked list and binary Learning [21]
Instructional Module for Teaching science trees module
Introductory Programming
Game Theme Based Instructional computer teaching binary trees Non-immersive [22]
Module to Teach Binary Trees Data science VR learning
Structure module
S10 Usability and Learning Effectiveness computer teaching stacks and ques Learning [23]
of Game-Themed Instructional (GTI) science module
Module for Teaching Stacks and
Queues
S11 Investigating the effect of imikode computer understanding of the Serious game [24]
virtual reality game in enhancing science subject (OOP), improving
object oriented programming programming skills
concepts among university students
in Nigeria
S12 iProgVR: Design of a Virtual Reality computer incite intrinsic motivation Framework [25]
Environment to Improve science
Introductory Programming Learning
performance measurement). To address this issue, we
decided to use the gamification codebook [12] as an
4. Results
analysis tool to ensure a uniform analysis of the
applied game elements. The gamification codebook The 𝑁 = 13 papers describe 𝑛 = 12 individual studies
contains a list of game elements with a descriptive (see Table 1). One study includes two papers [21][22].
definition of each element and is supposed to help Although exclusion criteria Ex6 excludes papers with
gamification practitioners to choose the best game focus on exergaming from the study, we kept the
elements for their project [12]. By using it as an exergame of S6 because the proposed outcome of
analysis tool, we apply one wording and description to playing the game is learning and not fitness or doing
all papers. We had to add some game elements, that are sport. The studies were analyzed with focus on the
not included in the gamification codebook neither by research questions and the results are presented in the
name nor definition. following sections. We refer to each analyzed study by
the ID assigned in Table 1, column “Paper ID”.
49
4.1. Use case attributes gamification (S8, S9, S12). Hence, we needed to
identify most of the used game elements by analyzing
To answer RQ1 - In which use cases are descriptive texts and images of the applications. It is
gamification and VR for higher education applied?, possible, that the lists are incomplete, because there
we analyzed the studies application area, the goal of were not enough information to identify all game
the studies and the scope of the studies (see Table 1). elements used. For instance, if a text did not mention
Most of the studies (𝑛 = 9) are in the area of computer any sound effects, we were unable to account for them
sciences (see Table 1, column “Subject area”). The rest in our analysis, regardless of their potential presence
of the studies are also for STEM relevant courses: S1 in some application.
and S7 are set in medical education, S2 in chemical All in all, 𝑁 = 40 individual game elements were
engineering, and S3 in biomedical. The objectives of the used in den studies (see appendix 1). From these, 𝑛 =
studies exhibit a wider variety and heterogeneity (see 17 game elements were used in only one study. In
Table 1, column “Use case goal”). Although most of contrast to other research, the most used game
them (𝑛 = 10) aim to increase or support learning, elements are not points, badges and leaderboards
teaching, or understanding of the individual topics (S1, [11][2][8][11], but game environments and
S2, S3, S4, S5, S6, S8, S9, S10, and S11), increasing Visuals/Graphics (𝑛 = 11 each), and interactivity (𝑛 =
motivation, engagement and interactivity is also 9). Followed by learning and voluntary approach (𝑛 = 8
relevant for 𝑛 = 4 studies (S1, S6, S7, S12). S2 each) and the use of a tutorial (𝑛 = 7).
additionally aims to provide equal access to expensive Not all game elements that were explicitly
equipment with the use of VR, thereby supporting described are defined in the gamification codebook.
practical learning. In contrast, the scope of the studies However, since many game elements are not described
is very diverse (see Table 1, column “Use case scope”): at all, we want to emphasize these elements as
In S4 a single task application is developed and S9 and important enough for the researchers to be mentioned.
S10 are describing learning modules. S5 presents Overall, we identified 8 game elements that we could
virtual reality instructional modules, while S2, S3 and not match to a game element from the gamification
S8 provide some form of laboratory exploration codebook:
scenarios. Besides that, S1, S6 and S11 utilize full • Narrator/Guide
serious games, one of which is an exergame (S6). • Quiz
Lastly, S12 proposes a framework consisting of three • Interface/Character Control
individual modules. • Mini-Games
• Game Mechanics
• Virtual World
4.2. Gamification strategies • Virtual Instructor
• Non-Playable Character
To answer RQ2 - What kind of gamification is used Furthermore, one study emphasized the
in combination with VR for higher education? we personalization of the feedback game element, which
examined the use of gamification at two different was described in the gamification codebook, but
levels. Initially, we identified the overall gamification without the individualization.
strategy employed in each study. Meaning, if the study
followed a certain gamification design framework,
used gamification tools, or followed an approach that 4.3. Results of the studies
was not specifically designed for gamification and if so:
Did they specify how they choose their game To answer RQ3 - What are the learnings and
elements? The second level examines unique game recommendations from the use of gamification
elements, that are applied in the analyzed studies. and VR for higher education? we had a look at the
Our results show that no standards for reporting limitations and recommendations of the analyzed
gamification are used. 𝑛 = 4 studies gave no definition studies. Based on these, we can recommend best
of the used process (S2, S7, S8, S12). S4 named their practices that proved to be useful, as well as things to
strategy “play and learn” but gave no definition as to avoid, because they may have caused bigger or smaller
what this strategy includes. 𝑛 = 2 studies used problems.
previously developed games (S3, S11), one of which Several of the analyzed studies listed at least one
did not give details in the analyzed paper (S3). The limitation of their project. For example, S7 was not
details to the game elements had to be extracted from tested yet. Hence, the results of S7 are to be taken with
another paper [26]. The rest of the studies named caution. Other limitations of the studies lead to more
some kind of underlying process or concept (𝑛 = 3 used clear recommendations for future studies. Thereby,
some form of constructivism (S5, S9, S10), S6 used creating valuable learning for other researchers. S2
game-based-learning and S1 ASSURE instructional and S3 both acknowledge the difference in technical
design), but no study elaborated further on why they affinity and experience for potential users. While S2
specifically choose the game elements, they ended up only had test-users with prior VR experience, S3 sees a
using. challenge with the technical affinity of users in general.
The reporting of the used game elements is mostly Hence, we found that potential gamified higher
not very detailed: Only one of the analyzed studies education VR applications should be tested with tech-
gave a distinct list of the used game elements with inexperienced users to ensure the usability not only
definitions (S2). 𝑛 = 3 studies described the used game for a subgroup among the students. This would
elements in more detail in the text (S1, S5, S10) and 𝑛 become especially important for applications outside
= 3 studies at least described parts of the used the field of computer sciences. After all, one could
50
argue, that a certain technical experience could be a challenge in comparing the employed gamification
requirement for students in technical courses. S4 approaches. However, since we do not aim to evaluate
warns of health issues like motion sickness. They the single studies but are more interested in their
recommend using their app no longer than 10 minutes. individual approaches, we do not expect a relevant
The time-constraint could affect sustainable learning impact on our results.
and is not feasible for all application scenarios.
However, it serves as an important reminder that
accessibility and safety must be considered. 5.2. Implications from applied
Furthermore, S8 and S9 show a rather small number of gamification
participants (𝑛 = 6 and 𝑛 = 14). To generalize findings
of user studies and avoid bias, a larger number of test- Recent research has demonstrated that numerous
users is needed. Especially, since we should consider standards for gamification development and reporting
various skill-levels and characteristics of potential exist in form of various gamification frameworks or
users. processes [27]. Researchers are encouraged to utilize
The analyzed studies offer more insights by these standards to ensure comparability of studies and
recommending future steps for their own research or unambiguous understandings of results [11].
more generalized learnings: However, the findings from RQ2 indicate that these
In general, S2 sees their application only as a standards were not employed in our sample. This
supplement and not as a replacement for in-person absence was noted both in the development of
training. The potential of gamified VR applications for gamification strategies and in the reporting of game
higher education still offers research potential. elements.
Furthermore, S2 also recommends additional Many of our analyzed studies did not give a distinct
familiarization training for VR applications. S12 list or description of applied game elements. Hence, it
recommends including comparison studies when is not only possible, but likely, that our results about
evaluating VR applications with non-VR versions. the use of individual game elements is not
More detailed recommendations are provided for comprehensive. For example, only one study reports
specific aspects of gamified VR applications in higher the use of music and two report the use of sound
education. S4 and S10 both recommend the use of effects. However, although sound is of less importance
levels, S3, S8 and S10 underline the importance of in VR application than for non-VR ones, it is still an
high-quality graphics and animations. S3 also advise important and commonly used feature [28]. We
for a neutral guide who is not the lecturer, the use of presume the possibility, that more studies used sound
cooperative features and friendly competition, as well but may not felt the necessity to mention it. Maybe they
as tactile feedback and input mechanisms. All in all, found it too obvious, or they did not describe any game
they suggest utilization of interactive items. S10 and elements at all and the used game elements were
S11 recommend making use of time constraints, by identified by analyzing images, which can obviously
adding time limits or real time feedback. S11 also not convey sound. Similar challenges are possible for
suggests doing an AI integration in future research. the other game elements as well. The use of standards
for game element reporting and definition would have
increased the reliability and integrity of this and
5. Discussion similar analyses.
The most used game elements we identified (Game
In the following section, we discuss the results of our Environment and Graphics/Visuals) could be due to the
research in relation to our research questions and VR setting. When developing a VR application, simply
present implications for future research. being immersed in the VR setting often inherently
creates a game-like environment. Furthermore, the VR
is mostly built based on game-like graphics, except for,
5.1. Implications from use case e.g., 360°-video environments. Hence, these signature
attributes game elements are also somehow signature elements
of VR and not necessarily due to the gamification of the
The results of RQ2 show little variation in applied applications. Interactivity, which is the second most
research areas. All analyzed studies were done in identified game element, is also close at hand when
STEM or STEM-related subjects. This leaves developing a VR application, but VR also allows for the
interesting potentials for other application areas, such user to take a more spectating role. However, the
as visiting historic sites for archaeology or history advantages of practical learning are widely known, so
students, immersive scenarios for pedagogical or utilizing interactivity to support practical learning in
social studies, or interactive experiences for liberal VR comes naturally. Since the analyzed studies all
arts students. Another literature review that focused revolve around education applications, the high
on VR applications for higher education also identified percentage of applications that support the game
a high amount of STEM or STEM-related subjects (over element of learning is not surprising. Nevertheless, it is
60% of the analyzed studies) [3]w. However, they also noteworthy that a high number of applications build
identified a few studies in other application areas (5% upon the voluntary approach concept, because higher
nursing and art each). This is in line with our research education offers an easy opportunity to make the use
and underlines a research gap for future studies in of an application mandatory, which was mostly
more research areas. omitted. However, it is possible, that the voluntary
In contrast to the consistent application areas, the approach is due to the evaluation of new concepts and
diverse application scopes we identified could pose a more mandatory applications may be developed in the
51
future. The change over the next periods of time game elements in gamified VR applications for higher
remain to be seen. education. This compilation not only serves as
The next most used game element was the inspiration for future projects but also demonstrates
integration of some kind of tutorial. In higher that gamification is evolving beyond the criticized and
education contexts, it is important that everyone is limited pointsification approach.
able to use the learning materials. Hence, to have a
tutorial is important for the inclusion of all students
that are not used to VR applications, yet. Furthermore, Acknowledgements
in complex scenarios it can be necessary to prevent
frustration due to basic control issues. Thereby, the This research was conducted as part of the Future
students can focus on the learning content instead of Skills Applied project. The project is funded by the
struggling with the controls. We recommend the Foundation Innovation in University Teaching in the
integration of an (optional) tutorial for future gamified program line “Strengthening University Teaching
higher education VR applications to ensure the access through Digitization” in Germany. We thank Lars
and usability for all students. Pastoor for his participation in the literature review
All in all, our results show that there is no “how-to” process to ensure an objective selection of the
structure for gamified VR applications for higher analyzed papers.
education. This is in line with other research, that
shows heterogeneous gamification scenarios and
many variations of game element usages [1]. Thereby, References
we show a development in gamification research,
away from the criticized simplified and almost [1] M. Rauschenberger, A. Willems, M. Ternieden, J.
exclusive use of points, badges and leaderboards which Thomaschewksi, Towards the use of gamification
was sometimes dismissively called pointsification [29]. frameworks in learning environments, Jl. of
The increased variation of the used game elements Interactive Learning Research 2019 (2019) 147–
give more justice to the vast world of games and may 165.
develop even further in the future. [2] M. Jarnac de Freitas, M. Mira da Silva, Systematic
literature review about gamification in moocs,
Open Learning: The Journal of Open, Distance
5.3. Learnings and and eLearning 2020 (2020) 1–23.
recommendations doi:10.1080/02680513. 2020.1798221.
[3] J. Radianti, T. A. Majchrzak, J. Fromm, I.
For RQ3, we explored the limitations, learnings, and Wohlgenannt, A systematic review of immersive
recommendations of the analyzed papers authors. virtual reality applications for higher education:
Thereby, our results offer insights into difficult-to- Design elements, lessons learned, and research
measure qualitative experiences of each study. agenda, Computers & Education 147 (2020)
The limitations of the analyzed studies are, in some 103778. doi:10.1016/ j.compedu.2019.103778.
cases, quite severe, thereby raising concerns about the [4] D. Checa, A. Bustillo, A review of immersive
overall quality of the research. Standardized virtual reality serious games to enhance learning
approaches for development and reporting could and training, Multimedia Tools and Applications
create comparability and trust of the results, as well as 79 (2020) 5501–5527. URL:
avoiding errors from external factors [11]. https://link.springer.com/
All in all, the recommendations and learnings of the article/10.1007/s11042-019-08348-9.
analyzed studies show high potentials and various doi:10.1007/ s11042- 019- 08348- 9.
possibilities for future research. On one hand, there is [5] S. Deterding, D. Dixon, R. Khaled, L. E. Nacke,
still a need for generalized research like the From game design elements to gamefulness:
comparison between VR and non-VR or the potential Defining gamification, MindTrek 2011 (2011) 9–
of VR as a replacement for in-person training. On the 15.
other hand, there are some very detailed findings for [6] R. Timovski, N. Koceska, S. Koceski, The use of
single game elements and their applicability. augmented and virtual reality in education
(2020).
[7] M. I. Baig, L. Shuib, E. Yadegaridehkordi, E-
5.4. Conclusion & future work learning adoption in higher education: A review,
Information Development (2021)
In this study, we conducted a systematic literature 026666692110082. doi:10.
review on a total of 12 studies that implemented 1177/02666669211008224.
gamified VR applications for higher education. Our [8] J. Majuri, J. Koivisto, J. Hamari, Gamification of
findings provide an overview of current research, education and learning: A review of empirical
revealing numerous opportunities for future literature, Proceedings of the 2nd International
investigation. We successfully identified several GamiFIN Conference (GamiFIN 2018) 2 (2018)
research gaps, that merit further exploration: For 11–19. URL: http://ceur-ws.org/Vol-
instance, the application area of gamified VR in higher 2186/paper2.pdf.
education closely revolves around STEM and STEM- [9] B. Kitchenham, S. M. Charters, Guidelines for
related topics, although gamification and VR also offer performing systematic literature reviews in
numerous chances for every other subject area as well. software engineering: Ebse technical report,
Furthermore, we identified a list of the most used 2007.
52
[10] A. Hinderks, F. J. D. Mayo, J. Thomaschewski, M. J. expedition mini-games to facilitate
Escalona, An slr-tool: search process in practice, computational thinking, Computers & Education:
in: G. Rothermel, D.-H. Bae (Eds.), Proceedings of X Reality 2 (2023) 100016. URL:
the ACM/IEEE 42nd International Conference on https://www.sciencedirect.
Software Engineering: Companion Proceedings, com/science/article/pii/S2949678023000107.
ACM, New York, NY, USA, 2020, pp. 81–84. doi:10. 1016/j.cexr.2023.100016.
doi:10. 1145/3377812.3382137. [21] S. Rajeev, S. Sharma, Educational game-theme
[11] S. Hallifax, M. Altmeyer, K. Kölln, M. based instructional module for teaching
Rauschenberger, L. E. Nacke, From points to introductory programming, in: IECON 2018 -
progression: A scoping review of game elements 44th Annual Conference of the IEEE Industrial
in gamification research with a content analysis Electronics Society, IEEE, 2018.
of 280 research papers, Proceedings of the ACM doi:10.1109/iecon.2018. 8592835.
on Human-Computer Interaction 7 (2023). [22] S. Rajeev, S. Sharma, A. Sahu, Game theme based
[12] A. Thomas, F. Bader, J. Thomaschewski, M. instructional module to teach binary trees data
Rauschenberger, Gamification codebook, 2021. structure, Proceedings of ISCA 26th International
doi:10.13140/RG.2.2.22625.02403. Conference on Software Engineering and Data
[13] H. Cecotti, M. Callaghan, B. Foucher, S. Joslain, Engineering 2017 (2017) 13–18. URL:
Serious game for medical imaging in fully https://sharadonly. github.io/papers/19.pdf.
immersive virtual reality, in: 2021 IEEE [23] J. Stigall, S. Sharma, Usability and learning
International Conference on Engineering, effectiveness of game-themed instructional (gti)
Technology & Education (TALE), IEEE, 2021, pp. module for teaching stacks and queues,
615–621. doi:10.1109/ SoutheastCon 2018 (2018) 1–6.
TALE52509.2021.9678721. [24] K. Sunday, S. Y. Wong, B. O. Samson, I. T. Sanusi,
[14] Q. Chen, S.-E. Low, J. W. Yap, A. K. Sim, Y.-Y. Tan, Investigating the effect of imikode virtual reality
B. W. Kwok, J. S. Lee, C.-T. Tan, W.-P. Loh, B. L. Loo, game in enhancing object oriented programming
A. C. Wong, Immersive virtual reality training of concepts among university students in nigeria,
bioreactor operations, in: 2020 IEEE Education and Information Technologies 27
International Conference on Teaching, (2022) 6819–6845. URL:
Assessment, and Learning for Engineering https://link.springer.com/
(TALE), IEEE, 2020, pp. 873–878. article/10.1007/s10639-022-10886-z.
doi:10.1109/TALE48869.2020.9368468. doi:10.1007/ s10639- 022- 10886- z.
[15] T. H. Froland, I. Heldal, E. Ersvar, G. Sjoholt, [25] C. Wee, K. M. Yap, W. N. Lim, iprogvr: Design of a
Merging 360°-videos and game-based virtual virtual reality environment to improve
environments for phlebotomy training: Teachers introductory programming learning, IEEE Access
and students view, in: 2021 International 10 (2022) 100054–100078.
Conference on e-Health and Bioengineering doi:10.1109/ACCESS.2022. 3204392.
(EHB), IEEE, 2021, pp. 1–6. [26] T. H. Frøland, I. Heldal, G. Sjøholt, E. Ersvær,
doi:10.1109/EHB52898.2021.9657611. Games on mobiles via web or virtual reality
[16] S. Nicola, L. Stoicu-Tivadar, A. Patrascoiu, VR for technologies: How to support learning for
Education in Information and Tehnology: biomedical laboratory science education,
application for Bubble Sort, IEEE, Piscataway, NJ, Information 11 (2020) 195. URL:
2018. URL: https://www.mdpi.com/2078-2489/11/4/195.
http://ieeexplore.ieee.org/servlet/opac? doi:10. 3390/info11040195.
punumber=8556141. [27] K. Kölln, Maybe we don’t need a new gamification
[17] J. Stigall, S. Sharma, Virtual reality instructional framework after all, in: K. Gerling, J. Iacovides, R.
modules for introductory programming courses, Malaka, B. Bonsignore, J. Frommel (Eds.),
in: 2017 IEEE Integrated STEM Education Extended Abstracts of the Annual Symposium on
Conference (ISEC), IEEE, 2017, pp. 34–42. Computer-Human Interaction in Play, ACM, New
doi:10.1109/ISECon. 2017.7910245. York, NY, USA, 2022, pp. 384–387. doi:10.1145/
[18] T. Theethum, A. Arpornrat, S. Vittayakorn, 3505270.3558368.
Thinkercise: An educational vr game for python [28] K. Rogers, G. Ribeiro, R. R. Wehbe, M. Weber, L. E.
programming, in: 2021 18th International Nacke, Vanishing importance: Studying
Conference on Electrical immersive effects of game audio perception on
Engineering/Electronics, Computer, player experiences in virtual reality, in:
Telecommunications and Information Proceedings of the 2018 CHI Conference on
Technology (ECTI-CON), IEEE, 2021, pp. 439– Human Factors in Computing Systems, ACM, New
442. doi:10.1109/ECTI- York, NY, USA, 2018.
CON51831.2021.9454730. doi:10.1145/3173574.3173902.
[19] S. Nicola, I. Virag, Vr medical gamification for [29] S. Nicholson, A user-centered theoretical
training and education, in: D. Hayn, G. Schreier framework for meaningful gamification,
(Eds.), Health informatics meets eHealth, Studies Games+Learning+Society 8.0 (2012). URL:
in health technology and informatics, IOS Press, https://api.semanticscholar.org/CorpusID:9384
Amsterdam and Washington DC, 2017, pp. 97– 012.
103.
[20] A. S. Oyelere, F. J. Agbo, S. S. Oyelere, Formative
evaluation of immersive virtual reality
53
A. Appendix 1 – Game elements
distribution
Table 2
The identified game elements in the analyzed studies
Game Element 𝑁 Included in Study
Game Environments 11 S1, S2, S4, S5, S6, S7, S8, S9, S10, S11, S12
Visuals/Graphics 11 S1, S2, S3, S4, S5, S6, S8, S9, S10, S11, S12
Interactivity 9 S1, S2, S4, S5, S6, S7, S8, S9, S11
Learning 8 S2, S4, S5, S6, S7, S8, S9, S11
Voluntary Approach 8 S1, S2, S4, S5, S6, S8, S9, S11
Tutorial 7 S1, S4, S5, S6, S8, S9, S10
(Personalized) Feedback 5 S1, S2, S5, S8, S11
Points 5 S3, S5, S6, S8, S10
Visual Cues 5 S1, S2, S7, S10, S12
Challenge 3 S2, S6, S8
Free Exploration 3 S2, S8, S9
Goals 3 S4, S5, S8
Level 3 S2, S6, S9
Rewards 3 S8, S9, S12
Score 3 S1, S4, S6
Time Challenge 3 S1, S4, S6
Achievement 2 S1, S2
Any Progress 2 S6, S8
Disincentives/Negative Rewards 2 S3, S8
Rules 2 S6, S8
Sound Effects 2 S5, S12
Theme 2 S5, S11
Virtual Instructor 2 S9, S10
Attractive/Aesthetic design 1 S6
Avatar 1 S5
Competition 1 S1
Feedback System 1 S6
Framing 1 S8
Game Mechanics 1 S8
Leaderboard 1 S1
Mini Games 1 S8
Music Effects 1 S6
Narrative 1 S12
Narrator/Guide 1 S1
Non-Playable Character 1 S11
Onboarding 1 S2
Quiz 1 S1
Scoreboard 1 S9
Virtual World 1 S8
Interface/Character Control 1 S7
54