The use of gamification and virtual reality in higher education: A literature review Kristina Nagel1,2, Maria Rauschenberger3 1 University of applied sciences Emden/Leer, Constantiaplatz 4, 26723 Emden, Germany 2 University Sevilla, C. San Fernando 4, 41004 Seville, Spain Abstract This paper summarizes findings from a systematic literature review about how gamified virtual reality (VR) is used in the context of higher education. We analyzed 12 unique studies to understand their context, how they applied gamification, and the lessons learned. Our results reveal a research gap especially outside STEM subjects. Additionally, we found the main number of studies do not follow gamification standards, making it tough to compare and reproduce their results. Also, we noticed a shift from the conventional “points-badges-leaderboard” approach that was popular in earlier research, to a more diverse use of game elements in gamified VR applications for higher education. This paper provides a clear overview of the current state of gamified VR in higher education, providing practical insights for researchers and practitioners. Keywords Gamification, game elements, virtual reality, higher education, literature review, slr 1 education VR applications, and identify research gaps 1. Introduction in the implementation of gamification and VR in higher education. By synthesizing current research, we aim to Integrating gamification into education or utilizing present an understanding of the state-of-the-art virtual reality (VR) for educational purposes are not practices and their implications in higher educational recent innovations. The application of gamification settings. principles in educational settings is an established practice [1][2] and the use of VR for educational settings has been prevalent since the introduction of 2. Related work consumer VR products at the latest [3][4]. Gamification entails the application of game Several literature reviews surrounding the topics elements and principles in non-game contexts to, e.g., gamification, e-learning, or VR were conducted in motivate and engage learners [5]. Game elements are recent years: For example, [7] analyzed research on defined as “elements that are characteristic to games — the topic of eLearning in a higher education context. elements that are found in most (but not necessarily all) They found, that most current studies use a games” [5]. In education, VR immerses users in quantitative research approach and that the focus of e- simulated environments, offering unique and learning research has shifted over the years from interactive learning experiences [3]. This approach “integration of e-learning into higher education is yields great advantages for learning, including demanding issue” in 2011 to “development of enhanced enjoyment and communication, as well as customized e-learning environments according to improved collaboration [6]. learners’ needs” in 2019. Nevertheless, the simultaneous utilization of both The maturity of Virtual Reality (VR) has made it a concepts for learning in higher education holds reliable option for educational approaches that are promise but appears to be less common. Conducting a now being increasingly utilized. Hence, [3] reviewed literature review is valuable for pinpointing both 38 studies using VR in higher education. Their results existing pitfalls and potential solutions in the criticize the focus on usability instead of learning development of new VR applications that integrate outcome, emphasizing that VR is predominantly in an gamification for educational purposes. experimental stage, and has not been effectively In this paper, we provide an overview of current incorporated in the daily teaching routine. They did research concerning the integration of gamification not analyze the details of gamification or game-based and VR in higher educational contexts, investigate the learning in the studies but focused more on the applied game elements which are utilized in gamified higher 8th International GamiFIN Conference 2024 (GamiFIN 2024), April 2- 5, 2024, Ruka, Finland Kristina.nagel@ux-researchgroup.com (K. Nagel); maria.rauschenberger@hs-emden-leer.de (M. Rauschenberger); 0000-0002-8625-4903 (K. Nagel); 0000-000-5722-576X (M. Rauschenberger); © 2024 Copyright for this paper by its authors. The use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 47 learning theories and research methodologies in applied the same search string for title, keywords and general. abstract of all identified papers (𝑁𝑃0 = 4246) and However, the use of gamification for educational ended up with 𝑁𝑃1 = 20 papers for the manual scanning purposes is still a popular research topic. For example, process. We applied the following inclusion and the literature review of [8] about gamification for exclusion criteria at the screening process: learning purposes in general found 128 research In1 The paper describes a practical application of papers regarding the topic. They analyzed, e.g., gamification and virtual reality in a higher affordances, psychological, and behavioral outcomes education learning context of empirical studies. 5 of 128 studies used some kind In2 In2 Paper containing a literature review that of virtual world as a gamification element. However, it aligns with inclusion criteria 1 is retained for is not specified, whether these virtual worlds are part the snowballing process. of VR or just any kind of virtual environment. In Ex1 The paper does not specify the target group as another review, [1] analyzed studies about higher education gamification frameworks in different learning Ex2 The paper does not actually apply gamification environments. They had a closer look on the study or serious games concepts contexts and the game design elements that were used Ex3 The paper does not actually use VR to enhance learning. [2] reviewed studies about Ex4 The paper does not describe the use of the gamification more specifically in Massive Open Online gamification strategy in detail Courses and found that gamification can be a possible Ex5 The paper is not about a practical project solution to the problem of a high no-show and a low Ex6 The paper focuses on exergaming/sports or retention rate. To sum up, to the best of our knowledge therapy/rehabilitation/behavior change there is no review about the use of gamification and VR in higher education contexts that provides an We identified 𝑁𝑃4 = 6 relevant papers for overview of the current situation. snowballing. Snowballing in a SLR involves recursively exploring references of identified papers to discover additional relevant studies. We applied backward 3. Methodology snowballing (references used in the already identified papers) and forward snowballing (papers, that We followed an established process for systematic reference the already identified papers). This process literature reviews [9] to answer the following research led us to 𝑛 = 7 additional papers. Finally, we considered questions: 𝑁𝑃5 = 13 papers for the analysis. RQ1 In which use cases are gamification and VR for higher education applied? RQ2 What kind of gamification is used in combination with VR for higher education? RQ3 What are the learnings and recommendations from the use of gamification and VR for higher education? To answer these research questions, we analyzed peer reviewed research papers that provide example applications of a combination of gamification and virtual reality in a higher education context. We identified relevant papers by applying the following search-string: “Gamif*” AND (“Virtual Reality” OR “VR” OR “Virtuality” OR “Mixed Reality”) AND (“higher education” OR “university” OR “college” OR “students”) The search string has been adjusted to the search engines of the databases IEEE Xplore, ACM, SpringerLink and Science Direct. For IEEE Xplore, ACM and Springer link, we applied the search as a full text Figure 1: The selection process to identify the search, for Science Direct we used the option to restrict literature for the analysis the search to title, keywords and abstract. We only considered publications from 2016 to 2022, to ensure A recent study showed that gamification project current relevance. Figure 1 shows the selection reports are not standardized [11]. For example, the process of relevant papers from the search results. used game elements are often ambiguously defined We used the SLR tool from Hinderks et al. [10] to among various studies. Which means, two papers may manage the search results and to support the selection name a game element similarly but have different process. The SLR Tool offers the functionality to apply meanings (e.g., one might use points as a form of an extended search to all papers within a project. currency, while another uses them in a reward system) Scientific databases have a variety of differing search or two papers may name a game element differently and filter algorithms. With the extended search, we but mean the same (e.g., points or score for the 48 Table 1 All papers included for analysis Paper Title Subject Use case goal Use case scope Sourc ID area e S1 Serious Game for Medical Imaging medical general teaching, increase Serious game [13] in Fully Immersive Virtual Reality education student engagement, increase student retention S2 Immersive Virtual Reality Training chemical provide equal access to Virtual lab [14] of Bioreactor Operations engineering expensive equipment, environment practical learning S3 Merging 360°-videos and Game- biomedical support training Full laboratory [15] Based Virtual Environments for exploration and Phlebotomy Training: Teachers and training Students View S4 VR for Education in Information and computer Increase Learnability Single task [16] Tehnology: application for Bubble science (Gamification) application Sort S5 Virtual reality instructional modules computer Support Learning VR instructional [17] for introductory programming science modules courses S6 Thinkercise: An educational VR computer Increase Motivation, Exergame/Serio [18] game for Python programming science Increase Learning, do us game physical exercise S7 VR Medical Gamification for medical increase interactivity Mobile Android [19] Training and Education education platform application S8 Formative evaluation of immersive computer understanding users’ VR exploration [20] virtual reality expedition mini- science learning processes, application + games to facilitate computational facilitating students’ mini games thinking computational thinking skills S9 Educational Game-Theme Based computer teach linked list and binary Learning [21] Instructional Module for Teaching science trees module Introductory Programming Game Theme Based Instructional computer teaching binary trees Non-immersive [22] Module to Teach Binary Trees Data science VR learning Structure module S10 Usability and Learning Effectiveness computer teaching stacks and ques Learning [23] of Game-Themed Instructional (GTI) science module Module for Teaching Stacks and Queues S11 Investigating the effect of imikode computer understanding of the Serious game [24] virtual reality game in enhancing science subject (OOP), improving object oriented programming programming skills concepts among university students in Nigeria S12 iProgVR: Design of a Virtual Reality computer incite intrinsic motivation Framework [25] Environment to Improve science Introductory Programming Learning performance measurement). To address this issue, we decided to use the gamification codebook [12] as an 4. Results analysis tool to ensure a uniform analysis of the applied game elements. The gamification codebook The 𝑁 = 13 papers describe 𝑛 = 12 individual studies contains a list of game elements with a descriptive (see Table 1). One study includes two papers [21][22]. definition of each element and is supposed to help Although exclusion criteria Ex6 excludes papers with gamification practitioners to choose the best game focus on exergaming from the study, we kept the elements for their project [12]. By using it as an exergame of S6 because the proposed outcome of analysis tool, we apply one wording and description to playing the game is learning and not fitness or doing all papers. We had to add some game elements, that are sport. The studies were analyzed with focus on the not included in the gamification codebook neither by research questions and the results are presented in the name nor definition. following sections. We refer to each analyzed study by the ID assigned in Table 1, column “Paper ID”. 49 4.1. Use case attributes gamification (S8, S9, S12). Hence, we needed to identify most of the used game elements by analyzing To answer RQ1 - In which use cases are descriptive texts and images of the applications. It is gamification and VR for higher education applied?, possible, that the lists are incomplete, because there we analyzed the studies application area, the goal of were not enough information to identify all game the studies and the scope of the studies (see Table 1). elements used. For instance, if a text did not mention Most of the studies (𝑛 = 9) are in the area of computer any sound effects, we were unable to account for them sciences (see Table 1, column “Subject area”). The rest in our analysis, regardless of their potential presence of the studies are also for STEM relevant courses: S1 in some application. and S7 are set in medical education, S2 in chemical All in all, 𝑁 = 40 individual game elements were engineering, and S3 in biomedical. The objectives of the used in den studies (see appendix 1). From these, 𝑛 = studies exhibit a wider variety and heterogeneity (see 17 game elements were used in only one study. In Table 1, column “Use case goal”). Although most of contrast to other research, the most used game them (𝑛 = 10) aim to increase or support learning, elements are not points, badges and leaderboards teaching, or understanding of the individual topics (S1, [11][2][8][11], but game environments and S2, S3, S4, S5, S6, S8, S9, S10, and S11), increasing Visuals/Graphics (𝑛 = 11 each), and interactivity (𝑛 = motivation, engagement and interactivity is also 9). Followed by learning and voluntary approach (𝑛 = 8 relevant for 𝑛 = 4 studies (S1, S6, S7, S12). S2 each) and the use of a tutorial (𝑛 = 7). additionally aims to provide equal access to expensive Not all game elements that were explicitly equipment with the use of VR, thereby supporting described are defined in the gamification codebook. practical learning. In contrast, the scope of the studies However, since many game elements are not described is very diverse (see Table 1, column “Use case scope”): at all, we want to emphasize these elements as In S4 a single task application is developed and S9 and important enough for the researchers to be mentioned. S10 are describing learning modules. S5 presents Overall, we identified 8 game elements that we could virtual reality instructional modules, while S2, S3 and not match to a game element from the gamification S8 provide some form of laboratory exploration codebook: scenarios. Besides that, S1, S6 and S11 utilize full • Narrator/Guide serious games, one of which is an exergame (S6). • Quiz Lastly, S12 proposes a framework consisting of three • Interface/Character Control individual modules. • Mini-Games • Game Mechanics • Virtual World 4.2. Gamification strategies • Virtual Instructor • Non-Playable Character To answer RQ2 - What kind of gamification is used Furthermore, one study emphasized the in combination with VR for higher education? we personalization of the feedback game element, which examined the use of gamification at two different was described in the gamification codebook, but levels. Initially, we identified the overall gamification without the individualization. strategy employed in each study. Meaning, if the study followed a certain gamification design framework, used gamification tools, or followed an approach that 4.3. Results of the studies was not specifically designed for gamification and if so: Did they specify how they choose their game To answer RQ3 - What are the learnings and elements? The second level examines unique game recommendations from the use of gamification elements, that are applied in the analyzed studies. and VR for higher education? we had a look at the Our results show that no standards for reporting limitations and recommendations of the analyzed gamification are used. 𝑛 = 4 studies gave no definition studies. Based on these, we can recommend best of the used process (S2, S7, S8, S12). S4 named their practices that proved to be useful, as well as things to strategy “play and learn” but gave no definition as to avoid, because they may have caused bigger or smaller what this strategy includes. 𝑛 = 2 studies used problems. previously developed games (S3, S11), one of which Several of the analyzed studies listed at least one did not give details in the analyzed paper (S3). The limitation of their project. For example, S7 was not details to the game elements had to be extracted from tested yet. Hence, the results of S7 are to be taken with another paper [26]. The rest of the studies named caution. Other limitations of the studies lead to more some kind of underlying process or concept (𝑛 = 3 used clear recommendations for future studies. Thereby, some form of constructivism (S5, S9, S10), S6 used creating valuable learning for other researchers. S2 game-based-learning and S1 ASSURE instructional and S3 both acknowledge the difference in technical design), but no study elaborated further on why they affinity and experience for potential users. While S2 specifically choose the game elements, they ended up only had test-users with prior VR experience, S3 sees a using. challenge with the technical affinity of users in general. The reporting of the used game elements is mostly Hence, we found that potential gamified higher not very detailed: Only one of the analyzed studies education VR applications should be tested with tech- gave a distinct list of the used game elements with inexperienced users to ensure the usability not only definitions (S2). 𝑛 = 3 studies described the used game for a subgroup among the students. This would elements in more detail in the text (S1, S5, S10) and 𝑛 become especially important for applications outside = 3 studies at least described parts of the used the field of computer sciences. After all, one could 50 argue, that a certain technical experience could be a challenge in comparing the employed gamification requirement for students in technical courses. S4 approaches. However, since we do not aim to evaluate warns of health issues like motion sickness. They the single studies but are more interested in their recommend using their app no longer than 10 minutes. individual approaches, we do not expect a relevant The time-constraint could affect sustainable learning impact on our results. and is not feasible for all application scenarios. However, it serves as an important reminder that accessibility and safety must be considered. 5.2. Implications from applied Furthermore, S8 and S9 show a rather small number of gamification participants (𝑛 = 6 and 𝑛 = 14). To generalize findings of user studies and avoid bias, a larger number of test- Recent research has demonstrated that numerous users is needed. Especially, since we should consider standards for gamification development and reporting various skill-levels and characteristics of potential exist in form of various gamification frameworks or users. processes [27]. Researchers are encouraged to utilize The analyzed studies offer more insights by these standards to ensure comparability of studies and recommending future steps for their own research or unambiguous understandings of results [11]. more generalized learnings: However, the findings from RQ2 indicate that these In general, S2 sees their application only as a standards were not employed in our sample. This supplement and not as a replacement for in-person absence was noted both in the development of training. The potential of gamified VR applications for gamification strategies and in the reporting of game higher education still offers research potential. elements. Furthermore, S2 also recommends additional Many of our analyzed studies did not give a distinct familiarization training for VR applications. S12 list or description of applied game elements. Hence, it recommends including comparison studies when is not only possible, but likely, that our results about evaluating VR applications with non-VR versions. the use of individual game elements is not More detailed recommendations are provided for comprehensive. For example, only one study reports specific aspects of gamified VR applications in higher the use of music and two report the use of sound education. S4 and S10 both recommend the use of effects. However, although sound is of less importance levels, S3, S8 and S10 underline the importance of in VR application than for non-VR ones, it is still an high-quality graphics and animations. S3 also advise important and commonly used feature [28]. We for a neutral guide who is not the lecturer, the use of presume the possibility, that more studies used sound cooperative features and friendly competition, as well but may not felt the necessity to mention it. Maybe they as tactile feedback and input mechanisms. All in all, found it too obvious, or they did not describe any game they suggest utilization of interactive items. S10 and elements at all and the used game elements were S11 recommend making use of time constraints, by identified by analyzing images, which can obviously adding time limits or real time feedback. S11 also not convey sound. Similar challenges are possible for suggests doing an AI integration in future research. the other game elements as well. The use of standards for game element reporting and definition would have increased the reliability and integrity of this and 5. Discussion similar analyses. The most used game elements we identified (Game In the following section, we discuss the results of our Environment and Graphics/Visuals) could be due to the research in relation to our research questions and VR setting. When developing a VR application, simply present implications for future research. being immersed in the VR setting often inherently creates a game-like environment. Furthermore, the VR is mostly built based on game-like graphics, except for, 5.1. Implications from use case e.g., 360°-video environments. Hence, these signature attributes game elements are also somehow signature elements of VR and not necessarily due to the gamification of the The results of RQ2 show little variation in applied applications. Interactivity, which is the second most research areas. All analyzed studies were done in identified game element, is also close at hand when STEM or STEM-related subjects. This leaves developing a VR application, but VR also allows for the interesting potentials for other application areas, such user to take a more spectating role. However, the as visiting historic sites for archaeology or history advantages of practical learning are widely known, so students, immersive scenarios for pedagogical or utilizing interactivity to support practical learning in social studies, or interactive experiences for liberal VR comes naturally. Since the analyzed studies all arts students. Another literature review that focused revolve around education applications, the high on VR applications for higher education also identified percentage of applications that support the game a high amount of STEM or STEM-related subjects (over element of learning is not surprising. Nevertheless, it is 60% of the analyzed studies) [3]w. However, they also noteworthy that a high number of applications build identified a few studies in other application areas (5% upon the voluntary approach concept, because higher nursing and art each). This is in line with our research education offers an easy opportunity to make the use and underlines a research gap for future studies in of an application mandatory, which was mostly more research areas. omitted. However, it is possible, that the voluntary In contrast to the consistent application areas, the approach is due to the evaluation of new concepts and diverse application scopes we identified could pose a more mandatory applications may be developed in the 51 future. The change over the next periods of time game elements in gamified VR applications for higher remain to be seen. education. This compilation not only serves as The next most used game element was the inspiration for future projects but also demonstrates integration of some kind of tutorial. In higher that gamification is evolving beyond the criticized and education contexts, it is important that everyone is limited pointsification approach. able to use the learning materials. Hence, to have a tutorial is important for the inclusion of all students that are not used to VR applications, yet. Furthermore, Acknowledgements in complex scenarios it can be necessary to prevent frustration due to basic control issues. Thereby, the This research was conducted as part of the Future students can focus on the learning content instead of Skills Applied project. The project is funded by the struggling with the controls. We recommend the Foundation Innovation in University Teaching in the integration of an (optional) tutorial for future gamified program line “Strengthening University Teaching higher education VR applications to ensure the access through Digitization” in Germany. We thank Lars and usability for all students. Pastoor for his participation in the literature review All in all, our results show that there is no “how-to” process to ensure an objective selection of the structure for gamified VR applications for higher analyzed papers. education. This is in line with other research, that shows heterogeneous gamification scenarios and many variations of game element usages [1]. Thereby, References we show a development in gamification research, away from the criticized simplified and almost [1] M. Rauschenberger, A. Willems, M. Ternieden, J. exclusive use of points, badges and leaderboards which Thomaschewksi, Towards the use of gamification was sometimes dismissively called pointsification [29]. frameworks in learning environments, Jl. of The increased variation of the used game elements Interactive Learning Research 2019 (2019) 147– give more justice to the vast world of games and may 165. develop even further in the future. [2] M. Jarnac de Freitas, M. Mira da Silva, Systematic literature review about gamification in moocs, Open Learning: The Journal of Open, Distance 5.3. Learnings and and eLearning 2020 (2020) 1–23. recommendations doi:10.1080/02680513. 2020.1798221. [3] J. Radianti, T. A. Majchrzak, J. Fromm, I. For RQ3, we explored the limitations, learnings, and Wohlgenannt, A systematic review of immersive recommendations of the analyzed papers authors. virtual reality applications for higher education: Thereby, our results offer insights into difficult-to- Design elements, lessons learned, and research measure qualitative experiences of each study. agenda, Computers & Education 147 (2020) The limitations of the analyzed studies are, in some 103778. doi:10.1016/ j.compedu.2019.103778. cases, quite severe, thereby raising concerns about the [4] D. Checa, A. Bustillo, A review of immersive overall quality of the research. Standardized virtual reality serious games to enhance learning approaches for development and reporting could and training, Multimedia Tools and Applications create comparability and trust of the results, as well as 79 (2020) 5501–5527. URL: avoiding errors from external factors [11]. https://link.springer.com/ All in all, the recommendations and learnings of the article/10.1007/s11042-019-08348-9. analyzed studies show high potentials and various doi:10.1007/ s11042- 019- 08348- 9. possibilities for future research. On one hand, there is [5] S. Deterding, D. Dixon, R. Khaled, L. E. Nacke, still a need for generalized research like the From game design elements to gamefulness: comparison between VR and non-VR or the potential Defining gamification, MindTrek 2011 (2011) 9– of VR as a replacement for in-person training. On the 15. other hand, there are some very detailed findings for [6] R. Timovski, N. Koceska, S. Koceski, The use of single game elements and their applicability. augmented and virtual reality in education (2020). [7] M. I. Baig, L. Shuib, E. Yadegaridehkordi, E- 5.4. Conclusion & future work learning adoption in higher education: A review, Information Development (2021) In this study, we conducted a systematic literature 026666692110082. doi:10. review on a total of 12 studies that implemented 1177/02666669211008224. gamified VR applications for higher education. Our [8] J. Majuri, J. Koivisto, J. Hamari, Gamification of findings provide an overview of current research, education and learning: A review of empirical revealing numerous opportunities for future literature, Proceedings of the 2nd International investigation. We successfully identified several GamiFIN Conference (GamiFIN 2018) 2 (2018) research gaps, that merit further exploration: For 11–19. URL: http://ceur-ws.org/Vol- instance, the application area of gamified VR in higher 2186/paper2.pdf. education closely revolves around STEM and STEM- [9] B. Kitchenham, S. M. Charters, Guidelines for related topics, although gamification and VR also offer performing systematic literature reviews in numerous chances for every other subject area as well. software engineering: Ebse technical report, Furthermore, we identified a list of the most used 2007. 52 [10] A. Hinderks, F. J. D. Mayo, J. Thomaschewski, M. J. expedition mini-games to facilitate Escalona, An slr-tool: search process in practice, computational thinking, Computers & Education: in: G. Rothermel, D.-H. Bae (Eds.), Proceedings of X Reality 2 (2023) 100016. URL: the ACM/IEEE 42nd International Conference on https://www.sciencedirect. Software Engineering: Companion Proceedings, com/science/article/pii/S2949678023000107. ACM, New York, NY, USA, 2020, pp. 81–84. doi:10. 1016/j.cexr.2023.100016. doi:10. 1145/3377812.3382137. [21] S. Rajeev, S. Sharma, Educational game-theme [11] S. Hallifax, M. Altmeyer, K. Kölln, M. based instructional module for teaching Rauschenberger, L. E. Nacke, From points to introductory programming, in: IECON 2018 - progression: A scoping review of game elements 44th Annual Conference of the IEEE Industrial in gamification research with a content analysis Electronics Society, IEEE, 2018. of 280 research papers, Proceedings of the ACM doi:10.1109/iecon.2018. 8592835. on Human-Computer Interaction 7 (2023). [22] S. Rajeev, S. Sharma, A. Sahu, Game theme based [12] A. Thomas, F. Bader, J. Thomaschewski, M. instructional module to teach binary trees data Rauschenberger, Gamification codebook, 2021. structure, Proceedings of ISCA 26th International doi:10.13140/RG.2.2.22625.02403. Conference on Software Engineering and Data [13] H. Cecotti, M. Callaghan, B. Foucher, S. Joslain, Engineering 2017 (2017) 13–18. URL: Serious game for medical imaging in fully https://sharadonly. github.io/papers/19.pdf. immersive virtual reality, in: 2021 IEEE [23] J. Stigall, S. Sharma, Usability and learning International Conference on Engineering, effectiveness of game-themed instructional (gti) Technology & Education (TALE), IEEE, 2021, pp. module for teaching stacks and queues, 615–621. doi:10.1109/ SoutheastCon 2018 (2018) 1–6. TALE52509.2021.9678721. [24] K. Sunday, S. Y. Wong, B. O. Samson, I. T. Sanusi, [14] Q. Chen, S.-E. Low, J. W. Yap, A. K. Sim, Y.-Y. Tan, Investigating the effect of imikode virtual reality B. W. Kwok, J. S. Lee, C.-T. Tan, W.-P. Loh, B. L. Loo, game in enhancing object oriented programming A. C. Wong, Immersive virtual reality training of concepts among university students in nigeria, bioreactor operations, in: 2020 IEEE Education and Information Technologies 27 International Conference on Teaching, (2022) 6819–6845. URL: Assessment, and Learning for Engineering https://link.springer.com/ (TALE), IEEE, 2020, pp. 873–878. article/10.1007/s10639-022-10886-z. doi:10.1109/TALE48869.2020.9368468. doi:10.1007/ s10639- 022- 10886- z. [15] T. H. Froland, I. Heldal, E. Ersvar, G. Sjoholt, [25] C. Wee, K. M. Yap, W. N. Lim, iprogvr: Design of a Merging 360°-videos and game-based virtual virtual reality environment to improve environments for phlebotomy training: Teachers introductory programming learning, IEEE Access and students view, in: 2021 International 10 (2022) 100054–100078. Conference on e-Health and Bioengineering doi:10.1109/ACCESS.2022. 3204392. (EHB), IEEE, 2021, pp. 1–6. [26] T. H. Frøland, I. Heldal, G. Sjøholt, E. Ersvær, doi:10.1109/EHB52898.2021.9657611. Games on mobiles via web or virtual reality [16] S. Nicola, L. Stoicu-Tivadar, A. Patrascoiu, VR for technologies: How to support learning for Education in Information and Tehnology: biomedical laboratory science education, application for Bubble Sort, IEEE, Piscataway, NJ, Information 11 (2020) 195. URL: 2018. URL: https://www.mdpi.com/2078-2489/11/4/195. http://ieeexplore.ieee.org/servlet/opac? doi:10. 3390/info11040195. punumber=8556141. [27] K. Kölln, Maybe we don’t need a new gamification [17] J. Stigall, S. Sharma, Virtual reality instructional framework after all, in: K. Gerling, J. Iacovides, R. modules for introductory programming courses, Malaka, B. Bonsignore, J. Frommel (Eds.), in: 2017 IEEE Integrated STEM Education Extended Abstracts of the Annual Symposium on Conference (ISEC), IEEE, 2017, pp. 34–42. Computer-Human Interaction in Play, ACM, New doi:10.1109/ISECon. 2017.7910245. York, NY, USA, 2022, pp. 384–387. doi:10.1145/ [18] T. Theethum, A. Arpornrat, S. Vittayakorn, 3505270.3558368. Thinkercise: An educational vr game for python [28] K. Rogers, G. Ribeiro, R. R. Wehbe, M. Weber, L. E. programming, in: 2021 18th International Nacke, Vanishing importance: Studying Conference on Electrical immersive effects of game audio perception on Engineering/Electronics, Computer, player experiences in virtual reality, in: Telecommunications and Information Proceedings of the 2018 CHI Conference on Technology (ECTI-CON), IEEE, 2021, pp. 439– Human Factors in Computing Systems, ACM, New 442. doi:10.1109/ECTI- York, NY, USA, 2018. CON51831.2021.9454730. doi:10.1145/3173574.3173902. [19] S. Nicola, I. Virag, Vr medical gamification for [29] S. Nicholson, A user-centered theoretical training and education, in: D. Hayn, G. Schreier framework for meaningful gamification, (Eds.), Health informatics meets eHealth, Studies Games+Learning+Society 8.0 (2012). URL: in health technology and informatics, IOS Press, https://api.semanticscholar.org/CorpusID:9384 Amsterdam and Washington DC, 2017, pp. 97– 012. 103. [20] A. S. Oyelere, F. J. Agbo, S. S. Oyelere, Formative evaluation of immersive virtual reality 53 A. Appendix 1 – Game elements distribution Table 2 The identified game elements in the analyzed studies Game Element 𝑁 Included in Study Game Environments 11 S1, S2, S4, S5, S6, S7, S8, S9, S10, S11, S12 Visuals/Graphics 11 S1, S2, S3, S4, S5, S6, S8, S9, S10, S11, S12 Interactivity 9 S1, S2, S4, S5, S6, S7, S8, S9, S11 Learning 8 S2, S4, S5, S6, S7, S8, S9, S11 Voluntary Approach 8 S1, S2, S4, S5, S6, S8, S9, S11 Tutorial 7 S1, S4, S5, S6, S8, S9, S10 (Personalized) Feedback 5 S1, S2, S5, S8, S11 Points 5 S3, S5, S6, S8, S10 Visual Cues 5 S1, S2, S7, S10, S12 Challenge 3 S2, S6, S8 Free Exploration 3 S2, S8, S9 Goals 3 S4, S5, S8 Level 3 S2, S6, S9 Rewards 3 S8, S9, S12 Score 3 S1, S4, S6 Time Challenge 3 S1, S4, S6 Achievement 2 S1, S2 Any Progress 2 S6, S8 Disincentives/Negative Rewards 2 S3, S8 Rules 2 S6, S8 Sound Effects 2 S5, S12 Theme 2 S5, S11 Virtual Instructor 2 S9, S10 Attractive/Aesthetic design 1 S6 Avatar 1 S5 Competition 1 S1 Feedback System 1 S6 Framing 1 S8 Game Mechanics 1 S8 Leaderboard 1 S1 Mini Games 1 S8 Music Effects 1 S6 Narrative 1 S12 Narrator/Guide 1 S1 Non-Playable Character 1 S11 Onboarding 1 S2 Quiz 1 S1 Scoreboard 1 S9 Virtual World 1 S8 Interface/Character Control 1 S7 54