Towards the Development of Immersive Virtual Reality Games for Spatial Skills Training

Abstract

The purpose of this paper is to set the foundations for a serious game development to train spatial skills in virtual reality. Spatial competences relevance is highlighted as key indicator for STEM careers, as a requirement under the EHEA for technical studies as in engineering, architecture or chemistry degrees. Through immersive experiences simulated with virtual reality technologies, the objective is improving user capacity to understand and visualize threedimensional objects morphology, identify orthogonal projections, perform rotation and cutting spatial operations as well as being able to assemble sets of parts.

Keywords

• Spatial Skills Training
• Immersive Virtual Reality
• Serious Games

Introduction

Since the last 20 years there have been a growing interest on how to increase the interest and engagement for learning via empowering tools such as videogames. Several applications have been recognized as potential tools for improving cognitive processes for mathematics, programming or specific sets of skills such as spatial abilities. Some examples such as Medal of Honor [ 1 ], Minecraft [ 2 ] or Portal 2 [ 3 ] have been proven to help training spatial skills in controlled environments.

Despite these results, there are still open discussion on how to enhance cognitive skills outside laboratory-constrained conditions, meaning applying gamification to academic environments with serious games, which precise purpose is not only enjoying the experience but to learn by playing [ 4 ].

One of the main focus for research is to identify potential game experiences that can improve subset of spatial skills as key indicator for STEM careers success [ 5 ]. Following literature on this matter, there are several factors consider when facing a study on spatial skills training: First, expected higher impact for training low spatial skills population, that tend to struggle with early courses of STEM careers that will likely result on failing or dropping from the course. The issue has been addressed particularly for undergraduates with non-technical education background as well as females, that have been consistently reported as worse performing on spatial abilities tasks than males [ 6 ], therefore considered a relevant factor on the gender gap on technical and scientific degrees. Secondly, students that already possess high level of spatial skills might face little improvement by training or the so called “ceiling effect”. These factors are to be translated into key design principles for developing engaging and effective spatial skills training experience.

Recent growth of virtual reality technology and accessibility to development engines such as Unity have made possible a proposal where immersive qualities of virtual reality allow direct interaction with a gamified experience empowering learning process and motivation for the students, especially those less trained for the competences required to achieve academic goals. 2 2.1

Spatial Skills

Spatial Ability Competence & Relevance

Following Gardner’s Multiple Intelligence theory [ 7 ], intellectual coefficient or capacity cannot be considered a univocal quality such as height or weight, as it is constructed from eight dimensions or types of intelligence. This theory was a complete revolution at educational level, driving the attention for multiple research studies on each of the intelligence areas, one of them being spatial intelligence. The perception of the visual world and the capacity of making transformations on its representations depend on this intelligence. The subset of spatial skills is derived from it as the ability to solve problems between the relation of the environment and the observer – spatial orientation – or the relation between elements and its transformations – spatial visualization [ 8 ].

On the Science, Technology, Engineering and Mathematics (STEM) aspect, there is a relevant number of students accessing higher education courses with less trained spatial skills, this factor impacts negatively on early academic abandon rates [ 9 ]. Thus, first year subjects for technical degrees introduce methods for thinking spatially and understand 2D and 3D representations, such as graphic expression, computer assisted graphics or engineering graphics, where main objective is to solve spatial problems. 2.2

Training and Assessment

In spatial skills development research studies, measurement tools are used to quantify results and gains. Enhancing and assessing skills is the main challenge faced by authors previously as introduced in [ 10 ], where a wide range of measurement tools are used, from which the most common are standardized tests: Mental Rotation Test (MRT). Introduced by Vanderberg and Kuse (1978) [ 11 ], this tests presents 20 exercises to be completed in 20 minutes, where a model figure is presented. To this model certain rotation transformation is applied and the user has to choose two from the four possible options. In these exercises, mirror objects are usually presented as incorrect answers. Spatial rotation and visualization are assessed in this test.

Mental Cutting Test (MCT). Initially developed as higher education access test for USA in 1930, afterwards adopted in Australia, Germany, Poland or Japan [ 12 ]. It consists in 25 exercises where guide model is presented on left side and user has to choose unique correct answer from total five possible results from cutting operation.

Differential Aptitude Test: Spatial Rotations (DAT:SR). This test purpose is to pre

sent a 2D pattern that is to be mentally folded by the student as to result in any number from the four possible options. There are a total of 50 exercises where spatial visualization and spatial relations are evaluated.

Purdue Spatial Visualization Test: Rotations (PSVT:R). Developed with the aim of assessing the ability for solids rotation visualization [ 13 ]. Upper figures show an example of a three-dimensional object rotation, while on the second row the figure to be rotated is presented. Unique answer corresponds with the result of applying the same rotations to the proposed figure. In this rest both mental rotation and spatial visualization are tested.

Immersive Virtual Reality

Definition

Immersive Virtual Reality systems are able to present simulated virtual scenarios that surround the user in a three-dimensional space. Generally, they will provide a firstperson perspective based on and synchronized with head position and orientation from the user. These is achieved through the usage of Head-mounted Displays (HMD) inside out tracking, as the sensors responsible for monitoring are included in the device [ 14 ]. Since the first introduction of VR HMD in 1963 by Hall and Miller, subsequent evolutions such as 3D visual display in 1968 by Sutherland [ 15 ], integrated binaural audio or hand-computer interactions [ 16 ] set the foundations for the technological uprise that Oculus VR headset produced in 2013. Other VR systems have been explored in educational applications such as CAVE or semi-immersive virtual reality, but in this paper is focused on head-mounted displays. 3.2

Hardware

Among current VR HMD we found an increasing number of devices in the market. For this proposal study we have considered three critical characteristics to be present in the device: 1. Inside-out tracking and 6 Degrees Of Freedom (6DOF). Yaw (Z), roll (Y) and nod (X) in addition to head positioning in 3D space. 2. Standalone. Device can be used without being physically connected to a computer, which enables better mobility for the user. 3. High resolution display. Providing good visual and refresh rate needed for planned sessions with the experience.

These guidelines suggest that Oculus Quest 2 device is currently one of the best options available. Future development will consider these hardware characteristics to adjust performance appropriately and produce enjoyable and reliable experience. 4

Related Work

Considerable amount of research has been performed on games as learning environments. Similar to the proposal presented in this paper, other researches found that the level of engagement that videogames create has a positive effect on learning and training purposes. Key factor for this benefit is the level of immersion and progression that a game typically demonstrates. It challenges the players to continuously adapt and take action to complete each level [ 17 ].

Videogames are no longer considered a negative influence on educational growth, in fact, it is now proven that games provide tools for encourage critical thought. This is especially valuable when knowledge is abstract or depends on virtual representations, such as science or technology. Serious games are presented as valid tool for teaching problem-solving skills combining scientific contents with game mechanics linking virtual reality with real world experiences, providing an experience that can be repeatedly played and otherwise impossible to experiment [ 18 ].

In relation to spatial abilities training, there are several proposals consisting on providing immersive virtual reality games that allows users to locate in a simulated scenario where different tasks are presented with different objectives. In Indy [ 19 ], a collaborative treasure hunting trial is presented in an industrial facility environment. These game offers the challenge of finding specific elements hidden by instructors as to train professionals and familiarize with the industrial building structure. Not only spatial orientation is target metrics in this experience but also foster communication between workgroups in order to coordinate every player and achieve common goal.

In the area of standardized tests, there are also gamification proposals such as [ 20 ], for improving spatial skills through MCT. It presents some gamification mechanics applicable to a test-based application with scoreboards shared between users as to compare and reward competitivity. Virtual reality environments are usually used as a tool for digitalized standardized tests, good example can be PSVT:R in [ 21 ] and [ 22 ]. Some applications refer to positive effects on spatial memory for retrieving key aspects of certain elements located in the simulated space or Memory Palace, work from [ 23 ] highlights this effect of using VR in contrast to traditional desktop application.

Spatial Skills evaluation is mentioned one of the challenges that researchers usually face in training courses, for this regard VR game Homeworld Bound: Redux is presented as an experience that combines gathering and resource mechanics and building strategies with spatial transformations for players abilities development [ 24 ]. From this experience, promoting creative modes and avoid using simple 3D objects such as symmetric or regular polyhedrons is recommended for challenges tasks that keeps the user engaged for spatial problems solving.

Another phenomenon studied in this field is embodied cognition in relation to spatial intelligence. This involves physical interfaces combined with virtual simulations so the user can physically interact with a mockup controller influencing or controlling elements of the simulation. In [ 25 ], authors present a VR experience where embodied cognition is the main mechanic for performing spatial puzzles, involving different changeable first person perspectives to acknowledge position and scale of the model as to improve spatial orientation.

Interesting mechanics and gamified strategies are extracted to be evaluated as possible elements of the experience to be developed in order to train and assess spatial skills. From previous studies, it is acknowledged that main target is to consistently provide an engaging experience, that enables data gathering for a pre-post test of a experimental and control group.

Serious Game for Spatial Skill Development

For the proposed VR Spatial Skills Training experience, Unity game engine is selected as a low learning curve software and it is already integrated with Extended Reality Toolkit for Oculus platform. This package provides basic set of interactions for virtual reality that can be modified to adjust development requirements. Since the aim is to produce a Serious Game for training, game mechanics are present to motivate users taking spatial exercises presented as levels, that will increase in difficulty. Not only visual stimuli will be included, audio and haptic for controller-based experience are considered to play key role in the game. Sound effects may indicate if correct action was taken, the start and end of a timer countdown. Haptics, such as controller vibration, will be used as to indicate boolean interactions such as grab or leave object, with light vibration or once player navigate to an option from the main menu. It is important showing players the reason behind a wrong choice, the system should tell correct solution right after wrong answer is provided [ 20 ].

A design principle related to spatial operations is that each task or beforementioned standardized test (MRT, MCT, PSVT and combination of all three) will be presented as a challenge with progression in the shape of unlockable levels. This sense of progression will be presented through user interface through progression bars to show students how many levels have been completed and remain to unlock next one. In this regard difficulty will increase as player will face higher levels. At the end of every level, scoreboards are presented to show player’s performance in comparison with others, to promote competitivity and compare results for each task completed. In order to ensure replayability, new game modes are to be introduced such as Time Trial, where tasks are to be completed within a time limit and Treasure Hunt, where players have to locate through limited spatial operations a key to the next level located inside a 3D model.

Duration of each level should not be superior to 5-10 minutes per player. This way, mental exhaustion is avoided, which helps student’s expectation for next session as well as to ensure experimental phase can be adapted to an undergraduate academic course. For gains evaluation, results from playing the VR experience are to be logged in network shared file so researcher directly exports to data sheet and prepare statistical analysis. 6

Future line of Research

In the upcoming phase, the experience will be developed and tested for fine tuning and usability matters. Then, start of the tests are planned for next academic course in 2022. The main goal is to start the measurements with different user groups. Implementing real-time activities in VR game complemented with theory sessions in engineering graphics course is expected to lead to better results in spatial tasks as well to improve performance at higher level in students from the experimental group than in control group. It is also interesting to perform Likert scale and usability questionnaires for user feedback on improving experience’s immersion and usability for future experimentation. 7

Conclusions

In this paper, relevance of spatial competences for higher education STEMs careers is highlighted. Currently there is a need for training spatial skills in early years of technical degrees as it is a required competence, is also an indicator for academic success and may help reducing gender gap differences on performance and early abandon rates.

A new VR Training game is presented as concept of an application whose goal is to engage and motivate users who would like to improve their visual skills by completing engaging exercises presented as levels, especially those with lower level of spatial abilities [ 26 ]. 8

References

1. J. Feng , I. Spence , and J. Pratt : Playing an action video game reduces gender differences in spatial cognition . Psychological Science , vol. 18 , no. 10 , pp. 850 - 855 , ( 2007 )

2. C. Carbonell-Carrera, A. J. Jaeger , J. L. Saorín , D. Melián , and J. de la Torre-Cantero: Minecraft as a block building approach for developing spatial skills . Entertainment Computing , vol. 38 , no. March , ( 2021 )

3. V. J. Shute , M. Ventura , and F. Ke : The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills . Computers and Education , vol. 80 , pp. 58 - 67 , 2015

4. M. Romero: Serious Games para el desarrollo de las competencias del siglo XXI. RED . Revista de Educación a Distancia, no. 34 , pp. 1 - 22 , ( 2012 )

5. J. Wai , D. Lubinski , and C. P. Benbow : Spatial Ability for STEM Domains: Aligning Over 50 Years of Cumulative Psychological Knowledge Solidifies Its Importance . J. Educ. Psychol. , vol. 101 , no. 4 , pp. 817 - 835 , ( 2009 )

6. C. A. Lawton: Gender, Spatial Abilities, and Wayfinding . in Handbook of Gender Research in Psychology, Chrisler, New York, NY: Springer New York, pp. 317 - 341 ( 2010 )

7. H. Gardner , Estructuras de la mente. La Teoría de Las Inteligencias Múltiples. ( 1994 )

8. M. G. McGee: Human spatial abilities: Sources of sex differences , Praeger . ( 1979 )

9. D. H. Uttal and C. A. Cohen : Spatial Thinking and STEM Education . When, Why, and How?, vol. 57 , no. October 2017 . ( 2012 )

10. R. Gorska and S. Sorby: Testing instruments for the assessment of 3-D spatial skills . ASEE Annu. Conf. Expo. Conf. Proc. , June ( 2008 )

11. S. G. Vanderberg and A. R. Kuse : Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization . Percept. Mot. Skills , vol. 47 , no. 2 ( 1978 )

12. CEEB: Special Aptitude Test in Spatial Relations (MCT) . ( 1939 )

13. G. M. Bodner and R. B. Guay : The Purdue Visualization of Rotations Test . Chem. Educ., vol. 2 , no. 4 , pp. 1 - 17 , ( 1997 )

14. W. S. Alhalabi: Virtual reality systems enhance students' achievements in engineering education . Behav . Inf. Technol., vol. 35 , no. 11 , pp. 919 - 925 , ( 2016 )

15. I. E. Sutherland : A head-mounted three dimensional display . in AFIPS '68 - Proceedings of the Fall Joint Computer Conference , vol. 1 , pp. 757 - 764 ( 1968 )

16. T. G. Zimmerman, J. Lanier , C. Blanchard , S. Bryson , and Y. Harvill : Hand Gesture Interface Device . Proc. - Graph. Interface , pp. 189 - 192 , ( 1987 )

17. J. Hamari , D. J. Shernoff , E. Rowe , B. Coller , J. Asbell-Clarke , and T. Edwards: Challenging games help students learn: An empirical study on engagement, flow and immersion in gamebased learning . Comput. Human Behav ., vol. 54 , pp. 170 - 179 , ( 2016 )

18. M. T. Cheng, H. C. She, and L. A. Annetta : Game immersion experience: Its hierarchical structure and impact on game-based science learning . J. Comput. Assist. Learn. , vol. 31 , no. 3 , pp. 232 - 253 , ( 2015 )

19. A. Mas , I. Ismael , and N. Filliard: Indy: A virtual reality multi-player game for navigation skills training . 2018 IEEE 4th VR Int. Work. 3D Collab. Virtual Environ. 3DCVE 2018 , no. March , pp. 1 - 4 , ( 2019 )

20. R. Toth , M. Zichar , and M. Hoffmann: Gamified mental cutting test for enhancing spatial skills . 11th IEEE Int. Conf. Cogn. Infocommunications, CogInfoCom 2020 - Proc. , pp. 299 - 304 , ( 2020 )

21. D. F. Ali , M. Omar , M. Mokhtar , and A. N. M. Nasir: Application of virtual learning in the teaching of engineering drawing to enhance students' mental rotation skills . Proc. 1st Int. Conf. Innov. Sci. Technol . (IICIST 2015 ), vol. 1 , no. January , pp. 469 - 472 , ( 2015 )

22. T. Guzsvinecz, É. Orbán-Mihálykó , E. Perge , and C. Sik-Lányi: Analyzing the spatial skills of university students with a virtual reality application using a desktop display and the gear VR . Acta Polytech. Hungarica , vol. 17 , no. 2 , pp. 35 - 56 , ( 2020 )

23. B. Berki: Better Memory Performance for Images in MaxWhere 3D VR Space than in Website . 9th IEEE Int. Conf. Cogn. Infocommunications, CogInfoCom 2018 - Proc., no. CogInfoCom , pp. 281 - 284 , ( 2018 )

24. H. Wauck , B. S. Woodard , Z. Xiao , T. W. Li , and B. P. Bailey : A Data-Driven, PlayerCentric Approach to Evaluating Spatial Skill Training Games . CHI Play 2020 - Proc. Annu. Symp. Comput. Interact. Play , pp. 349 - 361 , ( 2020 )

25. J. S. K. Chang et al.: Evaluating the effect of tangible virtual reality on spatial perspective taking ability . SUI 2017 - Proc . 2017 Symp. Spat. User Interact., pp. 68 - 77 , ( 2017 )

26. H. C. Wauck , E. D. Mekler , and W.-T. Fu: A Player-Centric Approach to Designing Spatial Skill Training Games . In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , New York, NY, USA: Association for Computing Machinery, pp. 1 - 13 ( 2019 )