Walking in VR: Measuring Presence and Simulator Sickness in First-Person Virtual Reality Games Manuel López Ibáñez and Federico Peinado Departamento de Ingenierı́a del Software e Inteligencia Artificial Universidad Complutense de Madrid c/ Profesor José Garcı́a Santesmases 9, 28040 Madrid (Spain) manuel.lopez.ibanez@ucm.es - email@federicopeinado.com http://nil.fdi.ucm.es Abstract. Presence is often used as a quality measure for virtual reality experiences. It refers to the sensation of “being there” that users feel while wearing a head-mounted display. In contrast, simulator sickness refers to the feeling of unease of some users while experiencing virtual motion. Nowadays, many virtual reality games do not allow the player to walk, trying to minimize the generation of unpleasant symptoms. This study explores how presence is affected by the ability to walk in VR games, as well as how simulator sickness actually grows when the player takes a virtual stroll. For this purpose, two prototypes of a small puzzle were built. In the first one, the player is able to walk, whereas the second one does not allow the user to move in any way. Presence and simulator sickness were measured using standard questionnaires while real players faced our puzzle. The results point to a strong correlation between the action of walking and an increment of the level of presence achieved by the subjects. However, there is no clear correspondence between walking and simulator sickness in our experiment. This last observation opens the way for further research and questioning of early studies about simulator sickness, as technical differences between current virtual reality devices and older ones may influence how uncomfortable users feel while wearing them. Keywords: Virtual Movement · Virtual Environments · Head-Mounted Displays · Human-Computer Interaction · Interactive Multimedia 1 Introduction In spite of the big success virtual reality (VR) is experiencing these days, most games designed for VR headsets do not include one fundamental feature: the ability to walk in a first-person (FP) perspective through the virtual environ- ment. The reason for making this decision is that both academics and developers seem to agree that such an immersive virtual movement produces a disagreeable effect known as simulator sickness (SS). However, this presumption is based on dated papers [1–3] which were not written taking into consideration current ad- vances in VR technologies and experiences. We are interested in knowing how relevant walking inside a VR environment is for achieving higher presence levels, as well as checking whether the mere act of walking actually increases SS. Thus, the aim of this research is to measure how presence is related to the action of walking in a FP VR environment, and to check how this relation is connected to SS when using a specific VR device: the Oculus Rift Development Kit 2. The alternative hypothesis (H1 ) is that presence increases when the user is able to walk in a VR experience, while the null hypothesis (H0 ) is that presence does not change at all. Usoh, Arthur, Whitton et al. [4] have already stated that oculomotor dis- comfort, one of the diagnostic subscales of the Simulator Sickness Questionnaire (SSQ) [1] increased when virtually walking or flying, and so reduced the level of presence achieved while exposed to VR. However, VR devices and games have changed significantly since 1999, and now allow more comfortable experiences, which means there could be differences in the level of SS felt while moving in a VR environment. Presence [5–7] is a widespread way of measuring the capacity of an interactive experience to make its users feel “they really are” in a virtual environment. Experts in this field, however, refer to this concept in a variety of ways [8]. The most common are: telepresence, which is the capacity of a virtual environment to create an illusion of “being there”, and social presence, which describes the feeling of “being there with another” and analyses, for example, interactions between real and virtual humans when they confront each other. For this article, we have picked the definition of presence that refers to a “transportation” [9] of the user’s conscience to a virtual environment, which is useful when evaluating the kind of situations that will be described throughout this paper. Moreover, as there are not any interactions with human-like characters in our experiment, the concept of social presence lacks relevance. Besides, now that virtual reality is becoming a commonly available technol- ogy, and considering how high is the game industry’s contribution to its popu- larization, it would be useful to gather presence measures and relate them to SS fluctuations. Simulator sickness is a byproduct of high quality simulation tech- nologies [1]. Vection, a “visually induced illusory self-motion” [10], is considered to be at the root of SS, and acceleration seems also more important than speed when it comes to generating SS [11]. The concept of SS refers to the feeling of unease generated by a simulation in which the user experiences virtual motion that is not synchronized with real movements of his or her body. Devices like the HTC Vive let users move to a certain extent, and reflect real-world movement inside the experience with very low latency (with a target of around 11 ms from moving the head to perceiving the effects of that movement). However, their tracking capacities are not enough to allow a player to walk long distances in VR — just up to an area of 5 x 5 meters. Previous studies [12, 13] have already shown video games are a type of simu- lation that can increase SS. Serge & Moss [13] also stated that SS did not seem to change between observational and navigational tasks while using an Oculus Rift, but it did rise over time in both experiences. Earlier studies [4, 14–16] suggest that virtual walking representation tech- niques have influence over presence in virtual environments. However, they do not go deep into the relation between staying still or walking and presence. That is why we designed the following experiment, aiming at discovering how an in- dependent boolean variable (walking or not walking) is related to two numerical variables: presence and SS. 2 Experiment Design During this investigation, we used a series of principles to design two virtual re- ality experiences which constitute the main core of our experiment. Sherman & Craig [17] defined four key elements of virtual reality: virtual world, immersion, sensory feedback and interactivity. Those elements are contained in a virtual reality system, which has inputs (user and world monitoring) and outputs (vi- sual, aural, haptic and vestibular displays). We used this taxonomy to create two experiences which explicitly contained all of the described elements that can be considered when using a HMD like the Oculus Rift Development Kit 2. In our simulation, walking was similar to what Usoh, Arthur, Witton et al. [4] define as flying —that is, locomotion along head direction, with an absence of real world walking movement. In order to navigate the virtual environment, the player could use the left stick of a standard Xbox 360 controller. Forward movement would follow the direction of the player’s camera, while it was also possible to move sideways (strafe) and back. The speed of the camera while moving was similar to that of a human while jogging (approximately 2 m/s), and acceleration depended on the amount of tilting of the controller’s left stick. This kind of movement is how walking is usually implemented in a conventional first-person game. There was no head bobbing when moving. So as to measure the effect of walking on presence and SS, we built two slightly different VR game prototypes. Both of them were based on a small puzzle that had to be resolved to trigger the end of the experience; however, they presented differences in the distribution of in-game elements, as we will explain later. These prototypes were made in Unreal Engine 4, and used some free assets from the video game Infinity Blade (made available for use with this engine by its developer, Epic Games), together with some self-made assets. The above-mentioned categories defined by Sherman & Craig [17] were used to design the content of both experiences in the following way: – Inputs: – World input and persistence: The virtual world was responsive and be- haved in a realistic manner. The player could collide with walls and objects, and a physics engine was implemented, so that smaller objects could be moved thanks to the force of gravity and the impulses coming from the player. – User input and monitoring: Input from the player was collected by using two devices. On one hand, the HMD built-in positional tracker was used to track the head; on the other, a Xbox 360 gamepad was used to track player movement. – Outputs: – Visual outputs: An Oculus Rift was used as the visual output during both experiences. Its screen has a resolution of 960 x 1080 pixels per eye, and a maximum refresh rate of 75 Hz, which was stable in both experiences. – Aural outputs: Every user wore a pair of headphones (JVC HA-S400- B) during the experiment. This was the only source of aural output used. The prototypes included both ambient sound and responsive sound (when the player stepped over a metallic object, for example). – World representation: The world had a low abstraction level, and the user could easily face all visual elements required for the completion of the puzzle. Both experiences took place in the same virtual world, the only difference being the position of key elements. A neutral subject was asked to identify all actions and elements present in both experiences while wearing a HMD, so as to ensure the noticeable core elements were the same. This subject is a 25 years old male, with a background in Computer Science, and no knowledge about this experiment beforehand. When he was told to separate actions and game elements present in the two prototypes, he identified the following ones: – Game actions: move, look at, interact and listen. – Game elements: walls, floor, rocks, ceramic and metal objects, windows, ceiling, pedestals, orbs of three different colours, statues, fire, fog effects, sounds and ambiance. The first scenario (W ) (see figures 1 & 2) consisted on a 3D environment that the user could explore by using the specified HMD and game controller. This environment contained a puzzle which could be resolved by following a series of steps: the player had first to walk towards one of the three statues (S), located behind a pillar (P ) on top of which there was a lit orb. This orb could be red, green or blue, depending on its position. Once in front of one of the statues, the player had to extinguish a fire that the statue held in her hands by pressing the ’A’ button in the controller (an in-game tooltip indicated which button to press). The correspondent orb would also turn off as a result, leaving only the other two lit. The player had to repeat this process until no orbs were lit. Then, a gate would open in the place where the player first spawned (I). If the player stepped through it, the experience ended, and the puzzle was considered resolved. W NW S P P S S P P S P S I S P I Fig. 1. Schematic representation of the layout of the two prototypes. Fig. 2. View of a section of the 3D environment designed for walking. Fig. 3. View of a section of the 3D environment designed for not walking. The second scenario (N W ) (see figures 1 & 3) was designed in a similar fashion, and contained the same puzzle, but in-game elements were distributed in a way that made all of them accessible from the player’s spawn location. The only movement allowed for the player in this case was to rotate their head — and so, the camera view. The solution to the puzzle was, however, the same: the player had to turn off three fires so that the gate appeared in the spawn location. This gate acted as a portal in this case, activating and ending the experience as soon as the lights turned off. 2.1 Measures The measures collected during the experiment were the following: – Presence questionnaires: The Slater-Usoh-Steed Questionnaire (SUS) [18] was used in conjunction with the Temple Presence Inventory (TPI) [9]. Re- sults were normalized in order to be able to compare them for verification purposes. – Simulator Sickness Questionnaire: The Simulator Sickness Questionnaire (SSQ) [1] was also used to measure the level of sickness of each user after the experience. – Completion of the in-game puzzle: We also measured if the player finished the experiment by completing the puzzle or not. The puzzle was optional, and this was told to every subject before beginning the experiment. Only the engagement, spatial presence and perceptual realism subscales of the TPI were used, due to the nature of our experiment. Social presence, social richness and social realism were not interesting in this case, as there are not social interactions throughout the experiences. All users were asked simply to play each game, and then to answer a series of questions about it. The order in which questionnaires were offered to the subjects was determined by the need of passing the SSQ first, while the potential effects of SS were still active. SUS and TPI came after it, in that order. 2.2 Demography Our experimental group was a set of 12 subjects that went through both experi- ences, so as to be able to compare their reactions to them. Half of them did the two experiments in one order; while the other half did them inversely. Besides, none of the subjects who took part in the experiment knew about the contents of the second experience before finishing the first one. To participate in this experiment, all subjects had to sign-up by answering an email which retrieved the following pieces of information: – Name. – Age. – Gender. – Education. – VR device familiarity. – VR devices present at home. – Video game familiarity. Among the subjects, 11 (91.67 %) were men, and 1 (8.33 %) was a woman. Their ages ranged from 18 to 26 years old. About their background, 8 (66.67 %) had an education in Computer Science, while 3 (25 %) studied a degree related to Video Game Development and 1 (8.33 %) had studied Graphic Design. About VR, 4 (33.33 %) of them did not have any familiarity with VR devices, and had never tested a HMD, while the rest had used at least one VR system; 8 (66.67 %) did not own any VR device, and 4 (33.33 %) had a Google Cardboard at home. All of them said to be familiar with video games, and declared themselves frequent players. 3 Results Analysis The obtained results point to a strong correlation between the action of walking and an increment in presence. Both SUS and TPI tests presented a p-value much lower than 0.05, and both offered very similar normalized results, as can be seen in tables 1 and 2. This is relevant if we consider that in-game movement did not imply real-world motion to any extent. The lack of correspondence between in-game and real-life motion does not seem to affect how presence rises when the user is able to walk (or fly, as we could call it) in VR. Figures 4 and 5 show subject numbers in the abscissa axis, and normalized levels of presence achieved in the ordinate axis. Figure 4 presents a more irregular shape than figure 5, but this seems to be caused by the different number of questions in both tests. The first one had only 6, whereas the version of the TPI that we used had 18, which allowed for a less polarized mean. The correlation is, in both cases, high enough to allow us to think the results of these tests are consistent. Table 1. Normalized SUS test results (One-way ANOVA). Walking Yes No N 12 12 SUM (X) 8.285714 7 Mean 0.690476 0.583333 Variance 0.010101 0.003556 p-value 0.00413 Table 2. Normalized TPI test results (One-way ANOVA). Walking Yes No N 12 12 SUM (X) 7.759259 6.092593 Mean 0.646605 0.507716 Variance 0.007386 0.004502 p-value < 0.0001 Fig. 4. Normalized results of the SUS presence test for both walking (W ) and not walking (N W ) experiences. Fig. 5. Normalized results of the TPI test for both walking (W ) and not walking (N W ) experiences. The SSQ, however, offered results which, in spite of varying between both prototypes, did not present a significant correlation (with a p-value of approx- imately 0.12) when processed with a one-way ANOVA, as can be seen in table 3. Walking did not seem to notoriously affect SS. Only subject number 7 (see figure 6) presented a clear increment in SS while walking. However, this subject had defined himself, before taking part in our experiment, as especially prone to nausea and motion sickness. Table 3. Normalized SSQ test results (One-way ANOVA) Walking Yes No N 12 12 SUM (X) 2.416667 1.0625 Mean 0.201389 0.088542 Variance 0.045718 0.006323 p-value 0.120654 Fig. 6. Normalized results of the Simulator Sickness Questionnaire for both walking (W ) and not walking (N W ) experiences. 4 Discussion Notwithstanding the strong relation between the SUS and TPI tests results, the reduced group of subjects we had to work with leads us to think the obtained data can be considered reliable just to a certain extent, and should be broadened in the future. Besides, most voluntaries who took part in the experiment were men, which means there could exist a genre bias in the data. Though the results of both SUS and TPI questionnaires were consistent, it would also have helped to count on a strongest numeric measure, such as the data from an electrocardiogram (ECG). This was not possible due to limited time and resources, but would be desirable in future research. SS was not measured over time, as we wanted to check if only the mere act of walking increased it. However, we do not know the effects a higher level of SS granted by a longer experience may have when comparing the results of both prototypes. Besides, the order in which both experiences were taken could have influenced the level of SS achieved. Moreover, all of our subjects said to have a notable experience in playing video games, and some of them had already tested VR devices, which means they could be more used to virtual motion, and therefore more resistant to simulator sickness in short and focused experiences as those of our experiment. 5 Conclusions In conclusion, the results of this experiment negate the null hypothesis (H0 ) and enforce our alternative (H1 ): there is a significant difference between walkable and non-walkable first person VR games when comparing the independent vari- able to presence. Being able to move longer distances than what the tracking capacities of the HMD allow, even if there are not real body movements associ- ated with the act of walking, generates a clear increment in the level of presence achieved by the player. On the other hand, judging solely by the results of this experiment, there does not seem to be a clear correlation between the act of walking and SS. Though this may have been caused by the lack of a broader set of subjects, the stable nature of the results obtained from SUS and TPI presence tests with the same, narrow group, leads us to think there is actually a lack of SS generation in our game prototypes. This opens a new line of investigation, as it would be useful to know if it is related to the technical properties of current VR devices, the design principles assumed by the authors, or to something else. Current game developers could benefit from this combination of results. Most VR experiences lack the possibility to walk the environment due to a fear of SS; however, brief and simple experiences like the ones we designed for this experiment have shown no clear correlation between walking and SS. This paves the way to more walkable experiences in the future, as we think player movement is a fundamental asset in modern video games. It allows for complex interaction while increasing presence, and differentiates video games from static experiences which also use a HMD, like VR video. Acknowledgements We would like to thank all volunteers who took part in our experiment for giving us their time and effort. This research was supported through EC funding for the project WHIM 611560 by FP7, the ICT theme, and the Future Emerging Technologies (FET) programme. References 1. Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The In- ternational Journal of Aviation Psychology 3(3), 203–220 (1993) 2. Rolnick, A., Lubow, R.E.: Why is the Driver Rarely Motion Sick? The Role of Controllability in Motion Sickness. Ergonomics 34(7), 867–879 (1991) 3. Ehrlich, J.A., Singer, M.J.: Simulator Sickness in Stereoscopic vs. Monoscopic Hel- met Mounted Displays. In: Proceedings of the Human Factors and Ergonomics So- ciety Annual Meeting. Human Factors and Ergonomics Society, Philadelphia (1996) 4. Usoh, M., Arthur, K., Whitton, M.C., Bastos, R., Steed, A., Slater, M., Brooks, F.P.: Walking, Walking-in-place, Flying, in Virtual Environments. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pp. 359–364. ACM Press, New York (1999) 5. Sheridan, T.B.: Musings on Telepresence and Virtual Presence. Presence: Teleoper- ators and Virtual Environments 1(1), 120–126 (1992) 6. Barfield, W., Weghorst, S.: The Sense of Presence Within Virtual Environments: A Conceptual Framework. Advances in Human Factors Ergonomics. 19, 699 (1993) 7. Mestre, D., Fuchs, P., Berthoz, A., Vercher, J.L.: Le Traité de La Réalité Virtuelle. Presses des Mines, Paris (2006) 8. Biocca, F., Harms, C., Burgoon, J.K.: Toward a More Robust Theory and Measure of Social Presence: Review and Suggested Criteria. Presence: Teleoperators and Virtual Environments 12(5), 456–480 (2003) 9. Lombard, M., Ditton, T., Weinstein, L.: Measuring Presence: The Temple Presence Inventory. In: Proceedings of the 12th Annual International Workshop on Presence, pp. 1–15. Institute for Creative Technologies, Los Angeles (2009) 10. Hettinger, L.J., Berbaum, K.S., Kennedy, R.S., Dunlap, W.P., Nolan, M.D.: Vec- tion and Simulator Sickness. Military Psychology 2(3), 171–181 (1990) 11. So, R.H.Y., Lo, W.T., Ho, A.T.K.: Effects of Navigation Speed on Motion Sick- ness Caused by an Immersive Virtual Environment. Human Factors 43(3), 452–461 (2001) 12. Merhi, O., Thomas, A.: Motion Sickness, Video Games, and Head-Mounted Dis- plays. In: Proceedings of the Human Factors and Ergonomics Society Annual Meet- ing, pp. 2618–2622. Human Factors and Ergonomics Society, New Orleans (2004) 13. Serge, S.R., Moss, J.D.: Simulator Sickness and the Oculus Rift: A First Look. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 761–765. Human Factors and Ergonomics Society, Los Angeles (2015) 14. Slater, M., Usoh, M., Steed, A.: Taking Steps: The Influence of a Walking Technique on Presence in Virtual Reality. ACM Transactions on Computer-Human Interaction 2(3), 201–219 (1995) 15. Lorenz, M., Busch, M., Rentzos, L., Tscheligi, M., Klimant, P., Frohlich, P.: I’m There! The Influence of Virtual Reality and Mixed Reality Environments Combined with Two Different Navigation Methods on Presence. In: Proceedings of IEEE 2015 Virtual Reality, pp. 223–224. IEEE, Arles (2015) 16. Nielsen, M.R., Toft, C., Nilsson, N.C., Nordahl, R., Serafin, S.: Evaluating Two Alternative Walking in Place Interfaces for Virtual Reality Gaming. In: Proceedings of IEEE 2016 Virtual Reality, pp. 299–300. IEEE, Greenville (2016) 17. Sherman, W., Craig, A.: Understanding Virtual Reality: Interface, Application, and Design. Elsevier Science, San Francisco (2003) 18. Slater, M., Usoh, M., Steed, A.: Depth of Presence in Virtual Environments. Pres- ence: Teleoperators and Virtual Environments 3, 130–144 (1994)