=Paper= {{Paper |id=Vol-1682/CoSeCiVi16_paper_8 |storemode=property |title=Walking in VR: Measuring Presence and Simulator Sickness in First-Person Virtual Reality Games |pdfUrl=https://ceur-ws.org/Vol-1682/CoSeCiVi16_paper_8.pdf |volume=Vol-1682 |authors=Manuel López Ibáñez,Federico Peinado |dblpUrl=https://dblp.org/rec/conf/cosecivi/Lopez-IbanezP16 }} ==Walking in VR: Measuring Presence and Simulator Sickness in First-Person Virtual Reality Games== https://ceur-ws.org/Vol-1682/CoSeCiVi16_paper_8.pdf

Walking in VR:
Measuring Presence and Simulator Sickness in
First-Person Virtual Reality Games

Manuel López Ibáñez and Federico Peinado

Departamento de Ingenierı́a del Software e Inteligencia Artificial
Universidad Complutense de Madrid
c/ Profesor José Garcı́a Santesmases 9, 28040 Madrid (Spain)
manuel.lopez.ibanez@ucm.es - email@federicopeinado.com
http://nil.fdi.ucm.es

Abstract. Presence is often used as a quality measure for virtual reality
experiences. It refers to the sensation of “being there” that users feel
while wearing a head-mounted display. In contrast, simulator sickness
refers to the feeling of unease of some users while experiencing virtual
motion. Nowadays, many virtual reality games do not allow the player to
walk, trying to minimize the generation of unpleasant symptoms. This
study explores how presence is affected by the ability to walk in VR
games, as well as how simulator sickness actually grows when the player
takes a virtual stroll. For this purpose, two prototypes of a small puzzle
were built. In the first one, the player is able to walk, whereas the second
one does not allow the user to move in any way. Presence and simulator
sickness were measured using standard questionnaires while real players
faced our puzzle. The results point to a strong correlation between the
action of walking and an increment of the level of presence achieved by
the subjects. However, there is no clear correspondence between walking
and simulator sickness in our experiment. This last observation opens the
way for further research and questioning of early studies about simulator
sickness, as technical differences between current virtual reality devices
and older ones may influence how uncomfortable users feel while wearing
them.

Keywords: Virtual Movement · Virtual Environments · Head-Mounted
Displays · Human-Computer Interaction · Interactive Multimedia

1 Introduction

In spite of the big success virtual reality (VR) is experiencing these days, most
games designed for VR headsets do not include one fundamental feature: the
ability to walk in a first-person (FP) perspective through the virtual environ-
ment. The reason for making this decision is that both academics and developers
seem to agree that such an immersive virtual movement produces a disagreeable
effect known as simulator sickness (SS). However, this presumption is based on
dated papers [1–3] which were not written taking into consideration current ad-
vances in VR technologies and experiences. We are interested in knowing how
relevant walking inside a VR environment is for achieving higher presence levels,
as well as checking whether the mere act of walking actually increases SS.
Thus, the aim of this research is to measure how presence is related to the
action of walking in a FP VR environment, and to check how this relation is
connected to SS when using a specific VR device: the Oculus Rift Development
Kit 2. The alternative hypothesis (H1 ) is that presence increases when the user is
able to walk in a VR experience, while the null hypothesis (H0 ) is that presence
does not change at all.
Usoh, Arthur, Whitton et al. [4] have already stated that oculomotor dis-
comfort, one of the diagnostic subscales of the Simulator Sickness Questionnaire
(SSQ) [1] increased when virtually walking or flying, and so reduced the level of
presence achieved while exposed to VR. However, VR devices and games have
changed significantly since 1999, and now allow more comfortable experiences,
which means there could be differences in the level of SS felt while moving in a
VR environment.
Presence [5–7] is a widespread way of measuring the capacity of an interactive
experience to make its users feel “they really are” in a virtual environment.
Experts in this field, however, refer to this concept in a variety of ways [8]. The
most common are: telepresence, which is the capacity of a virtual environment
to create an illusion of “being there”, and social presence, which describes the
feeling of “being there with another” and analyses, for example, interactions
between real and virtual humans when they confront each other. For this article,
we have picked the definition of presence that refers to a “transportation” [9] of
the user’s conscience to a virtual environment, which is useful when evaluating
the kind of situations that will be described throughout this paper. Moreover,
as there are not any interactions with human-like characters in our experiment,
the concept of social presence lacks relevance.
Besides, now that virtual reality is becoming a commonly available technol-
ogy, and considering how high is the game industry’s contribution to its popu-
larization, it would be useful to gather presence measures and relate them to SS
fluctuations. Simulator sickness is a byproduct of high quality simulation tech-
nologies [1]. Vection, a “visually induced illusory self-motion” [10], is considered
to be at the root of SS, and acceleration seems also more important than speed
when it comes to generating SS [11].
The concept of SS refers to the feeling of unease generated by a simulation
in which the user experiences virtual motion that is not synchronized with real
movements of his or her body. Devices like the HTC Vive let users move to a
certain extent, and reflect real-world movement inside the experience with very
low latency (with a target of around 11 ms from moving the head to perceiving
the effects of that movement). However, their tracking capacities are not enough
to allow a player to walk long distances in VR — just up to an area of 5 x 5
meters.
Previous studies [12, 13] have already shown video games are a type of simu-
lation that can increase SS. Serge & Moss [13] also stated that SS did not seem
to change between observational and navigational tasks while using an Oculus
Rift, but it did rise over time in both experiences.
Earlier studies [4, 14–16] suggest that virtual walking representation tech-
niques have influence over presence in virtual environments. However, they do
not go deep into the relation between staying still or walking and presence. That
is why we designed the following experiment, aiming at discovering how an in-
dependent boolean variable (walking or not walking) is related to two numerical
variables: presence and SS.

2 Experiment Design

During this investigation, we used a series of principles to design two virtual re-
ality experiences which constitute the main core of our experiment. Sherman &
Craig [17] defined four key elements of virtual reality: virtual world, immersion,
sensory feedback and interactivity. Those elements are contained in a virtual
reality system, which has inputs (user and world monitoring) and outputs (vi-
sual, aural, haptic and vestibular displays). We used this taxonomy to create two
experiences which explicitly contained all of the described elements that can be
considered when using a HMD like the Oculus Rift Development Kit 2.
In our simulation, walking was similar to what Usoh, Arthur, Witton et al.
[4] define as flying —that is, locomotion along head direction, with an absence
of real world walking movement. In order to navigate the virtual environment,
the player could use the left stick of a standard Xbox 360 controller. Forward
movement would follow the direction of the player’s camera, while it was also
possible to move sideways (strafe) and back. The speed of the camera while
moving was similar to that of a human while jogging (approximately 2 m/s),
and acceleration depended on the amount of tilting of the controller’s left stick.
This kind of movement is how walking is usually implemented in a conventional
first-person game. There was no head bobbing when moving.
So as to measure the effect of walking on presence and SS, we built two
slightly different VR game prototypes. Both of them were based on a small
puzzle that had to be resolved to trigger the end of the experience; however,
they presented differences in the distribution of in-game elements, as we will
explain later.
These prototypes were made in Unreal Engine 4, and used some free assets
from the video game Infinity Blade (made available for use with this engine by
its developer, Epic Games), together with some self-made assets.
The above-mentioned categories defined by Sherman & Craig [17] were used
to design the content of both experiences in the following way:

– Inputs:
– World input and persistence: The virtual world was responsive and be-
haved in a realistic manner. The player could collide with walls and
objects, and a physics engine was implemented, so that smaller objects
could be moved thanks to the force of gravity and the impulses coming
from the player.

– User input and monitoring: Input from the player was collected by using
two devices. On one hand, the HMD built-in positional tracker was used
to track the head; on the other, a Xbox 360 gamepad was used to track
player movement.

– Outputs:
– Visual outputs: An Oculus Rift was used as the visual output during
both experiences. Its screen has a resolution of 960 x 1080 pixels per
eye, and a maximum refresh rate of 75 Hz, which was stable in both
experiences.
– Aural outputs: Every user wore a pair of headphones (JVC HA-S400-
B) during the experiment. This was the only source of aural output
used. The prototypes included both ambient sound and responsive sound
(when the player stepped over a metallic object, for example).
– World representation: The world had a low abstraction level, and the
user could easily face all visual elements required for the completion of
the puzzle. Both experiences took place in the same virtual world, the
only difference being the position of key elements.

A neutral subject was asked to identify all actions and elements present
in both experiences while wearing a HMD, so as to ensure the noticeable core
elements were the same. This subject is a 25 years old male, with a background in
Computer Science, and no knowledge about this experiment beforehand. When
he was told to separate actions and game elements present in the two prototypes,
he identified the following ones:

– Game actions: move, look at, interact and listen.
– Game elements: walls, floor, rocks, ceramic and metal objects, windows,
ceiling, pedestals, orbs of three different colours, statues, fire, fog effects,
sounds and ambiance.

The first scenario (W ) (see figures 1 & 2) consisted on a 3D environment that
the user could explore by using the specified HMD and game controller. This
environment contained a puzzle which could be resolved by following a series of
steps: the player had first to walk towards one of the three statues (S), located
behind a pillar (P ) on top of which there was a lit orb. This orb could be red,
green or blue, depending on its position. Once in front of one of the statues, the
player had to extinguish a fire that the statue held in her hands by pressing the
’A’ button in the controller (an in-game tooltip indicated which button to press).
The correspondent orb would also turn off as a result, leaving only the other two
lit. The player had to repeat this process until no orbs were lit. Then, a gate
would open in the place where the player first spawned (I). If the player stepped
through it, the experience ended, and the puzzle was considered resolved.
W NW
S
P
P S
S P P S P S I S P
I

Fig. 1. Schematic representation of the layout of the two prototypes.

Fig. 2. View of a section of the 3D environment designed for walking.

Fig. 3. View of a section of the 3D environment designed for not walking.
The second scenario (N W ) (see figures 1 & 3) was designed in a similar
fashion, and contained the same puzzle, but in-game elements were distributed
in a way that made all of them accessible from the player’s spawn location. The
only movement allowed for the player in this case was to rotate their head —
and so, the camera view. The solution to the puzzle was, however, the same: the
player had to turn off three fires so that the gate appeared in the spawn location.
This gate acted as a portal in this case, activating and ending the experience as
soon as the lights turned off.

2.1 Measures

The measures collected during the experiment were the following:

– Presence questionnaires: The Slater-Usoh-Steed Questionnaire (SUS) [18]
was used in conjunction with the Temple Presence Inventory (TPI) [9]. Re-
sults were normalized in order to be able to compare them for verification
purposes.
– Simulator Sickness Questionnaire: The Simulator Sickness Questionnaire
(SSQ) [1] was also used to measure the level of sickness of each user after
the experience.
– Completion of the in-game puzzle: We also measured if the player finished
the experiment by completing the puzzle or not. The puzzle was optional,
and this was told to every subject before beginning the experiment.

Only the engagement, spatial presence and perceptual realism subscales of
the TPI were used, due to the nature of our experiment. Social presence, social
richness and social realism were not interesting in this case, as there are not
social interactions throughout the experiences.
All users were asked simply to play each game, and then to answer a series of
questions about it. The order in which questionnaires were offered to the subjects
was determined by the need of passing the SSQ first, while the potential effects
of SS were still active. SUS and TPI came after it, in that order.

2.2 Demography

Our experimental group was a set of 12 subjects that went through both experi-
ences, so as to be able to compare their reactions to them. Half of them did the
two experiments in one order; while the other half did them inversely. Besides,
none of the subjects who took part in the experiment knew about the contents
of the second experience before finishing the first one.
To participate in this experiment, all subjects had to sign-up by answering
an email which retrieved the following pieces of information:

– Name.
– Age.
– Gender.
– Education.
– VR device familiarity.
– VR devices present at home.
– Video game familiarity.
Among the subjects, 11 (91.67 %) were men, and 1 (8.33 %) was a woman.
Their ages ranged from 18 to 26 years old. About their background, 8 (66.67 %)
had an education in Computer Science, while 3 (25 %) studied a degree related
to Video Game Development and 1 (8.33 %) had studied Graphic Design. About
VR, 4 (33.33 %) of them did not have any familiarity with VR devices, and had
never tested a HMD, while the rest had used at least one VR system; 8 (66.67
%) did not own any VR device, and 4 (33.33 %) had a Google Cardboard at
home. All of them said to be familiar with video games, and declared themselves
frequent players.

3 Results Analysis
The obtained results point to a strong correlation between the action of walking
and an increment in presence. Both SUS and TPI tests presented a p-value much
lower than 0.05, and both offered very similar normalized results, as can be seen
in tables 1 and 2. This is relevant if we consider that in-game movement did
not imply real-world motion to any extent. The lack of correspondence between
in-game and real-life motion does not seem to affect how presence rises when the
user is able to walk (or fly, as we could call it) in VR.
Figures 4 and 5 show subject numbers in the abscissa axis, and normalized
levels of presence achieved in the ordinate axis. Figure 4 presents a more irregular
shape than figure 5, but this seems to be caused by the different number of
questions in both tests. The first one had only 6, whereas the version of the TPI
that we used had 18, which allowed for a less polarized mean. The correlation
is, in both cases, high enough to allow us to think the results of these tests are
consistent.

Table 1. Normalized SUS test results (One-way ANOVA).

Walking Yes No

N 12 12

SUM (X) 8.285714 7

Mean 0.690476 0.583333

Variance 0.010101 0.003556

p-value 0.00413
Table 2. Normalized TPI test results (One-way ANOVA).

Walking Yes No

N 12 12

SUM (X) 7.759259 6.092593

Mean 0.646605 0.507716

Variance 0.007386 0.004502

p-value < 0.0001

Fig. 4. Normalized results of the SUS presence test for both walking (W ) and not
walking (N W ) experiences.
Fig. 5. Normalized results of the TPI test for both walking (W ) and not walking (N W )
experiences.

The SSQ, however, offered results which, in spite of varying between both
prototypes, did not present a significant correlation (with a p-value of approx-
imately 0.12) when processed with a one-way ANOVA, as can be seen in table
3. Walking did not seem to notoriously affect SS. Only subject number 7 (see
figure 6) presented a clear increment in SS while walking. However, this subject
had defined himself, before taking part in our experiment, as especially prone to
nausea and motion sickness.

Table 3. Normalized SSQ test results (One-way ANOVA)

Walking Yes No

N 12 12

SUM (X) 2.416667 1.0625

Mean 0.201389 0.088542

Variance 0.045718 0.006323

p-value 0.120654
Fig. 6. Normalized results of the Simulator Sickness Questionnaire for both walking
(W ) and not walking (N W ) experiences.

4 Discussion

Notwithstanding the strong relation between the SUS and TPI tests results, the
reduced group of subjects we had to work with leads us to think the obtained
data can be considered reliable just to a certain extent, and should be broadened
in the future. Besides, most voluntaries who took part in the experiment were
men, which means there could exist a genre bias in the data.
Though the results of both SUS and TPI questionnaires were consistent, it
would also have helped to count on a strongest numeric measure, such as the
data from an electrocardiogram (ECG). This was not possible due to limited
time and resources, but would be desirable in future research.
SS was not measured over time, as we wanted to check if only the mere act
of walking increased it. However, we do not know the effects a higher level of SS
granted by a longer experience may have when comparing the results of both
prototypes. Besides, the order in which both experiences were taken could have
influenced the level of SS achieved.
Moreover, all of our subjects said to have a notable experience in playing
video games, and some of them had already tested VR devices, which means
they could be more used to virtual motion, and therefore more resistant to
simulator sickness in short and focused experiences as those of our experiment.
5 Conclusions
In conclusion, the results of this experiment negate the null hypothesis (H0 ) and
enforce our alternative (H1 ): there is a significant difference between walkable
and non-walkable first person VR games when comparing the independent vari-
able to presence. Being able to move longer distances than what the tracking
capacities of the HMD allow, even if there are not real body movements associ-
ated with the act of walking, generates a clear increment in the level of presence
achieved by the player.
On the other hand, judging solely by the results of this experiment, there does
not seem to be a clear correlation between the act of walking and SS. Though
this may have been caused by the lack of a broader set of subjects, the stable
nature of the results obtained from SUS and TPI presence tests with the same,
narrow group, leads us to think there is actually a lack of SS generation in our
game prototypes. This opens a new line of investigation, as it would be useful to
know if it is related to the technical properties of current VR devices, the design
principles assumed by the authors, or to something else.
Current game developers could benefit from this combination of results. Most
VR experiences lack the possibility to walk the environment due to a fear of
SS; however, brief and simple experiences like the ones we designed for this
experiment have shown no clear correlation between walking and SS. This paves
the way to more walkable experiences in the future, as we think player movement
is a fundamental asset in modern video games. It allows for complex interaction
while increasing presence, and differentiates video games from static experiences
which also use a HMD, like VR video.

Acknowledgements We would like to thank all volunteers who took part in
our experiment for giving us their time and effort.
This research was supported through EC funding for the project WHIM
611560 by FP7, the ICT theme, and the Future Emerging Technologies (FET)
programme.

References
1. Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator Sickness
Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The In-
ternational Journal of Aviation Psychology 3(3), 203–220 (1993)
2. Rolnick, A., Lubow, R.E.: Why is the Driver Rarely Motion Sick? The Role of
Controllability in Motion Sickness. Ergonomics 34(7), 867–879 (1991)
3. Ehrlich, J.A., Singer, M.J.: Simulator Sickness in Stereoscopic vs. Monoscopic Hel-
met Mounted Displays. In: Proceedings of the Human Factors and Ergonomics So-
ciety Annual Meeting. Human Factors and Ergonomics Society, Philadelphia (1996)
4. Usoh, M., Arthur, K., Whitton, M.C., Bastos, R., Steed, A., Slater, M., Brooks,
F.P.: Walking, Walking-in-place, Flying, in Virtual Environments. In: Proceedings
of the 26th Annual Conference on Computer Graphics and Interactive Techniques
(SIGGRAPH), pp. 359–364. ACM Press, New York (1999)
5. Sheridan, T.B.: Musings on Telepresence and Virtual Presence. Presence: Teleoper-
ators and Virtual Environments 1(1), 120–126 (1992)
6. Barfield, W., Weghorst, S.: The Sense of Presence Within Virtual Environments: A
Conceptual Framework. Advances in Human Factors Ergonomics. 19, 699 (1993)
7. Mestre, D., Fuchs, P., Berthoz, A., Vercher, J.L.: Le Traité de La Réalité Virtuelle.
Presses des Mines, Paris (2006)
8. Biocca, F., Harms, C., Burgoon, J.K.: Toward a More Robust Theory and Measure
of Social Presence: Review and Suggested Criteria. Presence: Teleoperators and
Virtual Environments 12(5), 456–480 (2003)
9. Lombard, M., Ditton, T., Weinstein, L.: Measuring Presence: The Temple Presence
Inventory. In: Proceedings of the 12th Annual International Workshop on Presence,
pp. 1–15. Institute for Creative Technologies, Los Angeles (2009)
10. Hettinger, L.J., Berbaum, K.S., Kennedy, R.S., Dunlap, W.P., Nolan, M.D.: Vec-
tion and Simulator Sickness. Military Psychology 2(3), 171–181 (1990)
11. So, R.H.Y., Lo, W.T., Ho, A.T.K.: Effects of Navigation Speed on Motion Sick-
ness Caused by an Immersive Virtual Environment. Human Factors 43(3), 452–461
(2001)
12. Merhi, O., Thomas, A.: Motion Sickness, Video Games, and Head-Mounted Dis-
plays. In: Proceedings of the Human Factors and Ergonomics Society Annual Meet-
ing, pp. 2618–2622. Human Factors and Ergonomics Society, New Orleans (2004)
13. Serge, S.R., Moss, J.D.: Simulator Sickness and the Oculus Rift: A First Look. In:
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp.
761–765. Human Factors and Ergonomics Society, Los Angeles (2015)
14. Slater, M., Usoh, M., Steed, A.: Taking Steps: The Influence of a Walking Technique
on Presence in Virtual Reality. ACM Transactions on Computer-Human Interaction
2(3), 201–219 (1995)
15. Lorenz, M., Busch, M., Rentzos, L., Tscheligi, M., Klimant, P., Frohlich, P.: I’m
There! The Influence of Virtual Reality and Mixed Reality Environments Combined
with Two Different Navigation Methods on Presence. In: Proceedings of IEEE 2015
Virtual Reality, pp. 223–224. IEEE, Arles (2015)
16. Nielsen, M.R., Toft, C., Nilsson, N.C., Nordahl, R., Serafin, S.: Evaluating Two
Alternative Walking in Place Interfaces for Virtual Reality Gaming. In: Proceedings
of IEEE 2016 Virtual Reality, pp. 299–300. IEEE, Greenville (2016)
17. Sherman, W., Craig, A.: Understanding Virtual Reality: Interface, Application,
and Design. Elsevier Science, San Francisco (2003)
18. Slater, M., Usoh, M., Steed, A.: Depth of Presence in Virtual Environments. Pres-
ence: Teleoperators and Virtual Environments 3, 130–144 (1994)