-
INTRODUCTION
Virtual reality (VR) is a system of principles, methods and
techniques intended to give the player a more realistic way
of perceiving and experiencing the surroundings by
reproducing via modelling and simulation an artificial,
three-dimensional space. Especially speaking of games, VR
is now on the rise, benefitting from price cuts on VR
GHItaly18: 2nd Workshop on Games-Human Interaction, May 29th, 2018,
Castiglione della Pescaia, Grosseto (Italy)
Copyright © 2018 for the individual papers by the papers' authors. Copying
permitted for private and academic purposes. This volume is published and
copyrighted by its editors.
headsets and some must-have contents (revenue up from
1.8 in 2016, to 2.2B in 2017, and it is estimated to grow to
4.5B in 2018 [
23
]). The main profile devices are
PlayStation VR, Oculus Rift and HTC Vive.
What changes – and what is also challenged – in
comparison with the traditional way of playing video
games, is the player sense of interaction and immersion.
Experiencing VR games, players interact with their
surroundings (artificial environment) through senses and
limbs, and the information flow is bidirectional: through
senses, it goes from the environment to players; through
limbs, it goes backward. Conceptually, this way of
interacting with the virtual world is more realistic than the
mouse-and-keyboard (for pc games) or pad/joystick (for
consoles and arcades games) mediated way: indeed, players
use motor schemes that have been previously acquired in
the naturalistic setting of the physical environment to
perform everyday actions. On the contrary, traditional ways
of mediating interaction involve acquiring motor patterns
that are only loosely related with the resulting in-game
meaning (as pressing the “w” key for moving the avatar
forward) and often suffer from cross-modality interferences
(as using buttons on the left to have the avatar perform
actions on the right [
13
]). However, even by taking realism
into account, there are recurrent perceptual UX problems.
Ever since their conception in the 60’s, VR via
headmounted systems has been involved firstly in visual and
secondary auditory senses [
24
], revealing a quite persistent
lack of coverage on other senses. In particular, the absence
of haptic stimuli and of a physicality of the virtual world
impacts on VR realism and immersion [
15,16
]. Although
the use of wearable technology, physical props, and the
possibility of having players walking while navigating VR
environments (e.g.: [
5
]), the problem is still far from an
easy solution. In fact, such approaches cannot fully nor
smoothly recreate the experience of touching objects.
That said, we are in front of haptic and proprioception
issues, where the first refers to the sensory domain of touch,
that have already been mentioned, while the second relates
to those stimuli that are produced and perceived because of
the position of our body in the environment/space and its
locomotion. Indeed, our brain continuously checks the
proprioceptive and visual consequences of motor
commands (e.g.: [
2
]). In some cases, a mismatch occurs,
such as when patients with an amputated limb try to move
it. The majority of these patients experience the vivid
presence of a “phantom limb” associated with extreme pain.
Crucially, by restoring the visual feedback, the pain is also
instantly reduced [
19
], suggesting that the sensorimotor
mismatch may be interpreted by the brain as pain [
18
].
Similarly, when looking at virtual reality, the matching
between what is being perceived via visual and non-visual
channels may evoke bizarre experiences. One such
experience is known as the cybersickness or VR sickness, a
feeling that closely resembles motion sickness.
Interestingly, while for motion sickness vestibular
stimulation is necessary, with visual stimulation being a
possible contributing factor, in VR sickness only the visual
stimulation occurs [
11
]. In line with the phantom limb
example, this suggests that the mismatch between the
information flowing through the visual channel and that
elaborated through another channel (in this case,
proprioceptive information) is the triggering factor. This
issue is still being dealt with.
In developing the first VR contents, multiple UX problems
popped out concerning UI. To lessen its impact, answering
the necessity of delivering complex information, a design
solution can be embedding information into the
environment, following the idea that the best interface is no
interface [
10
] – trying not to expose it, and not even to refer
to the way it is used in two dimensional games. A
threedimensional environment offers different affordances than a
two-dimensional one, including those that refer to our
innate perception-action patterns (since we live in a three
dimensional space) [
4,14,17
]. Taking advantage of this, VR
designers could approach UX problems simply demanding
the interaction learning to already known patterns.
In the light of this reasoning, our research question regards
how players grasp information from the VR environment,
and hence understand how to interact with the game
elements. Sense-making issues (as wayfinding or
interaction with the environment), complications with
actual navigation in real spaces, and interaction with the UI
(techniques from handheld to full-body), are just some of
the main problems. Recognizing their existence, as well as
the inconsistencies and discrepancies that tag along, we
propose to go through a case study for ruminating how
hands-on experiences can serve to unpack some recurrent
problem and overcome frequent usability issue. Then, based
on our observations we discuss the game affordances as
usability, discoverability, feedback, and what kind of
information the UI convey to those who play.
METHODOLOGY
From a methodological perspective, the research conducted
on the artifact is based on qualitative research on Virtew’s
Run of Mydan (2017) as our case study. The analysis
follows how the investigation of the game experience
according to user-centered approach resulted into
implications and implementations of the VR game,
throughout its design process (from the first demo to the
current version). We used variable methods: informal
interviews, direct observation, participant observation via
moderate participation in the design phase (impacting on
the implementation of the game) and playing with the
current version, collective discussions, and self-analysis. As
a matter of fact, one of the authors of this paper has been
the first playtester of the early access release, the first
public version of the game. The benefits of conducting
observation and interacting over an extend span of time
(about 1 year, at the time of writing) lies in the collection of
those reasonings that are not influenced by a posteriori fact,
but rather progressive improvements, for example how
discrepancies have been resolved.
According to Howell [
7
], we conducted the research: 1)
Establishing encounters with the developer team before
starting the study; 2) In the field, entering the community
since the game was in a demo phase; 3) Recording
observations and data via (a) field notes and (b)
semistructured interviews, being aware of possible subjective
biases and prejudices [
1,6,21
]; 4) Analyzing data by (a)
thematic and (b) narrative analysis.
RESULTS AND DISCUSSION
Run of Mydan is a first person single-player and multiplayer
flying VR shooter. The two game modes differ not only in
the number of players involved but also in the navigation
mechanics. Thus, we go from describing common features
to vetting into the main differences. Finally, we analyse the
menu interface and its UI, as an element deliberately
designed as separate from the game environment. The game
can be played either with the HTC Vive VR system or with
the Oculus Rift one, alongside a pair of controllers and
turrets to track the player. In the following, we embrace a
twofold perspective: the one of the player who experiences
the game, and the one of the developers who took specific
decisions in terms of interaction, aesthetics and so on.
General features
The entire gameplay is based on a singular assumption: the
player’s avatar and the enemies can be damaged until they
die. As such, the main goal of the player is to survive and
kill enemies. In Run of Mydan players can perform the
following actions: moving, attacking (with the currently
selected weapon) or blocking (by generating and using a
shield). Depending on the game mode, these actions have
different effects in terms of gameplay, since can affect
different game elements.
Acknowledging Lee and colleagues’ research on avatars’
spatial navigation of virtual environments [
12
], and relying
on the concept of peripersonal space alongside the
extrapersonal and personal ones, we draw our attention on
its implication in terms of interactions between avatar,
player, and environment. According to the authors,
navigating the space with their avatar, players tend to
ignore visual stimuli located “outside of the avatars’
peripersonal spaces in which the avatars cannot interact,
thereby irrelevant informational space” [
12
]. To increase
immersivity, the game uses no head-up display (HUD) to
convey information, but it is either embeded it in the
environment or embodied it in the three-dimensional
elements themselves. Following the "zero interface"
principle of the VR medium, the UI and its elements have
been made contextual to the environment itself, therefore
fluidly merged with the it or with the avatar, as explained
below. In terms of affordances, embedded or embodied UIs
have an impact on usability: they do not communicate their
presence to players, but players get to intuitively know
about their existence and function. Run of Mydan UI mainly
relies on discoverability, as the degree of ease with which
the player discovers the elements and features of the game
system as far as they are first encountered, and on the game
ability to contextually and timely provide understandable
feedback of what is going on. For example, if the flying
modes are grounded on quite intuitive and graspable
interactions, especially because they rely on the well-known
Ironman and Superman imaginary with equally clear
affordances, the use of weapons is otherwise based on a
different and “less natural” reasoning and affordances that
require to start a learning process – as discussed below.
Multiplayer mode(s)
The multiplayer mode features a 3D, gravity free
environment in which player’s avatar floats and moves.
Such environment is a finished world, and its extension is
signaled by an invisible wall that appears once encountered:
when a player reaches such border, a visual effect that
could be described as a disintegrating net of floating
orange-stroked triangles appears. If the avatar does not
touch such fringe, nothing signals its existence neither its
proximity: matter-of-factly players are allowed to see the
rest of the world through such net, but they cannot
reach/explore it. This brings to a situation that provides
situated information just when it is needed, and in the
meanwhile it is providing enhancing player’s feeling of
being in a full world rather than in a mere portion.
Then, focusing on in-game locomotion, players can perform
360 movements to navigate the environment. Movements
can be performed in two modalities, selected in the settings.
Selecting the Ironman mode, for moving players need to
point the controllers at the direction that they want to be
pushed from (i.e., pointing in front of myself to be
propelled backwards); in the Superman mode, players point
at the direction they want to be pulled in (i.e., pointing in
front of myself to go forward). In the early design phases,
the Ironman mode was the only modality. However, tests
ran users showed that such modality of fly was not easily
grasped and handled by all those who played it. In answer
to such issue, the “superman” mode has been introduced.
That said, we switch from the interaction with the
environment to the one with other online players. Players
can enter several configurations (1 on 1 deathmatch, team
vs team deathmatch, dominion, etc), however while such
configurations are irrelevant to the current rumination, what
concerns us is that in the multiplayer mode, the interaction
with others consists of fights and occurs just through
weapons – not barehanded harm can be provided. When
damage is received, the avatar health decreases
proportionally to the hits; recovery starts few seconds after
the last hit. When the avatar dies, it respawns with full
health, and a point is given to the opposing enemy/team.
Single-player mode
The single-player mode differs from the multiplayer mode
mainly because it does not allow “free flight” movement:
the avatar is indeed enchained to a floating platform. As
such, the movement system previously seen for the
multiplayer mode only moves the platform slightly on the
left or right or makes it accelerate or slow down; the
platform follows an invisible path (a sort of rail) over which
the player has little control. However, the player can move
across the two-dimensional plane of the platform,
corresponding to about 2.5x2.5 meters. By walking around
the tracked area, the movement, centered on the avatar, is
recreated in the VR space. Because of some chains, which
are a simple, narrative-based and very effective visual
stratagem to communicate the game affordances, the player
is led to know that the platform is the only walkable space.
As such, the player is informed that by being on the
platform s/he can move as if s/he was affected by gravity
(even if the platform is not, since it moves floating along an
invisible rail). This aspect alongside the properties of some
environmental elements and enemies are a source of
ambiguity, since some elements are inexplicably subject to
gravity or not. This is certainly an unresolved issue that
produces a cognitive dissonance, due to the environmental
physics; however, the chain expedient provides a diegetic
reason to the player who cognitively matches what s/he sees
to what s/he understands of the world: in-game position and
locomotion, as well as the feeling of gravity itself.
Focusing on the interaction with the world, the player
simultaneously needs to 1) defend from environmental
perils and assaults/offences by standard enemies as well as
giant end-level bosses, through the use of shields and flying
skills, and 2) attack such enemies or dangerous
environmental elements using the available weapons. In
certain occasions, players also need to use an appropriate
combination of attack, defense and movement. In this
mode, enemies do not recover from damage, but the
player’s avatar does. Then, when the avatar dies, it
respawns in a previous checkpoint, full health. However,
checkpoints presence and position are not communicated,
and players are not aware of their existence, until the avatar
dies and respawns. This choice undermines the player’
sense of consistency of actions, since s/he is not informed
about a mechanic that is available in the game.
Menu UI and player UX
The menu on which the player selects the game mode,
configures settings, customizes avatars and so on is located
in a separate space from the one in which the the gameplay
occurs. By entering this space (fig. 1), the player leaves the
game environment and a new nightly environment appears,
with a menu consisting of a set of three-dimensional
buttons and writings spanning 180° of the visual field and
being centered on the player (fig. 1). The menu appears in
the peripersonal space [
12
], so that buttons are reachable by
extending the hand with no need for additional movements
such as steps. However, by not having physicality nor
haptic feedback, the virtual hand can use the same space
occupied by the button. The interaction with the buttons
requires to have the virtual index finger “inside” the virtual
button, where the status of “being pressed” is signaled by a
rather unexpected reaction: the button becomes partially
transparent. This interaction results counterintuitive and
unrealistic, and produces a certain ambiguity.
Moreover, to perform every choice here mediated by these
three-dimensional buttons (fig. 2), the player has to push
the trigger on the controller with the index finger. In so
doing, we obtain a sensorimotor mismatch: even if the
player is pressing the button on the controller, the virtual
hand does not move, and the button does not look as “being
pressed”. According to our experience, this mismatch is
perceived as a friction with the interaction, resulting into a
fracture of the immersion [
3,15
].
UI embedded in the gameplay
The UI has been designed and implemented to let the
player’s experience be as immersive as possible by
facilitating the path from engagement to engrossment and
total immersion [
3
]. In fact, all the information regarding
in-game meaningful statuses are embedded with
appropriately diegetic representations [
8
]. Recognizing the
central role of immersion, in the following we expand some
of the concepts that contribute to it (and its maintenance).
The first point regards information overload, possibly due
to an excessive amount of informative elements that could
affect in a negative way the player decision-making
process. In fact, dealing with a virtual representation, we
undergo a peculiar contradiction that stands “between our
impression of virtually unlimited perceptual content and the
existence of severe attentional limitations” [
22
]. To reduce
the information overload and smoot as much as possible the
play experience, the game developers firstly developed an
effective navigation system for both the single and
multiplayer modes, fitting the game general coherence and
timely providing answers to the task to accomplish in the
game space. Then the 3D environment has been designed
for balancing (and ameliorating) how information is
provided. Aptly, the way in which the game system shows
visual information should regard only the items needed to
accomplish a task (at hand), coherently and timely, instead
of forming an extended, detailed representation of the full
variety of objects in the surrounding environment [
20
]. The
UI should enable to handle multiple and dynamic
information, also exploiting our spatial cognition
capabilities. Just the player’s damage information is
traditionally provided, responding to the very habits of
players: when the player is repeatedly hit and damaged, the
vision of the world turns red with a contrast that gets
stronger the more serious the damage is. When life is
recovered the colour returns to normal and the life bar on
the forearm fills up. That said, during the gameplay, the
player is provided with some information that rather than
being overlaid on the screen, are wisely situated into the
virtual space. This allows players to bypass them by
“walking across the information themselves”, providing the
conceptual, and cognitive, implication that they are part of
the VR world. According to our direct experience, the way
in which the UI has been embedded in the avatar’s body as
well as in the environment itself contributes to increase
immersivity rather than producing a sort of detachment due
overlay of information. Indeed, recognizing the
potentialities of the avatar’s body in being a diegetic
element that can be used for providing supplementary
information, basic information as health and weapon
readiness states are displayed on the arm of the avatar (in
line with how it has been done in Dead Space 2, a solution
already discussed in [9], [
8
] and [
25
]).
On the contrary, more advanced information is conveyed by
means of several intuitive affordances. For example, the
avatar status is represented on the arm and the line of fire
can be inferred by aligning a set of three-dimensional
floating triangles resembling the behaviours of aiming with
a shotgun (fig. 3). From an UX point of view, these UI
design choices result as consistent as meaningful, in
addition to be diegetic. They allow players to behave in a
natural way, and simply check their arm for information
about their health and weapon recharge states, or point their
weapon using triangles to aim, rather than adding layers of
information in the environment – as a non-diegetic life bar
or aiming cross in the middle of the field of view.
Like navigation, also the interaction with the environment
occurs through movements of the upper part of the body:
pushing the controller buttons, or orienting them as an
extension of the player’ arms in the space to fly or shoot,
avoiding complications due to composite actions. The
weapon selection currently involves the use of a dedicated
button on the controller that can be pressed with each
thumb to change the selected weapon on the corresponding
virtual hand. The selected weapon is communicated by an
icon on the back of the corresponding hand of the avatar,
becoming in turn an embedded information. While no
visual feedback in the VR space corresponds to the thumb
movement, this design solution solves a number of issues
that were detected during the playtest sections. The first
iteration to select the weapon was a swipe on the controller
pad, that provided no feedback, but the appearance of the
selected weapon in the avatar hand in a following moment
than the selection itself. The second iteration involved the
appearance of a semi-transparent fan-like panel presenting
the possible choices on the back of the hand on which the
weapon was being selected. In this case, the player had to
reach the back of that hand with the opposing hand, and act
on it with a complex manipulation involving a spline
generated on the wrist that needed to be connected with the
weapon icon. Then, after performing such a complex
manipulation with the opposing hand, the weapon was
selected for the hand that was not manipulating. Playtesters
reported this solution as very counterintuitive. The third
iteration of the weapon-choice interface involved less
manipulation by the opposing hand. The panel of choices
was placed on the back of the shield: in so doing, to change
the weapon on one hand the player had to press the
controller shield button (a thumb press) with that hand.
However, the problem remained, as the hand performing
the selection still wasn’t the one being affected by it. The
second and third iterations did not solve the issue, since
they produced expectations later disappointed. The last and
current iteration simplifies the problem by limiting the
overall manipulation. Reducing the number of choices to a
maximum of three weapons to be selected among all those
present in the game while in the the menu space, the
cognitive load is relatively low and the manipulation
happening directly on the interested hand keeps the
selection intuitive. This iteration certainly took into
consideration the principle of discoverability, while the
second and third ones could not be described as
userfriendly, even if we recognise the attempt to maintain a
diegetic coherence. In Run of Mydan, the first playtests
highlighted troublesome interactions, showing a persistent
discrepancy between perceived affordances and unexpected
results. To obtain a coherence between perceived and real
affordances, the developers modified certain interactions
(as the weapon selection one) and introduced specific
feedback that are consistent throughout the gameplay [4],
but also meaningful in narrative terms.
Finally, to convey further information they introduced the
haptic feedback. Controllers vibrate when a player uses a
weapon to shoot, but also when some weapons are ready or,
conversely, when some other weapons are fully discharged.
Additionally, vibration occurs also in one instance in which
the shield is broken. In our experience, while vibration feels
like a nice feature of these few actions, the coherence with
which it is implemented conveys little meaning.
CONCLUSION
We analysed the use of UI in a three dimensional, virtual
environment in the game Run of Mydan, in which the
developer’s attempt was to adopt the diegetic approach to
facilitate both players’ immersion and their understanding
of the game world. While the resulting product reaches this
goal in many aspects, some issues are still left to be
satisfyingly solved. In fact, the diegetic informative
elements embedded in the environment/body are coherent
with both the design and psychological guidelines that
suggest to use intuitive patterns and affordances. These, in
turn, trigger already known motor patterns and facilitate the
learning process, while rendering the gameplay more
intuitive and the immersion deeper. Along with these
benefits, the sensorimotor matching and the cognitive
match between the bodily feelings and the visual
stimulation are taken into account and exploited with
diegetic solutions. The result is a game in which most of the
information is conveyed in an intuitive and straightforward
manner, and players can quickly grasp it and effortlessly
interact with the world. However, a number of issues
remain open: non-diegetic elements present various degrees
of interference with total immersion. A great deal of work
has been devoted to the weapon-selection interface, but the
“non-diegetic button press” solution still leaves the
cognitive load low while representing the current status in a
diegetic way. On the opposite, the menu interface is by
definition non-diegetic, in that the game needs to be paused
to access it. However, in this space the UI is represented in
a three dimensional space with convenient affordances. In a
sense, while the menu space is separate from the game
space, both have their own diegetic, but incompatible,
meanings. Unfortunately, the interaction with the menu
breaks the immersion, forcing the player to use body
movements with no in-game reconstruction and based on
the counterintuitive assumption that two objects may
occupy the same spatial position. This analysis suggests
that the diegetic, no interface is the best interface approach
[
10
] is useful in providing a barrier-less path to total
immersion in VR. However, while single case studies are
useful in exploring state-of-the-art solutions, whereas
evidence-based directions for UI and UX design are
required the topic needs further quantitative-methods
explorations.
ACKNOWLEDGMENTS
We thank Virtew and its team for their time and valuable
contribution, for providing us with materials, screenshots
and access to the game.