Over the last years, Augmented and Virtual Reality technology became more immersive. However, when users immerse themselves in these digital realities, they detach from their real-world environments. This detachment creates con icts that are problematic in public spaces such as planes but also private settings. Consequently, on the one hand, the detaching from the world creates an immerse experience for the user, and on the other hand, this creates a social con ict with bystanders. With this work, we highlight and categorize social con icts caused by using immersive digital realities. We rst present di erent social se ings in which social con icts arise and then provide an overview of investigated scenarios. Finally, we present research opportunities that help to address social con icts between immersed users and bystanders.
• Human-centered computing tual reality.
WHAT IS CROSS-REALITY INTERACTION?
We interpret cross-reality interaction as a cause-e ect
relationship that involves at least two di erent manifestations of the
reality–virtuality continuum [
In the past few years, there has been great progress in the areas of AR and VR. In particular, technological developments and the availability of head-mounted displays (HMDs) for end-users have made great strides. This enables users both at home and on the go to superimpose digital information in the form of graphic representations over the world or even to immerse themselves completely in virtual realities. Over the past years, technical availability has made it possible for researchers to understand, for the rst time, how people can use AR and VR. The main focus here was on how users can better interact with digital information and how information can be better presented to create a better user experience (UX) and better immersion.
The various realities such as AR, augmented virtuality (AV),
and VR can be mapped on a continuum – the reality-virtuality
continuum [
Clearly, the real world never stops surrounding the user and
moves at its own pace. People around the user could in uence the
user’s experience in a virtually enhanced or purely virtual
environment [
Today, the question arises of how the real world can be
appropriately integrated into alternative realities to raise the feeling of
immersion for the user on the one hand, but on the other hand,
not to lose complete contact with the outside world. Today there
are rst attempts to explore how persons around the user can
be directly integrated into the interaction or even enable a
conversation between di erent realities [
Therefore, the focus should be on the technical realization of
alternative realities, as seen up to now, and on the social component
of disconnecting the user [
In the following, we rst discuss di erent social settings in which the use of alternative realities have been envisioned. In detail, we discuss the di erent characteristics from which we, in a second step, derive three di erent relationships between users and bystanders. 3
To derive the di erent classes of social settings and corresponding characteristics, we crawled relevant literature using Google Scholar and focused on two online libraries in particular (ACM Digital Library and IEEE Xplore), considering work published in the last ve years in topic-related conferences (CHI, UIST, VRST, IEEE VR, IEEE ISMAR). To nd relevant papers, we searched for one of the following keywords and their abbreviations in title or abstract: “Mixed Reality,” “Augmented Reality,” “Virtual Reality,” and “Reality-Virtuality Continuum.” We collected all relevant papers in a spreadsheet and iteratively added information dimensions classifying the publications. For this paper, we identi ed di erent contexts of users and their interaction between them. In the following, we introduce these settings and scenarios in more detail. 4
Users can experience digital realities in di erent social settings.
While VR often is experienced for entertainment purposes in
private settings alone or with friends, the technology becomes more
relevant for professional contexts. Furthermore, previous research
suggests that public settings can become relevant in the future as
well [
Mostly for entertainment purposes, users experience digital
realities in private settings. The number of bystanders is relatively
low, and users and their activities are known to the bystanders. An
example is a group of friends that takes turns on a VR headset. For
the additional involvement of bystanders in these gaming
activities, di erent collaborative games exist [
In work or school environments, users and bystanders likely know
each other as they are mostly access controlled so that only
authorized people can enter and visitors often stand out (e.g. visitor
badge). Thus, the users’ experience is not necessarily of interest for
bystanders and the hurdle to engage in an interaction lower due to
the connection they share by being allowed in the space. Moreover,
education often requires collaboration between subjects [
In recent years, public settings became more relevant as a social
setting for experiencing digital realities. Compared to private or
semi-private settings, users and bystanders likely do not know each
other. In detail, we can divide the settings in public (e.g. streets, and
parks) and semi-public setting (e.g., co eehouses, and shops). Usage
of, for example, VR in public spaces can result in the unperceived
presence of others or a less immersive experience due to external
interruptions [
In previous work, di erent scenarios that involve users on the reality-virtuality continuum and their bystanders have been investigated. When studying these scenarios, it becomes visible that often the relationship between users and their bystander di ers. Here, relationship refers to the relative position of users and bystanders on the continuum. In the following, we grouped the research scenarios from previous work based on the described relationship between users and bystanders (see Figure 1). 5.1
Isolated experiences exploit the circumstances that users detach from their real-world environments. While this seems to contradict our work’s motivation, it still can make sense in certain situations. For example, in a noisy o ce environment with lots of visual elements causing distraction from work, a virtual environment may
Bystander
User
User is somewhere on the continuum
Isolated Experiences
Bystander Inclusion
Collaboration
far away, no transition
uni-directional transition
bi-directional transition
be a good idea to isolate from these negative in uences [
In many situations, users manifesting on certain spots on the
realityvirtuality continuum would like to include bystanders in their
experience, or bystanders would like to participate in the user’s
experience. Here, users’ experience on the continuum is at focus, and
bystanders get included in various ways. For example, bystanders
want to see what a user in VR experiences, and therefore,
previous work suggests using a CAVE [
With collaboration, we refer to scenarios that involve achieving a
shared goal. Compared to bystander inclusion, this means that both
users and bystanders alter their experience and come closer to each
other on the continuum, see Figure 1. An example would be a
bystander and user playing a game together where the bystander can
help the user reach their objective (e.g., to solve a maze puzzle [
In the following, we highlight future research opportunities derived from the introduced social settings and classes of relationships between users and their bystanders.
Expanding to Public and Work Places. First-generation VR and AR devices are designed for single-user experiences in private contexts. However, designing devices that foster interaction between users and bystanders can facilitate communication and interplay in various contexts.
Transitional Experiences. New technology might allow easy transitions along the continuum (e.g. video see-through HMDs). Depending on the scenario, it makes sense to create experiences that, for example, allow the user to transit from reality into VR gradually. Thereby users and bystanders can interact on di erent levels depending on how far they must transit into the reality of each other. Based on the bene ts it provides, we expect transitional interfaces to become more relevant in the future.
Visualization Helps Understanding. Researchers and developers
might face challenges while investigating or developing MR
scenarios and therefore need insights into the sessions of their users.
Tools and Frameworks that help us describe or analyze these
scenarios [
In this paper, we present di erent scenarios for the use of alternative realities. By categorizing them, we found three di erent social contexts: private, work, and public in which alternative realities can be used. From these, we derive characteristics of di erent relationships between the user and bystanders. Here, we argue that three types of interactions are important to explore in the future to enable easy switching between manifestations as this ultimately allows for a satisfying user-bystander interaction.