=Paper=
{{Paper
|id=Vol-3704/paper3
|storemode=property
|title=A Review of Virtual Display Layouts in HMD-based Extended Reality
|pdfUrl=https://ceur-ws.org/Vol-3704/paper3.pdf
|volume=Vol-3704
|authors=Parisa Daeijavad,Frank Maurer
|dblpUrl=https://dblp.org/rec/conf/realxr/DaeijavadM24
}}
==A Review of Virtual Display Layouts in HMD-based Extended Reality==
https://ceur-ws.org/Vol-3704/paper3.pdf
Review of Virtual Display Layouts in HMD-based
Extended Reality
Parisa Daeijavad1,* , Frank Maurer1
1
Department of Computer Science, University of Calgary, Calgary, Alberta, Canada
Abstract
Although industry experts and designers provide guidelines for designing multiple display layouts in
extended reality (XR), there is a lack of empirical evidence to support these recommendations. In our
study, we review papers that have conducted user research and evaluated display layouts in XR to provide
a more structured view of evidence-based guidelines in different applications. Our investigation extends
beyond exploring the existing studies in this domain, types of XR environments, and display layouts,
providing detailed insights on how specific layout choices impact users’ experience (UX) measures
in XR environments. Our research paper provides design suggestions for developers, designers, and
researchers, offering a summary of design guidelines for various display layouts in XR, supported by
existing empirical evidence.
Keywords
Extended Reality, Spatial Arrangements, Multiple displays, Human Computer Interaction
1. Introduction
This review explores multiple display layouts in extended reality (XR). The identified publications
address questions and problems related to the usability, design, and implications of virtual
reality (VR), augmented reality (AR) and mixed reality (MR) display layouts for various tasks,
and the impact of layout configurations on the user’s performance. In this review paper, we
explore the existing studies in this domain, their application, types of XR environments, and
display layouts. In addition, to find empirical evidence on how to lay out virtual displays in XR,
we report on the empirical outcomes derived from existing user studies in the literature.
Figure 1: Examples of XR multiple display layouts in [1, 2, 3]. A) Multiple displays in a shape of sphere
[1]. B) Floating planes around the user [3]. C) Vertical and Horizontal arrangements in [2].
RealXR: Prototyping and Developing Real-World Applications for Extended Reality, June 4, 2024, Arenzano (Genoa), Italy
*
Corresponding author.
$ parisa.daeijavad@ucalgary.ca (P. Daeijavad); fmaurer@ucalgary.ca (F. Maurer)
0000-0002-9892-9904 (P. Daeijavad); 0000-0002-0240-715X (F. Maurer)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Although industry experts and designers provide guidelines for designing multiple display
layouts in XR, there is a lack of empirical evidence to support these recommendations [4]. In
our study, empirical evidence refers to data and observations from experiments and user studies.
We collect studies that highlight evidence-based guidelines and are moving away from relying
on expert opinions. The study methodologies used in the reviewed publications contain a range
of approaches, including examining specific layout and cognitive features of three-dimensional
(3D) space, developing and implementing multiple display layouts in XR environments, and
comparing different layouts.
In this paper, we reported the results of each study as evidence-based design recommendations,
principles and guidelines, with measurements such as cognitive load and presence that have been
used for the evaluation of XR display layouts. Through this exploration, we aim to contribute
to innovative approaches and established frameworks used in publications, to understand
challenges and opportunities in multiple display layouts in XR.
2. Related Reviews
Many studies have focused on utilizing multiple displays in non-immersive environments
[5, 6, 4]. For instance, Roberts and Al-Maneea [6] reviewed 340 visualization tools from articles
published between 2012 and 2018 and evaluated different strategies for laying out multiple
displays in these papers. However, we found three review papers in immersive environments
[7, 8, 9]. We focused on reviews that aim to provide a more structured perspective on evidence-
based guidelines for display layouts in XR. Roberts et al. [7] use case studies to discuss the
challenges and opportunities of using multiple views 1 in immersive visualization. Knudsen
and Carpendale [8] explored the benefits and challenges of using multiple views in immersive
analytics. Ma and Millet [9] focused on designing immersive dashboards, which are interactive
interfaces that display multivariate data through coordinated views.
In this review, our specific emphasis is on displays that have not been the primary focus in
previous reviews. Our goal is to suggest existing empirical evidence on how to layout virtual
displays in XR.
3. Methodology
For conducting a review we used the four-phase process of: identification of research, paper
selection process, inclusion criteria and screening phases [11].
To identify relevant papers we used the following query: ("Augmented Reality" OR "Virtual
Reality" OR "Mixed Reality" OR "Extended Reality") AND (display2 AND layout). We included
the abbreviation of XR terms (AR, VR, MR, XR) and ran the query on Google Scholar linked to
our university’s library resources which has subscriptions to almost all computer science and
software engineering databases, including but not limited to IEEE Xplore, ACM Digital Library,
Scopus, and WebofScience. Figure 2(a) shows the PRISMA chart of the review process.
1
Multiple views is the general term that has been used in the literature for both multiple visualizations such as 2D
and 3D charts and multiple displays [10].
2
The inclusion of alternative terms for "display", e.g., "screen" or "monitor" did not yield any additional results.
Figure 2: (a) Flowchart of the review process. (b) Examples of varied shapes of multiple display layouts:
A) Lists, Grids, and Arcs layouts [17]. B) Flat, Semi-circle, and Full-circle arrangements [18].
After running the searches, we extracted papers that were published in English, after 1997.
Then, we excluded survey papers and deleted duplicates. Furthermore, we excluded patent
descriptions, standards, slideshows, and dissertation (e.g., [12]). Then we established a set
of inclusion and exclusion criteria to proceed to the screening phase. We kept papers that
were on multiple display layouts in XR and conducted quantitative and qualitative studies for
layout evaluation. We excluded papers with only one display, and those lacking a detailed study
description. In addition, we only focused on HMD XR and exclude mobile-based and desktop
XR studies (e.g., [13]). In the next phase, we performed screening on the studies’ title, abstract,
and full text using our inclusion and exclusion criteria. This final phase resulted in 14 unique
relevant papers on the topic [14, 3, 15, 1, 16, 17, 18, 19, 2, 20, 21, 22, 23, 24].
4. Results
Reviewed papers: In this section, we closely examine the main focus in 14 publications related
to XR multiple display layouts, highlighting their design implications. For instance, [15] and [3],
explores design considerations of AR monitors for productivity work and examines the usability
of AR monitors compared to physical monitors. [1] focuses on the interaction in multiple VR
screens for mobile knowledge workers. In [16] the research examines the cognitive features of
3D space for information displays in MR. [17] compares different display arrangements in the
searching process, and [18] investigates the effects of display layout on spatial memory, using
abstract tasks. In [19], the authors uses measures such as task load and presence to understand
the impact of curvature displays on users’ performance. [2] explores the layout of virtual
displays in shared transit AR environments, considering factors such as social effects, and the
impact of the number of displays. Mori et al. [21] tested the impact of layout configurations
of virtual displays in AR. [22] investigated the passenger experience of display layouts in a
simulated VR passenger airplane environment, while [23] identifies key factors to consider in
the design and evaluation of XR virtual displays for a workplace environment and "DesignAR"
[24] addresses design challenges related to mid-air VR/AR interaction on augmented displays.
Display Layouts: The reviewed papers used diverse layouts of displays such as spherical, and
cylindrical layouts [1], 3D layouts with attributes like depth distance, information layer number,
and target relative position [16], and specific shapes like lists, grids, and arcs [17] (Figure 2(b)-A).
Some papers represented arrangements, such as flat, semi-circle, and full-circle [18] (Figure
2(b)-B), while others explore layout classification based on position in 3D environments, such
as displays placed according to their position in relation to the user’s head position on both the
vertical and horizontal axes in [2] (Figure 1-C).
Empirical Evidence: The reviewed papers provide evidence-based guidelines for designing
multiple display layouts in XR, such as allowing an information touchscreen display to break out
of the screen while transitioning from 2D to 3D can be controlled with the original touchscreen
[1]. Three design principles for improving the searching process include using uncommon 3D
arrangements, faster scans with grid and arc arrangements, and users’ preferences for grid over
arc [17]. However, VR and XR have lower usability compared to the physical environment.
[19] reveals higher task load, frustration, anxiety, and eye strain in VR, while lower usability,
perceived productivity, and well-being in VR compared to the physical environment [19, 20].
In the context of AR displays, guidelines focus on factors like situational awareness, and social
norms. [20] suggests that AR can affect self-confidence and users’ feeling of safety. The guiding
principle derived from [2] is that in shared transit, social presence play a significant role in how
people positioned AR displays. [21] provides guidelines for the design of AREDs, suggesting
the significant dependence of the user’s perception on the AR display layouts. [23] provides
guidelines about user preferences for the design of XR virtual displays. For example, one
guideline suggests that most users (67%) prefer displays no bigger than the field of view with a
scaling factor of 1.5 times the size of a physical display of the same resolution [23]. Furthermore,
[24] defines the concept of Augmented Displays as the extension of an interactive surface such
as a wall display, and mentions the importance of the alignment of augmented displays with
the display itself. Although Pavanatto et al. [3] did not explicitly provide any design principles,
their studies show that virtual monitors were slower and required more head-turning than
a hybrid combination of both physical and virtual monitors. Wang et al. [16] suggest three
principles for designing a 3D spatial information layout for a holographic command cabin in an
MR environment. These include determining the optimal depth range of information (3.4 m∼3.6
m), limiting the number of layers to five, and ensuring optimal relative positions between layers
(0.2m∼0.35m). [22] identifies five principles for multiple display layouts in MR applications:
1) Users prefer virtual displays within their personal space; 2) Design factors should ensure
social acceptance; 3) Horizontal layouts may cause discomfort and physical demands; 4) Vertical
layouts are preferred for productivity tasks; and 5) Display preferences can vary based on task
type. Designers should consider these principles to avoid negative consequences.
Overall, the reviewed papers collectively offer design recommendations for diverse XR
applications of multiple display layouts.
5. Conclusion and Future Work
In conclusion, our study collected a set of recommended design guidelines for multiple display
layouts in XR, supported by empirical evidence. Additionally, our investigation explores the
existing studies in this domain, types of XR environments, and display layouts. While these
studies provide design implications of multiple display layouts in XR, future research in the
domain should address the identified limitations in the reviewed studies including small sample
sizes, absence of direct comparisons with similar systems, and a lack of evidence-based guidelines
for various layouts.
References
[1] V. Biener, D. Schneider, T. Gesslein, A. Otte, B. Kuth, P. O. Kristensson, E. Ofek, M. Pahud,
J. Grubert, Breaking the screen: Interaction across touchscreen boundaries in virtual
reality for mobile knowledge workers, IEEE transactions on visualization and computer
graphics 26 (2020) 3490–3502.
[2] D. Medeiros, M. McGill, A. Ng, R. McDermid, N. Pantidi, J. Williamson, S. Brewster, From
shielding to avoidance: Passenger augmented reality and the layout of virtual displays for
productivity in shared transit, IEEE Transactions on Visualization and Computer Graphics
28 (2022) 3640–3650.
[3] L. Pavanatto, C. North, D. A. Bowman, C. Badea, R. Stoakley, Do we still need physical
monitors? an evaluation of the usability of ar virtual monitors for productivity work, in:
2021 IEEE Virtual Reality and 3D User Interfaces (VR), IEEE, IEEE, Lisboa, Portugal, 2021,
pp. 759–767.
[4] A. R. Shaikh, D. Koop, H. Alhoori, M. Sun, Toward systematic design considerations of
organizing multiple views, in: 2022 IEEE Visualization and Visual Analytics (VIS), IEEE,
2022, pp. 105–109.
[5] J. C. Roberts, H. M. A. Al-Maneea, Study of multiple view layout strategies in visualisation,
in: IEEE Conference on Visualization: InfoVis, 2018.
[6] H. M. Al-maneea, J. C. Roberts, Towards quantifying multiple view layouts in visualisation
as seen from research publications, in: 2019 IEEE Visualization Conference (VIS), IEEE,
2019, pp. 121–121.
[7] J. C. Roberts, P. W. Butcher, P. D. Ritsos, One view is not enough: review of and encour-
agement for multiple and alternative representations in 3d and immersive visualisation,
Computers 11 (2022) 20.
[8] S. Knudsen, S. Carpendale, Multiple views in immersive analytics, in: IEEE VIS 2017
Workshop on Immersive Analytics, 2017.
[9] Q. Ma, B. Millet, Design guidelines for immersive dashboards, in: Proceedings of the
Human Factors and Ergonomics Society Annual Meeting, volume 65, SAGE Publications
Sage CA: Los Angeles, CA, 2021, pp. 1524–1528.
[10] J. C. Roberts, H. Al-maneea, P. W. Butcher, R. Lew, G. Rees, N. Sharma, A. Frankenberg-
Garcia, Multiple views: different meanings and collocated words, in: Computer Graphics
Forum, volume 38, Wiley Online Library, 2019, pp. 79–93.
[11] B. Kitchenham, P. Brereton, A systematic review of systematic review process research in
software engineering, Information and software technology 55 (2013) 2049–2075.
[12] A. Febretti, Multiview Immersion in Hybrid Reality Environments, Ph.D. thesis, University
of Illinois at Chicago, 2017.
[13] Y. Kudo, K. Takashima, M. Fjeld, Y. Kitamura, Adaptable: extending reach over large
tabletops through flexible multi-display configuration, in: Proceedings of the 2018 ACM
International Conference on Interactive Surfaces and Spaces, 2018, pp. 213–225.
[14] S. Butscher, S. Hubenschmid, J. Müller, J. Fuchs, H. Reiterer, Clusters, trends, and outliers:
How immersive technologies can facilitate the collaborative analysis of multidimensional
data, in: Proceedings of the 2018 CHI conference on human factors in computing systems,
2018, pp. 1–12.
[15] L. Pavanatto, Designing augmented reality virtual displays for productivity work, in:
2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-
Adjunct), IEEE, Bari, Italy, 2021, pp. 459–460.
[16] W. Wang, X. Hong, S. Dang, N. Xu, J. Qu, 3d space layout design of holographic command
cabin information display in mixed reality environment based on hololens 2, Brain Sciences
12 (2022) 971.
[17] A. R. Ward, R. Capra, Immersive search: using virtual reality to examine how a third
dimension impacts the searching process, in: Proceedings of the 43rd International ACM
SIGIR Conference on Research and Development in Information Retrieval, Association for
Computing Machinery, New York, NY, USA, 2020, pp. 1621–1624.
[18] J. Liu, A. Prouzeau, B. Ens, T. Dwyer, Effects of display layout on spatial memory for
immersive environments, Proceedings of the ACM on Human-Computer Interaction 6
(2022) 468–488.
[19] V. Biener, S. Kalamkar, N. Nouri, E. Ofek, M. Pahud, J. J. Dudley, J. Hu, P. O. Kristensson,
M. Weerasinghe, K. Č. Pucihar, et al., Quantifying the effects of working in vr for one
week, IEEE Transactions on Visualization and Computer Graphics 28 (2022) 3810–3820.
[20] V. Biener, S. Kalamkar, J. J. Dudley, J. Hu, P. O. Kristensson, J. Müller, J. Grubert, Working
with xr in public: Effects on users and bystanders, 2023. arXiv:2310.09786.
[21] S. Mori, Y. Kataoka, S. Hashiguchi, Exploring pseudo-weight in augmented reality extended
displays, in: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), IEEE,
IEEE, Christchurch, New Zealand, 2022, pp. 703–710.
[22] A. Ng, D. Medeiros, M. McGill, J. Williamson, S. Brewster, The passenger experience of
mixed reality virtual display layouts in airplane environments, in: 2021 IEEE International
Symposium on Mixed and Augmented Reality (ISMAR), IEEE, Bari, Italy, 2021, pp. 265–274.
[23] Y. Zhang, J. Sun, Q. Ding, L. Zhang, Q. Wang, X. Geng, Y. Rui, Towards workplace
metaverse: A human-centered approach for designing and evaluating xr virtual displays,
International Journal of Human–Computer Interaction 0 (2023) 1–16.
[24] P. Reipschläger, R. Dachselt, Designar: Immersive 3d-modeling combining augmented
reality with interactive displays, in: Proceedings of the 2019 ACM International Conference
on Interactive Surfaces and Spaces, Association for Computing Machinery, New York, NY,
USA, 2019, pp. 29–41.