Individual and Collaborative Cross-reality Immersive Analytics - Initial Ideas

Abstract

This paper presents research questions on supporting analytics work using cross reality (CR) and the initial idea of the investigation we want to conduct.

-

INTRODUCTION CR is a eld that aims to create interactive environments that allow users to bring virtual content into the physical or bring physical world objects into the virtual world to help users to have a better perception of data and information. Our work aims to develop the CR application to o er users a be er understanding of complex datasets in the immersive environment.

The term mixed reality has been around for decades , but realworld deployment has recently exploded due the availability of commodity hardware. Hardware is available for applications targeting the whole Milgram reality-virtuality spectrum. A ordable hardware and software enable researchers and developers to create augmented-reality (AR), extended reality (XR), mixed-reality (MR) and virtual-reality (VR) applications on smartphones, tablet or Head Mounted Displays (HMDs).[ 5 ] The term "cross-reality" (CR) is relatively new and describes a new kind of software systems that bridge between two or more points on the Milgram spectrum(Figure 1). Such applications allow their users to move data or visualizations from a PC to a physical space and/or a virtual space and back. In our studies, we want to explore how to support analytics work using CR from both individual and collaborative perspective.The SIERA project done by Lawton et al. is an example of using the virtual reality application for understanding complex datasets(Figure 2).[ 2 ] We consider the following Research Questions (RQs) as objectives of our studies: • RQ1: How can we support analytics work by moving visualizations across the Milgram continuum ? Data analysis is the process of discovering useful information by cleaning, inspecting, transforming, and modeling data.[ 6 ] Data visualization helps data analysis and interpretation. In a CR environment, information can be presented at any point of the Milgram spectrum and the application or the user can determine where it should be presented to t their needs.[ 4 ] In our study, we want to explore information visualization across the continuum to determine what kind of information is best presented at speci c levels of virtuality to provide better support to analytics work. • RQ2: What interactions do users naturally use for crossreality applications? With the support of XR, users could interact with data in 3D space without the limitation of the 2D screen, keyboard, and mouse. The XR environment that will be created for our study will be a mix of the physical world and the virtual world. Our study will explore under what circumstance users want to move visualization from monitors in the physical world to virtual space and how they will do it. We also want to study how users want to interact with visualization in XR space naturally when they have more options, including hand gestures and eye gazing. • RQ3: How does control over a shared reality impact team collaboration? Collaboration in the physical world can make reasonable assumptions about a shared reality: when two people work together in a room, they have an intuitive understanding of what the other person can see and not see. They also know that if they both look at an object in the room, they both see the same object. This creates an intuitive shared understanding of the world around them. CR applications can break this intuition: the CR system can show one object to one person, and a di erent object - in the same place - to the other person. This "unsharing" of real-world intuition can be accidental (e.g. incorrect calibration resulting in a shift of an object), on purpose (e.g. orienting text towards people on opposite sides or showing additional information to only a domain expert) or even malicious (e.g. a hacker creating a misunderstanding). We want to investigate how to balance between shared and unshared reality to enhance team collaboration e ciency. 2

INITIAL IDEAS TO INVESTIGATE

CR is a new research area with plenty of unknowns, creating immense research opportunities ranging from questions related to cognition all the way to engineering processes and development guidelines. In this spectrum, we will be focusing on a small number of speci c research topics. 2.1

Move information from a 2d screen to a 3d space and back

To better support analytic work, we want to investigate applying CR to increase productivity and enhance sense-making and communication. A previous study shows that work stations with more monitors could potentially increase productivity.[ 3 ] We want to investigate how to move information from a 2D monitor to a 3D space and back in order to overcome the limitation of the number of the physical display. Haskell and Wickens show that 3D displays are more useful for lateral and altitude ight path tracking. In the meantime, 2D displays have an advantage in the accurate measurement of airspeed.[ 1 ] We will conduct case studies to explore what visualizations should be moved to 3D AR space to support individual and team collaboration. We will also investigate how user want to interact with CR applications. 2.2

Impact of mixing AR and VR collaborators

We want to explore mixing analytic data collaborators with different systems work in the same extended reality space. We will investigate the potential impact of collaborators using devices with di erent degrees of virtuality across Milgram’s continuum. We are particularly interested in studying how collaboration works out when some participants are using AR while others are using VR spaces. 2.3

Natural gestures to support interactions

We will conduct an elicitation study to investigate how individuals and teams want to interact with the CR system with natural gestures intuitively. He hope that a common set of gestures and interactions will materialize so that toolkits can be developed to support them. 2.4

Engineering guidelines

In the long run, we would like to develop engineering guidelines to provide evidence-based advice to developers on how to design CR applications and which kind of data should be presented on speci c places of the virtuality continuum. 2.5 "Unshared" experiences in cross-reality visualizations As CR systems can overcome physical limitations, we are interested to investigate if creating realted but di erent realities can support collaboration of hinder them. We will conduct studies to explore how to present user-speci c perspectives on shared information while users are in di erent locations in a virtual space. Questions include: How far can a virtual representation deviate from intuitive physical norms without hindering collaboration? Can we enhance collaboration by "unsharing" visualizations of information?

ACKNOWLEDGMENTS

The authors would like of thank Bryson Lawton and the ASE team for providing a screenshot of SIERA, a Seismic Information Extended Reality Analytics tool. We are grateful the support of NSERC, the Natural Sciences and Engineering Research Council of Canada, for our work. Individual and Collaborative Cross-reality Immersive Analytics – Initial Ideas

References

[1] Ian D. Haskell and Christopher D. Wickens . 1993 . Two- and Three-Dimensional Displays for Aviation: A Theoretical and Empirical Comparison . The International Journal of Aviation Psychology 3 , 2 ( 1993 ), 87 - 109 . https://doi.org/10.1207/ s15327108ijap0302_ 1

[2] Bryson Lawton , Hannah Sloan Sloan, Patrick Abou Gharib, Frank Maurer, Marcelo Guarido De Andrade, Ali Fathalian, and Daniel Trad . 2020 . SIERA: The Seismic Information Extended Reality Analytics Tool . In In Companion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces (ISS '20 Companion) . Lisbon, Portugal. https://doi.org/10.1145/3380867.3426223

[3] Chen Ling , Alex Stegman, Chintan Barhbaya, and Randa Shehab . 2016 . Are Two Better Than One? A Comparison Between Single- and Dual-Monitor Work Stations in Productivity and User's Windows Management Style . International Journal of Human-Computer Interaction 33 ( September 2016 ). https://doi.org/10.1080/ 10447318. 2016 .1231392

[4] Paul Milgram , Haruo Takemura, Akira Utsumi, and Fumio Kishino . 1994 . Augmented reality: A class of displays on the reality-virtuality continuum . Telemanipulator and Telepresence Technologies 2351 ( January 1994 ). https://doi.org/10.1117/ 12.197321

[5] Steven Vi . 2020 . Exploring User Experience Guidelines for Designing HMD Extended Reality Applications . Master's thesis . University of Calgary, Calgary, Canada.

[6] Belle Xia and Peng Gong . 2014 . Review of business intelligence through data analysis . Benchmarking: An International Journal 21 (April 2014 ), 300 - 311 . https: //doi.org/10.1108/BIJ-08-2012-0050