Evaluation Framework for Improving 360 Virtual Tours User Experience

Abstract

In the context of the ongoing COVID-19 pandemic and the series of lockdowns it introduced, virtual tourism allows individuals to experience cultural places and artifacts remotely from the safety of their homes. A popular form of virtual tourism is 360-degree virtual tours: digital experiences that render a 360-degree audiovisual representation of the real-world environment on the users' personal device. However, despite the growing popularity of 360 virtual tours, there is no unified conceptual framework for designing such experiences. In our work, we analyzed multiple existing virtual tours and constructed a four-dimensional framework for virtual tour design. We discuss this initial framework, describe how we plan to extend the current work-in-progress, and propose future research directions.

CCS CONCEPTS

• Human-centered computing → Ubiquitous and mobile computing design and evaluation methods.

INTRODUCTION

The ever-growing network speeds and computational capabilities of modern computers make it possible for new types of tourismassociated activities and products. One of the most popular virtual tourism directions is 360-degree virtual tours – digital experiences that render a panoramic 360-degree audio-visual representation of real-world environments, such as museums, outdoor locations, or exhibitions, on users’ personal digital devices. Usually, these digital environments possess a certain level of interactivity, such as allowing the user to choose a particular location within the place or changing one’s angle of view, looking around as if they are physically there. Thus, the designers’ aim is to emulate an experience of visiting the real physical place and enrich it with interesting and innovative interaction techniques and modalities [4].

The ongoing COVID-19 pandemic put the traditional tourism industry in gridlock and further popularized virtual tours [6]. In the conditions where it is impossible to physically visit cultural locations and exhibitions, virtual tourism can partially recreate the experience by allowing the users to remotely view and interact with tourist destinations from the comfort and safety of one’s home [2]. In addition to the convenience and cost-efectiveness of virtual tours comparing to their physical counterparts [5], previous studies on virtual tourism demonstrated the emotional appeal of virtual tours for the users, reducing their psychological stress [6] and positively influencing their psychological well-being [ 6]. Moreover, virtual tours increase users’ desire to visit the actual place [3] and improve the learning outcomes and knowledge retention [1]. Notwithstanding the growing popularity and usefulness of virtual tourism, there is no unified conceptual framework that governs the design of 360-degree virtual tour experiences. Consequently, the usability and attractiveness of these experiences for the users vary widely from tour to tour, depending on the particular design features and afordances implemented in a specific tour. This lack of consistency and unified design knowledge negatively afects user experience and satisfaction, and more thorough research is required [3].

Motivated by the existing gap in the knowledge on virtual tour design, in this preliminary work, we analyzed more than 40 existing virtual tours, evaluating them in the context of their afordances, features, and cues designed to facilitate the interaction between the system and the user. The tours varied across multiple software platforms, interaction modalities, and informational content. Based on the analysis of these tours, we have created a high-level design framework that can guide the creation and evaluation of virtual tours. The design framework consists of four dimensions: navigation, information presentation, proactiveness, and interactivity and is described in Section 2: Design Framework For Virtual Tours.

We aim to provide concrete guidelines for the creation and evaluation of virtual tours. This would benefit the industry practitioners to better leverage their design and provide enriching, pleasant, and useful experiences for virtual visitors. We plan to build on the created framework and the newly formed knowledge to extend the research in the following directions: ( 1 ) Validating and evaluating the design framework and its dimensions, ( 2 ) Creating a demo 360 virtual tour that implements diferent design dimensions, ( 3 ) Assessing usability and user-experience of the created experience.

In the following sections, we describe the design framework and layout our proposed research direction on this topic. 2

DESIGN FRAMEWORK FOR VIRTUAL

TOURS

To construct our design framework, we first identified over 40 existing virtual tours for the initial review. After reviewing each tour, we subsequently reduced the sample to the 10 most comprehensive 360-degree virtual tours, which we analyzed using the qualitative techniques of design review and deconstruction. Based on the analysis, we define a virtual tour as an information system aimed at presenting the user with a panoramic 360-degree view of a cultural heritage site, consisting of static images and/or dynamic video sequences, with multiple points of interest mapped within the resulted virtual environment. The users can move between the points of interest and browse them at their discretion. Often, the accompanied multimedia information is available at each point of interest: text snippets, images, sound, and visual cues, etc.

Our analysis allowed us to detect the common and unique patterns of interaction design used in the reviewed virtual tours. We were able to classify the identified cues and techniques into four distinct conceptual dimensions: navigation, information presentation, pro-activeness, and interactivity. In addition, we were able to identify the particular afordances that comprised each dimension. Below, we describe our findings in detail.

Navigation dimension includes all afordances built into a virtual tour that supports the user’s ability to comfortably navigate and move around the environment. In particular, the following afordances were identified in this conceptual dimension: ( 1 ) Movement Experience: the ability of the user to fluidly move around the space, experiencing a smooth transition between one point of interest to another, as opposed to instantly appearing in the chosen point of interest within experiencing the transition. ( 2 ) Freedom of Movement - how many options and places the users can choose to move to within the tour’s environment. This afordance depends on the number of points of interest that the user can choose to move to, and the sheer size of the tour’s reconstructed environment. The more points are present in the environment, and the bigger is its size, the more opportunity the designers have to increase the users’ freedom of movement. Thus, we define the ratio of POI count/space size as a key determinant of the ability to explore and navigate the space.

In particular, the following levels of freedom of movement are possible. The lowest freedom of movement is in the tours with only a Single POI in each location (essentially making it a "no movement" tour with no transition between virtual points), which can fit really small exhibitions. The low-medium freedom of movement has several POIs in each location, and that is often associated with discontinuous navigation and unnatural transitions between points of interest. Low/medium freedom of movement thus can negatively affect the user experience. A high-level ratio indicates that there are many possible moves throughout the room, meaning that such tour will be experienced as having more or less "continuous transition". ( 3 ) Spatial Orientation: the design aspects of the virtual tours that support one’s sense of direction and location within the environment. One of the common ways in which spatial orientation of the user can be supported is the inclusion of the persistent mini-map, where the user sees his or her location in the environment constantly updated in the top-down schematic representation of the environment, similar to how to the experience provided by the navigation software, such Roman Shikhri, et al. as Google Maps. Other spatial orientation afordances commonly used in virtual tours are the panoramic overview of the environment, aerial top-down view of the points of interest, simple lists of available places and shortcuts, and a visual showcase of key or popular points of interest using their snapshot images. ( 4 ) User Controls and Input Modalities: the way in which the users’ input and interaction and navigation control mechanics are implemented in a virtual tour. The controls may involve physical input devices, such as mice, controllers, or keyboards to support user interaction. Another control modality is onscreen (soft) controls, for example, a set of the on-screen directional arrow, touching or clicking on which executes a moving command. Another common control type consists of visually marking the points of interest, with a mark serving as the interactive anchor to this point. For example, the user may click the hovering dot on top of the exhibit, which instantly transfers him or her to this exhibit.

A second conceptual dimension in our framework is Information Presentation which refers to ways in which the system presents the information available to the users in the virtual tour. The following particular information presentation afordances are commonly present in the reviewed tours: ( 1 ) Visual Information Cues - the visual elements that let the users know about available information, options, and interaction opportunities. Visual cues an important design element, since it supports the users’ spatial cognition, which may be significantly reduced in the virtual experiences that lack physical cues. Consequently, visual cues should be paid particular attention throughout the virtual tour. Visual cues can be implemented in the form of clickable hotspots, labels, dialog windows, hovering media elements (2D/3D/embedded images), etc. The four major attributes of visual cues are color, form, depth, and movement. ( 2 ) Multimedia support - in addition to the visual cues, virtual tours can include audio-visual and textual elements, for example, guides, items’ descriptions, and additional imagery, audio cues and examples, etc.

The third conceptual dimension in our framework is Proactiveness. This concept relates to the degree to which the system aids the user to perform or follow desired actions and mediates his or her experience. The completely proactive system will take all the decisions and actions from the hands of the users, similarly to a tourist guide managing her tourist groups in a tightly controlled tour around the cultural locations. Conversely, the system that lacks proactiveness will resemble the users freely sauntering up the location, completely unmediated by its personnel. In our analysis, we were able to identify the two types of proactiveness: ( 1 ) Navigation proactiveness: the system mediates the users’ transitioning along with the points of interests, trying to predict the users’ desired routes and movement, rather than reacting to their explicit commands. In terms of visual navigation guidance, virtual tour’s navigation needs to be supportive, it should motivate the visitor to proceed to the next accessible hotspots of the next point of interest by showing possible movements. Additionally, virtual tours may predict the user’s Evaluation Framework for Improving 360 Virtual Tours User Experience next moves, based on the preferences and past movement patterns. ( 2 ) Interaction proactiveness: the system mediates the interactions between the users and the active components of the virtual tour - often clickable, such as information components, audio-visual cues, message dialogues, and multimedia.

The fourth dimension in our framework is Interactivity which describes the extent to which users can interact with the system. The following types of interactions are possible.

( 1 ) Interaction with exhibits - textual annotation and augmentation, measurement of exhibits, and manipulation of 3D moveable figures in the space. ( 2 ) Onboarding experience- a set of introductory screens and lfows that gradually guides the user, introducing the available options, interactions, and content. Onboarding also includes contextual help, which appears during the tour when the particular interaction or content becomes available to the user. 3

EXAMINING THE EFFECT OF DIFFERENT

TOUR AFFORDANCES ON USER

EXPERIENCE

A good user experience is a central aspect of the success of interactive products. Thus, Our main research question asks what are the efects of various afordances of 360-degree virtual tours on the behavior and user experience of virtual visitors? Our premise is that a better understanding of how diferent features and afordances in virtual tours afect behavior, usability and user experience would enable better design of such tours. We plan to examine several key afordances listed in the framework above. We will do this by designing and creating a virtual tour and comparing how users use and experience diferent variants of this virtual tour diferentiated by selected key features.

To create the virtual tour we will use the 3D-VISTA virtual tour software. We chose this platform after surveying multiple software options and picked this one due to its ability to easily alter the main afordances mentioned above. We will be using an Insta360 One R camera which enables us to shoot 360 5.3k resolution photos and videos which automatically stitches the footage into single files. The planned virtual tour will take place in the cultural heritage location of Stella Maris Monastery, located on the slopes of Mount Carmel in Haifa, Israel.

After creating the tours, we will run a remote user study to examine user’s behavior and experience with virtual 360-degree tours. Because of the COVID-19 pandemic restrictions, conducting laboratory studies is challenging and somewhat problematic. However, for an evaluation of a virtual product such as 360-degree virtual tours, remote testing is quite natural and has the benefit of increased external validity. Thus, we will ask participants to explore the tours from the comfort of their homes. We will record the participant’s screen, and analyze all participants’ interaction within the tours using screenshots as well as the system logs. In addition, we will use the think-aloud protocol, asking the participant to comment out loud while experiencing the virtual tour. This will enable us to better understand participant’s opinions and experiences. Thus, we will examine the following measures comparing them between conditions: • Behavior - We will measure user behavior within the tours examining measures such as total time spent in a tour, number of locations in which the user stopped, number of exhibits the user interacted with, etc. • Usability - We will examine the perceived usability of the website using an analysis of the think-aloud and of user behavior as seen in the videos. • User Experience (UX) - We will examine user experience using the UEQ questionnaire that will be handed out to participants after each condition.

The results of the study will help to inform designers of the efect of the afordances stated in our framework on user behavior and experience in 360-degree virtual tours. Furthermore, it will be used to refine and validate the framework to better understand the key and important features of such tours. 4

SUMMARY

Virtual tours have developed from providing motionless 360-degree content to supporting proactive and immersive experiences. The market for virtual tours expands as technology capabilities mature and more and more people use these tours as a way to remotely experience cultural heritage sites. Still, the design of these tours is critical for their user experience. Better designed virtual tours will allow organizations to attract more users and provide immersive and personalized experiences to broader and diverse crowds. Our future research will examine key elements of 360-degree virtual tours and measure their influence on the user experience, behavior, and usability. Consequently, we will develop a unified conceptual framework for designing virtual tour experiences, as they gain popularity in the shade of the Covid-19 pandemic and its impact on tourism.

References

[1] Rahman Abidin , Nunuk Suryani , et al. 2020 . Students' Perceptions of 360 Degree Virtual Tour-Based Historical Learning About the Cultural Heritage Area of the Kapitan and Al-Munawar Villages in Palembang City . International Journal of Social Sciences and Management 7 , 3 ( 2020 ), 105 - 112 .

[2] Andreas Rio Adriyanto and Ananda Risya Triani. 2015 . 360 Virtual Reality Panorama Of Indonesia Tourism . Bandung Creative Movement (BCM) Journal 2 , 1 ( 2015 ).

[3] AONAW Mohammad and Hafiz Ismail . 2009 . Development and evaluation of an interactive 360 virtual tour for tourist destinations . ( 2009 ).

[4] M. White N. Mourkousis F. Liarokapis M. Sifniotis A. Basu Petridis , Panagiotis and C. Gatzidis . 2005 . Exploring and interacting with virtual museums . Proc. Of Computer Applications and Quantitative Methods in Archaeology (CAA) ( 2005 ).

[5] M. White N. Mourkousis F. Liarokapis M. Sifniotis A. Basu Petridis , Panagiotis and C. Gatzidis . 2008 . Presence-centered assessment of virtual museums' technologies . DESIDOC Journal of Library Information Technology 28 , 4 ( 2008 ), 55 - 61 .

[6] Ting Yang , Ivan Ka Wai Lai, Zhao Bin Fan, and Qing Min Mo . 2021 . The impact of a 360° virtual tour on the reduction of psychological stress caused by COVID-19 . Technology in Society 64 ( 2021 ), 101514 .