The experience of using virtual reality for interactive spatial visualisation of environmental data Isak de Villiers Bosman 1,2, Annique Smith 1,2, Kwan Sui Dave Ka 1, Koos de Beer 3 and Jan A. Maritz 4 1 Virtual Reality and Interaction Lab, Department of Information Science, University of Pretoria, Cnr Lynnwood and Roper Street, Pretoria, South Africa 2 Gamification Group, Faculty of Information Technology and Communication Sciences, Tampere University, Finland 3 XRi Research and Development, Pretoria, South Africa 4 Department of Mining Engineering, University of Pretoria, Cnr Lynnwood and Roper Street, Pretoria, South Africa Abstract Virtual reality possesses various properties that have the potential to be beneficial for the visualisation of spatial data, including intuitive gestural affordances for looking around and interacting with data and the illusion of being physically located within a virtual space. However, some properties of the medium might also be detrimental to this purpose, such as limitations of the display technology and the possibility of motion sickness. While the medium is already being used for a variety of 3D visualisation purposes, there is no formulation of clear use-cases for virtual reality as a visualisation tool based on medium-specific considerations. Our work provides a preliminary overview towards this purpose by comparing two versions of an application for visualising environmental data in a mine: a virtual reality version and a standard desktop version. Using an exploratory approach with 26 participants and both qualitative and quantitative methods, the results highlight the ability of virtual reality to engage with spatial cognition but also some pitfalls in the design of user interfaces for interacting with large datasets. Keywords 1 Virtual reality, data visualisation, user interface design, user experience 1. Introduction While information visualisation generally makes predominant use of 2-dimensional (2D) visuals, i.e., where data is mapped only on the x- Data visualisation utilises the visual and y-axis, the use of 3D visualisation that creates capabilities of technology to represent datasets in the illusion of a z-axis for the mapping of an intuitive manner, which facilitates pattern and properties for more complex/multi-dimensional trend recognition in individuals [1]. Visualising datasets has been utilised in previous research [2]. data can help individuals make intuitive sense of There are several arguments to consider for or their properties, even if such individuals are against the choice of using 3D to represent data. unable to explain these properties in technical Firstly, some data are inherently 3D, such as language [2]. Digital technologies also allow spatial data derived from representations of the visualisation parameters to be controlled and the physical world [4], which makes representing resulting data to be updated in real time [3]. them in 3D a logical choice in such cases. Beyond 7th International GamiFIN Conference 2023 (GamiFIN 2023), April 18-21, 2023, Lapland, Finland. EMAIL: isak.bosman@up.ac.za (A. 1); annique.smith@up.ac.za (A. 2) ORCID: 0000-0002-6840-9043 (A. 1); 0000-0002-7887-9655 (A. 2); 0000-0003-2516-8232 (A. 3); 0000-0003-0871-1782 (A. 4); 0000-0002-4176-8919 (A. 5); ©️ 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 164 this, however, 3D visualisation affords the ability Lastly, while research on extended VR use is to view data from various perspectives [3] which scarce, some research indicates substantially in turn provides a larger visual area on which to worse performance and experience measures map data points [4] and allows for larger and more compared to desktop setups due to factors such as complex datasets to be visualised than when motion sickness and discomfort [12]; this relying only on 2D. Conversely, criticism against currently limits the use of VR to short periods of the use of 3D visualisation includes the fact that it time. is often used without adding value through the While there have been explorations in the use potential to generate new insights and has the of VR as a visualisation tool in various ways, there ability to bias interpretation of data, e.g., by is still frequent occurrence of converting 2D occluding data points behind other data or techniques, such as bar graphs and scatter plots, to distorting the relative size of data points based on 3D [7] and such explorations are often limited to viewing perspective [2]. Appropriate use of 3D providing specific demonstrations or proof-of- visualisation of data therefore requires concept applications for using VR technology. As consideration of the perceived benefits against the such, there are a lack of broader perspectives on limitations and how these may be addressed. the use of VR for visualisation that consider the Virtual reality (VR) technology offers many benefits and limitations of the medium itself benefits that could improve 3D visualisation. A toward the formulation of sensible use-cases [8], VR user wearing a head-mounted display (HMD) [13]. Similar to the gratuitous use of 3D for data controls their viewpoint by moving their head visualisation, there are many examples of around, which is a much more intuitive way to unmotivated use of VR within visualisation explore 3D space than traditional approaches such contexts [13], which is especially pertinent given as mouse and keyboard [5], [6]. The display itself the considerable resources required to create such provides stereoscopic 3D by delivering different applications due to the lack of dedicated tools and images to either eye, which creates a more standardised approaches. To inform the sensible accurate representation of depth [7]. VR also use of VR, our study provides a starting point for facilitates the illusion of finding oneself located the development of use-cases for VR visualisation within the virtual environment created by the by focusing on the differences in user experience software, which is sometimes referred to as as a result of the technological differences immersion [5], embodiment [1], [8], or the place between two versions of an application for illusion as part of the experience of presence [6]. visualising 3D environmental data: one using a By creating spatially embodied experiences, VR VR headset and one using a standard desktop PC. has been anecdotally linked to ease of By comparing these two versions using an understanding information visualisation [5], [8]. exploratory approach, we provide insight into the In comparison with desktop-based visualisation, platform-specific differences that have the the use of VR has been linked to performance potential to impact the experiential differences of advantages, such as increased accuracy and depth the application as a visualisation tool. These of insights gleaned from data, as well as differences provide preliminary guidance on the experiential advantages, such as feeling more design of VR visualisation applications and point successful and satisfied in terms of task to future research areas that have the potential to performance for dataset exploration [1]. More yield fruitful results. broadly, the use of VR compared with desktop has The study was driven by the following also been linked with desirable experience research questions: outcomes, such as an increase in intense positive 1. How is the experience of using VR emotions, immersion, and flow [9] as well as a beneficial over a desktop application for the reduced sense of boredom and mental workload visualisation of environmental data? [10]. 2. How is the experience of using VR The use of VR, however, is not without its detrimental or ineffective over a desktop limitations. Firstly, the field of view (FOV) of the application for the visualisation of visual display technology for most commercial environmental data? HMDs is less than half of the average human FOV [11]. Compared to desktop screens, the resolution of HMDs also requires text to be relatively large to be easily readable [11], which makes these devices less useful for text-heavy applications. 165 2. Method device controllers. In all other ways, the versions are the same. A small-scale within-subjects exploratory study was conducted to compare the desktop (PC) version of the system to the VR version in order to determine the effectiveness of VR in this context. The participants in the study were students with a background in engineering or technology at the University of Pretoria. A total of 26 participants were recruited through a message posted on the institutional learning management system and chosen using convenience sampling. Figure 1: the mine environment with some land No incentives were offered for participation. blocks selected in blue (some details have been obscured for non-disclosure purposes). 2.1. Materials The study involved an application which is aimed at visualising environmental data in the form of land formations in the context of mining engineering. The 3D layouts of these formations indicate future mining face positions as per the mine plan. Users interact with the application by moving their input device to control a “laser pointer” and pressing buttons to click and/or drag interface elements (Figure 1). Users can alter the Figure 2: the control panel that shows the date appearance of the mine over time by controlling sliders and the “laser pointer” for interaction separate sliders for future days, months, and years (some details have been obscured for non- (Figure 2). Movement is performed by navigating disclosure purposes). to a minimap and selecting a position to instantaneously “teleport” there. Users can also click on blocks of land to view information about 2.2. Procedure them, such as the weight of mining material. The application’s intended goal is for mine- The study consisted of two parts, the first of planning purposes and it is meant to be used by which involved a usability test to compare the two employees of a mining company to make versions of the application. The participants were decisions about resource-use over time. randomly allocated to one of two groups, with one Traditionally, systems of this kind take the form group starting with the VR version and the other of complicated desktop applications. Presenting group starting with the PC version. This was done this data in VR is therefore a novel way of to counteract order effects from learning to use the visualising the changes to the mine over time. application [14]. Each test was carried out in a Furthermore, VR was chosen for its ability to private room, with a facilitator present to assist the make it easier for users to explore the large mine participants, deliver the tasks, and conduct the space and make informed decisions whilst being post-test interview. The test lasted about 40 far removed from the physical location. It was minutes. During the test, participants executed a assumed that the improved visualisation of the series of predefined tasks, which were defined terrain afforded by VR over PC would allow with the assistance of a mining engineer who was stakeholders to interact with the data more familiar with the purpose of the application, and intuitively. which could be considered typical tasks which Two versions of the application were might be carried out using such a system. The developed: a VR version and a desktop version. tasks included navigating around the virtual The versions differ only in terms of the interaction space, selecting blocks and viewing their techniques, where the PC version uses a mouse information, and modifying the time sliders to and keyboard, and the VR version runs on an view the changes to the mine over time. Oculus Quest 2 device and uses the standard 166 After using each version, participants and Clarke [16]. First, both researchers completed a validated user experience familiarised themselves with the data by reading questionnaire (UEQ) [15]. The UEQ is a through it and taking notes and then commonly-used instrument for measuring a range collaboratively coded the data. Using the initial of experiential aspects through six subscales. candidate set of codes, the first two authors Attractiveness is an overarching dimension which separately reviewed this list and made a list of describes the user’s overall subjective impression suggested changes, which were then resolved of the product. The pragmatic dimensions are together. Following this, the codes were analysed perspicuity (how easy it is to become familiar with and grouped into themes and then sub-themes. the product), efficiency (how much effort is The themes were then reviewed by reading the required to perform a task), and dependability collated extracts relating to each one and (whether the user feels in control of the determining whether they formed a coherent interaction). The hedonic dimensions are whole or whether they needed to be re-coded or stimulation (whether the product is exciting and the theme renamed. An initial candidate thematic motivating to use) and novelty (whether the map was created to gain a better understanding of product is innovative). the themes and this was used to further refine the Due to the intuitive gestural affordances of the themes. Lastly, the theme names were refined and VR technology it was expected that the VR the data read through again by each researcher to version would be easier to learn and perform tasks check for inconsistencies. with. We also expected the relative novelty of the As a result of the analysis, three primary technology to significantly influence participants’ themes were identified and a fourth affective experiences. The null hypothesis was “miscellaneous” theme was used to group the formulated as follows: codes that did not fit elsewhere. The themes H0: There is no statistically significant describe (1) outcomes directly related to the VR difference between the VR and PC versions display technology, (2) outcomes directly related regarding i, where i ɛ (attractiveness, perspicuity, to the design of the interface, and (3) experiential efficiency, dependability, stimulation, novelty). outcomes of using VR. The themes are discussed In the second part of the study, a semi- below. structured interview with open-ended questions was used to explore participants’ feelings about 3.1.1. Outcomes of VR display the two systems (see appendix for interview questions). These interviews were recorded and technology later transcribed. All participants provided their informed This theme describes outcomes of participants’ consent before the commencement of the study experiences that are specifically related to and the study was approved by the institutional attributes of the VR display technology, ethics review committee (protocol number: specifically the stereoscopic 3D display, EBIT/206/2022). Clearance was not granted to proximity of the display to one’s eyes, and the collect demographic information such as gender visual resolution. and age. Sub-theme A - Spatial cognition: This sub- theme is arguably one of the most important in 3. Results this study as it relates to the preference for VR with reference to the specific aspects that contributed to its visualisation capabilities. The This section presents the qualitative results VR version allowed participants to understand the from the interviews followed by the quantitative layout of the mine more effectively by providing results of the UEQ survey. a clearer indication of the distances between objects and giving them a broader perspective of 3.1. Qualitative data the mine as a whole. This clearer indication is aided by the inherent ability of VR displays to For the qualitative analysis, the interview provide stereoscopic 3D imagery by providing transcriptions were imported into ATLAS.ti 22. slightly different perspectives for the left and right The first two authors worked together and eye respectively. “You can see the depth, whereas performed a thematic analysis on the data on the desktop version you don't have stereo 3D, according to the procedure described by Braun it's less pronounced.” (P8). 167 Closely related to this was the concept of scale. In summary, current VR technology possesses Participants found the VR version more effective varying attributes that are especially relevant for at showing the relative sizes of objects, thus 3D data visualisation. The stereoscopic and making effective use of scale to visually represent surrounding display aid spatial cognition while information. Some also felt that on the desktop the low perceived resolution creates a negative version, the objects were smaller than they would sense of display quality and harms the readability be in real life or, conversely, that in the VR of text. The potential for some users to experience version things felt closer: “When you're working eye strain also limits the amount of time VR can on the [desktop] screen you can't really look or be used. see the scale of things…you can't get the depth or the width of the real value or size or scale of it, so 3.1.2. Outcomes of interface design VR definitely brings out the scale of the actual pit in relation to the benches and the height and all of that.” (P1) This theme describes outcomes directly related In addition to improved depth and scale, the to the way that the technology allows participants VR version also allowed the participants to view to interface with the application. This relates mostly to the design aspects of the VR hardware the data more easily. This was coded as “taking in and software on its own, but also compares this more” and it describes instances where with participants’ previous experience with participants explained that the VR version allowed them to see the minute changes in the life desktop hardware and software. of the mine more clearly: “The desktop, I think the Sub-theme C - Learnability: Participants difference was more visible on a larger scale, like expressed varying stances on the learnability of 2022 and 2026 for example, that's when I could the VR application. Firstly, some participants see an actual difference. Whereas with the VR it expressed a preference for the manner in which was more visible what happens within they could interact with the application, both in terms of navigating through the virtual months...The data made sense on a larger span, so on what was happening monthly or daily, it was environment and the use of the controls. The not very apparent to see that this has been mined concept of intuitive/natural interaction was raised out [on PC]…whereas the VR provided all of by some participants as the reason why they preferred the VR version. The interaction with the that....” (P9) system was described as “easier” and “more While some participants did note a similar level of understanding from both platforms, this natural” compared to a desktop and mouse, sub-theme underscores the benefit of the VR although the latter was considered by some to be version for applications where spatial data are faster: “...even though you can do it faster with the being visualised. The ability of VR to provide keyboard, but I would prefer the controls [of the VR version] because it's effortless, you just click, users with a more realistic representation of what they are seeing affords them the ability to grasp you don't have to think about stuff.” (P11) what is being shown more easily. On the other hand, some participants explained Sub-theme B - Visual quality: A common that they were “more comfortable” or “more problem in VR is the quality of display due to the familiar” with using a desktop and this made it proximity of the displays to the viewer’s eyes, easier for them to interact with the PC version causing lower perceived resolution and resulting initially: “…it took me some time to get used to the in problems with reading text. The lower VR controls, whereas with the PC and the mouse resolution of the VR version led to some it was quite easier [sic] for me to get used to it participants expressing a preference for the because with VR, I'm adding the fact that it was desktop version in that regard, causing blurry text the first time that I was using it, so the learning and eye strain for some users: “I think I would curve was a bit steeper than with the PC version.” have to say the desktop one was a bit more (P25) visually clear. So like, it's just a monitor, so you Lastly, some participants did not prefer one can just see it, and in the VR one you still have to particular system over the other when it came to look around a lot and the text is very hazy, so I visualising the data. The VR and PC versions think the font size is too small, so maybe if that's were designed to be as similar as possible, with bigger then you'll probably see it a lot better.” only the method of interaction differing between (P20) the two, as this allowed the users to compare the systems more easily. It is therefore not surprising 168 that some users would find little to no difference for these feelings, some participants only used the between the systems when it came to using them word “fun” when describing the VR version in to interact with the data. comparison to the desktop version: “...it feels Sub-theme D - Selection accuracy: While more fun to play with the VR versus the desktop participants noted that the natural interactions of version.” (P7). the VR version made the system easier to use, the The concept of novelty is also included in this lack of precision afforded by the VR version sub-theme because several participants mentioned somewhat harmed the experience. Participants that the VR experience was more interesting or described having difficulty selecting specific exciting because it was their first time sliders on the dashboard or specific benches to experiencing VR: “I definitely prefer VR more view information. One participant attributed this than the PC version, probably because it was the to having shaky hands and a lack of familiarity first time I used VR, so it was quite exciting…” with VR, while others spoke more generally of (P25). having less control and accuracy with the VR The benefits of novelty in terms of data controls. The problem of reduced accuracy by visualisation, however, are complex. On the one way of utilising larger arm movements rather than hand, novelty has been associated with desirable smaller actions (hand or finger movements) has outcomes such as increased learning and retention been discussed in previous research and [18] and satisfaction [19]. On the other hand, it is alternative approaches have been suggested to unclear how persistent these benefits might be improve interaction accuracy, such as using a once the novelty effects start to wear off with “pen grip” instead [17]. prolonged use [20]. Nevertheless, considering that Sub-theme E - Navigation: This theme the design of the application did not include any generally describes the navigability of the virtual direct attempts at improving its hedonic, i.e., non- space. The intuitive controls discussed in sub- goal-oriented qualities, the perceived positive theme C extended to navigating around the virtual affect experienced from the VR platform alone is mine. Some participants attributed this to worth mentioning. Novelty is also a double-edged controls, while others explained that being able to sword in this instance, as the lack of experience look around in the space made it easier to identify with VR controllers was seen as a drawback of where to go and how to get there. However, some VR by some participants, as discussed in sub- participants described navigation within the theme C. virtual space as a challenge, partially because the Sub-theme G - Immersion and presence: VR headset needed to be tethered to a computer This sub-theme collectively refers to all instances via a cable, which hampered head movement. where participants mentioned experiences that, in This is also related to a suggestion given by some the VR literature, are generally referred to as participants to show position on the minimap in a either immersion or presence. A notable example way that also indicates orientation, e.g., in the is that of facilitating the place illusion [6] where form of a cone. This is an important consideration participants felt like they were “in” the to make for 3D visualisation applications, where environment being visualised: “I think the VR one the navigable space may be too large for users to [contributed to understanding the data being easily keep track of their position within the visualised], because I was actually in the space, space. so you could see everything around you and it made you feel like you were there, I think, a lot 3.1.3. Experiential outcomes of VR more than the PC which was more like you were just looking at a simulation or something like that.” (P21) While the previous two themes are related to The term “immersive” was used by specific aspects of the VR technology, this theme participants to describe a wider range of describes participants’ descriptions of their experiences, but a central commonality was the experiences while using the VR application. surrounding nature of displays that replace These outcomes relate to the holistic experience sensory stimuli from the physical world and direct created by the VR technology, rather than specific more of their attention toward the application: aspects of the input/output mechanisms. “[Preference for] the VR version, because it's user Sub-theme F - Affect: This sub-theme friendly, you don't have the keyboard in front of described general feelings of enjoyment relating you, you don't have too many screens, you're only to the use of the VR version. As an explanation focusing on one thing…compared to the screen 169 where there's a laptop, there's people, so you're Table 1 kind of focusing on one thing with the VR.” (P11) Descriptive statistics for each subscale, “But other than [the resolution] the VR version categorised by system version (VR or PC). ATTR = felt natural to use, the clicking on the box, the attractiveness, PER = perspicuity, EFF = efficiency, pointer and the map, everything just felt like I was DEP = dependability, STIM = stimulation, NOV = engrossed in the system.” (P16) novelty As illustrated by the second quote, the UEQ System Mean Medi Std. immersive experience was also facilitated by the natural interaction metaphors provided by VR, as Subs version an Dev. discussed in sub-theme C. cale ATTR VR 6.08 1.05 5.859 3.1.4. Miscellaneous 3 3 5.50 1.15 5.244 PC 0 2 This theme contained one code which could not logically be grouped with any others, which PER 6.12 1.02 5.990 relates to the “learning effects” where a VR 5 3 participant described their experience of either the 5.87 0.77 5.904 PC or VR version of the system being made easier PC 5 8 because of their prior experience with the other EFF 6.25 1.37 5.721 version. While this is a limitation of the within- VR 0 2 subjects design, it was also countered to some 5.75 1.09 extent by randomly dividing the participants 5.596 PC 0 3 between the conditions and ensuring that half DEP 5.50 1.09 began with either condition. Furthermore, the 5.481 VR 0 8 other themes provide evidence that users did 5.75 0.98 experience a difference between the two versions 5.625 PC 0 8 in terms of how the data was presented and interacted with and that this difference was STIM 6.25 0.88 6.087 attributable to the nature of VR as a medium. VR 0 0 5.12 1.49 5.029 PC 5 1 3.2. Quantitative data NOV 6.00 0.83 5.990 VR 0 8 The quantitative analysis was performed in IBM 5.37 1.50 SPSS 28.0.1.0. First a Shapiro-Wilk normality 4.923 test was carried out on each of the six subscales of PC 5 5 the UEQ. The results showed that normality was violated for the subscales relating to dependability A Wilcoxon signed-rank test was conducted to (p = .009), stimulation (p = .008) and novelty (p < compare the ratings for each of the six UEQ .001), while it was not violated for attractiveness subscales for the VR and PC versions of the (p = .054), perspicuity (p = .132) and efficiency (p system (Table 2). Data are medians unless = .031). However, the non-parametric Wilcoxon otherwise stated. signed-rank test was still used to analyse all the For attractiveness, 17 out of the 26 participants scales due to the small sample size and to make it rated the VR version higher than the PC version, possible to compare the results. 6 rated the PC higher than the VR and 3 rated no Table 1 shows the descriptive statistics for the difference between the two systems. There was a survey results, categorised according to each statistically significant median difference (.333) system type. Each question in the survey was between the VR (6.08) and the PC (5.5) version, z rated on a Likert scale ranging from 1 to 7. = -2.684, p = .007 with a moderate effect size (r = .372). Therefore, the alternative hypothesis is supported. For perspicuity, 13 out of the 26 participants rated the VR version higher than the PC version, 9 rated the PC higher than the VR and 4 rated no difference between the two systems. There was no 170 statistically significant median difference (.125) In summary, the two hedonic aspects of the between the VR (6.13) and PC (5.88) version, z = UEQ (stimulation and novelty) were rated -0.717, p = .473. Therefore, we fail to reject the significantly higher for the VR version, while the null hypothesis. three pragmatic aspects (perspicuity, efficiency, For efficiency, 12 out of the 26 participants and dependability) did not differ significantly rated the VR version higher than the PC version, between the two systems. Attractiveness as an 10 rated the PC higher than the VR and 4 rated no overarching impression was also significantly difference between the two systems. There was no higher for the VR version. Due to the small statistically significant median difference (.0) sample size, these statistical results are intended between the VR (6.25) and the PC (5.75) version, to support the qualitative results, rather than z = -.717, p = .473. Therefore, we fail to reject the present a strong argument as to the differences null hypothesis. between the two systems. For dependability, 13 out of the 26 participants rated the VR version higher than the PC, 11 rated Table 3 the PC version higher than the VR and 2 Summary of hypotheses participants rated no difference between the two Subscale Result systems. There was no statistically significant Attractiveness Alternative hypothesis median difference (.125) between the VR (5.5) supported * and the PC (5.75) version, z = -.433, p = .665. Therefore, we fail to reject the null hypothesis. Perspicuity Fail to reject null hypothesis For stimulation, 20 out of the 26 participants Efficiency Fail to reject null hypothesis rated the VR version higher than the PC, 3 rated Dependability Fail to reject null hypothesis the PC version higher than the VR and 3 Stimulation Alternative hypothesis participants rated no difference between the two supported ** systems. There was a statistically significant Novelty Alternative hypothesis median difference (.75) between the VR (6.25) supported ** and the PC (5.13) version, z = -3.507, p < .001 *p < 0.05; ** p < 0.001 with a moderate effect size (r = .486). Therefore, the alternative hypothesis is supported. 4. Discussion For novelty, 21 out of the 26 participants rated the VR version higher than the PC, 1 rated the PC In order to discuss the main outcomes of the version higher than the VR and 4 participants study, this section discusses the results in terms of rated no difference between the two systems. the research questions of the study. We also There was a statistically significant median present suggestions for the design of user difference (.75) between the VR (6.0) and the PC interfaces based on the perceived benefits, (5.38) version, z = -4.034, p < .001 with a large shortcomings, and suggestions gleaned from our effect size (r = .559). Therefore, the alternative data. hypothesis is supported. The summary of hypotheses is provided in Table 3. 4.1. RQ1: How is the experience of Table 2 using VR beneficial over a desktop Wilcoxon signed-rank test for each of the six subscales application for the visualization of [Subscale] VR - Z Asymp. Sig (2- environmental data? PC tailed) The two scales of novelty and stimulation were Attractiveness -2.684b .007 b rated significantly higher for the VR than for the Perspicuity -.717 .473 desktop version. The “affect” sub-theme with its Efficiency -.737b .461 codes of enjoyment and novelty is especially Dependability -.433a .665 relevant here, since participants used terms such b Stimulation -3.507 <.001 as “fun” and “interesting” when describing their Novelty -4.034b <.001 preference for the VR version. However, other a Based on positive ranks, b based on negative ranks sub-themes also have to be considered as a contributing factor to feelings of novelty and stimulation, such as the immersive nature of the 171 experience, interaction that is intuitive as opposed rated significantly higher for the VR version than to traditional input devices, and the feeling of the desktop version. There are several “being there”, i.e., the place illusion. This considerations to be made here. provides evidence of the usefulness of VR as a Firstly, some participants expressed negative tool to create new and interesting visualisation reactions to limitations in the display resolution. experiences that individuals might want to This led to difficulty reading text, poor resolution, experience for the sake of the platform itself, and eye strain for some. Such discomfort could which could be used to extend the reach and put a limit on periods for which VR technologies impact of such applications. As also mentioned can be used in real-world settings and highlights above, the effects of novelty and stimulation in the necessity to keep text size in mind when this case are expected to be beneficial for designing VR applications for visualisation. desirable outcomes such as satisfaction and Secondly, the fact that some participants retention of information, although the long-term experienced the interaction mechanisms and their carryover of such effects are not clear. Research level of understanding to be largely similar in the into future applications of VR visualisations two versions supports the lack of a significant would thus benefit from deeper insight into how difference in the use of these mechanisms to these hedonic affective components could be interact with and retrieve data from the effectively harnessed toward accomplishing long- application. This is not surprising, considering lasting goals. that the two versions were intentionally designed Within the UEQ, attractiveness comprises an to be similar in every way, except for those overall impression of a product based on both the necessitated by the differences in platform. This pragmatic and hedonic aspects [15]. As such, it is does, however, emphasise that designers of VR worth noting that, even though the pragmatic visualisation applications need to consider components were not rated significantly higher in optimal utilisation of the platform itself in order either platform, the overall attractiveness for the to improve pragmatic aspects as well, for which VR version was rated higher. In addition to the we provide suggestions based on our data. hedonic aspects already discussed under novelty These suggestions relate to the benefits of and stimulation, it is expected that intuitive intuitive interaction through natural interaction interaction and realism would have played a role metaphors as well as the drawback of reduced here, since both were cited by participants as precision when using gesture-controlled having a positive effect on their overall controllers. The natural interaction metaphors experience of the VR version. Furthermore, while contributed to the learnability of the application, seemingly goal-oriented aspects such as improved which emphasises the value of this approach (sub- sense of depth perception and scale did not seem theme C), but this approach also tends to make use to have a significant impact on participants’ of larger muscle groups, such as the arms and impression of pragmatic aspects, these might also shoulders, which is appropriate for larger have contributed to general feelings of quality movements but can make it harder to perform preference of the VR version. Our results also precise motor actions (sub-theme D). This suggest that the distortion of 3D data could be problem is especially relevant for visualisation addressed by the improved perception of scale and applications that might have many adjustable depth that is facilitated by VR display options for displaying and modifying sets of data. technologies. It must, however, be emphasised There are several possible solutions to this that our results are preliminary and that our study problem, perhaps the most obvious being to make was not specifically aimed at testing the the menus and selectable elements themselves comprehension of data. larger (i.e., larger hitboxes as mentioned in sub- theme D). However, this is not necessarily ideal, 4.2. RQ2: How is the experience of since a larger menu uses up more screen real estate and blocks out more of the observable using VR detrimental or ineffective environment. Furthermore, it is not always over a desktop application for the feasible to make interactable elements larger in visualisation of environmental data? visualisation solutions, since size itself is often used to denote information. For general UI elements, a solution that affords Based on UEQ scores, none of the pragmatic, more precise motor movements might be to i.e., goal-oriented aspects of the application were approach the design of such elements in a way that 172 utilises smaller muscle group movements. An considered, such as avoiding reliance on large example of such an implementation would be blocks of text, using large/bold fonts, and utilising replacing sliders that afford up/down or left/right smaller muscle movements for selections where movement with dials/knobs that afford rotation possible. and thus allow users to anchor their arm in space Our study has pointed toward fruitful avenues and perform precise movements primarily with for future research. Firstly, we have provided their forearm and wrist. preliminary evidence that VR enhances cognitive engagement with 3D data through its affordance 5. Limitations of spatial perception, particularly through the perception of scale and depth. Future work might thus explore the effects that this has on desired The sample size of the study was small but outcomes of 3D data visualisation, such as considering the research questions and the goal of comprehension and retention. Furthermore, while the study to provide guidelines to improve the VR our study has not attempted to determine the exact system, the quantitative was considered causes of these cognitive benefits, such as supplementary to the qualitative data in this study. stereoscopic 3D vs. intuitive navigation through Secondly, a small amount of discomfort was movement, the similar capabilities offered by encountered by some participants during the study other XR technologies such as head-mounted when the VR version was being used due to the augmented reality (AR) suggest that these might short cable which was used to attach the headset provide similar benefits. to the computer. The cable was necessary since the large amount of data included in the application did not allow it to run on the headset 7. Acknowledgements alone. However, the short cable inhibited the participants’ movement in the virtual world This research was funded through the Exxaro somewhat and this was commented on by 7 Chair in XR Technology in the Department of participants and coded under “difficult Information Science at the University of Pretoria. navigation”. The participants filled in the UEQ The VR system used for this study was developed twice before being interviewed, which could have by Zander van Beest van Andel and Liam Botha. primed their interview responses to be more in We would like to thank Nicole Lou who helped line with UEQ measures. Finally, the design of the with some of the data collection and all the application could not be fully described here due participants who took part in this study. Finally, to non-disclosure agreements, thus making we would like to thank members of the replicability of the study difficult. Gamification Group for their insightful comments. 6. Conclusions and future research 8. Appendix Our study has provided preliminary evidence that the VR platform outperforms a traditional The interview questions used in this study are desktop in terms of providing a more attractive, provided below: (1) Between the desktop and VR novel, and stimulating experience for visualising platform, which did you prefer and why? (2) environmental data. These differences, however, Which platform facilitated your understanding of were not found to be significant for dependability, the visualisation more effectively and why? (3) perspicuity, and efficiency. We have also How did the different platforms affect your ability followed an inductive approach to provide factors to interact with the data? (4) Please provide some that contribute to these differences, or lack suggestions for improving the VR application. thereof, of which the affordance of spatial perception might be considered to be the most 9. References relevant. The combined results suggest that sensible use cases should consider the tradeoffs [1] P. Millais, S. L. Jones, and R. Kelly, between desired outcomes, such as enjoyment, spatial cognition, and presence, against undesired “Exploring Data in Virtual Reality: ones such as reduced text legibility, selection Comparisons with 2D Data Visualizations,” accuracy, and knowledge carryover from existing in Extended Abstracts of the 2018 CHI platforms. Design alternatives should also be Conference on Human Factors in 173 Computing Systems, New York, NY, USA, [10] H. C. Lum, R. Greatbatch, G. Waldfogle, Apr. 2018, pp. 1–6. doi: and J. Benedict, “How Immersion, 10.1145/3170427.3188537. Presence, Emotion, & Workload Differ in [2] D. A. Szafir, “The good, the bad, and the Virtual Reality and Traditional Game biased: five ways visualizations can mislead Mediums,” Proceedings of the Human (and how to fix them),” interactions, vol. 25, Factors and Ergonomics Society Annual no. 4, pp. 26–33, Jun. 2018, doi: Meeting, vol. 62, no. 1, pp. 1474–1478, Sep. 10.1145/3231772. 2018, doi: 10.1177/1541931218621334. [3] J. L. Wesson and P. R. Warren, “Interactive [11] M. H. Lynn, G. Luo, M. Tomasi, S. Pundlik, visualisation of large multivariate datasets and K. E. Houston, “Measuring Virtual on the world-wide web,” in Proceedings of Reality Headset Resolution and Field of the 2001 Asia-Pacific symposium on View: Implications for Vision Care Information visualisation - Volume 9, AUS, Applications,” Optometry and Vision Dec. 2001, pp. 151–157. Science, vol. 97, no. 8, pp. 573–582, Aug. [4] R. Brath, “3D InfoVis is here to stay: Deal 2020, doi: with it,” in 2014 IEEE VIS International 10.1097/OPX.0000000000001541. Workshop on 3DVis (3DVis), Nov. 2014, [12] V. Biener et al., “Quantifying the Effects of pp. 25–31. doi: Working in VR for One Week,” IEEE 10.1109/3DVis.2014.7160096. Transactions on Visualization and [5] D. Raja, D. A. Bowman, J. Lucas, and C. Computer Graphics, vol. 28, no. 11, pp. North, “Exploring the Benefits of 3810–3820, Nov. 2022, doi: Immersion in Abstract Information 10.1109/TVCG.2022.3203103. Visualization,” in In Proceedings of 8th [13] M. Kraus, “Assessing the Applicability of International Immersive Projection Virtual Reality for Data Visualization,” Technology Workshop, 2004, pp. 61–69. 2021, Accessed: Jun. 07, 2022. [Online]. [6] M. Slater, “Place illusion and plausibility Available: https://kops.uni- can lead to realistic behaviour in immersive konstanz.de/handle/123456789/55791 virtual environments,” Philosophical [14] J. Lazar, J. H. Feng, and H. Hochheiser, Transactions of the Royal Society B: Research Methods in Human-Computer Biological Sciences, vol. 364, no. 1535, pp. Interaction. Morgan Kaufmann, 2017. 3549–3557, Dec. 2009, doi: [15] M. Schrepp, J. Thomaschewski, and A. 10.1098/rstb.2009.0138. Hinderks, “Construction of a Benchmark for [7] E. H. Korkut and E. Surer, “Visualization in the User Experience Questionnaire (UEQ),” virtual reality: a systematic review,” Virtual Jun. 2017, doi: 10.9781/ijimai.2017.445. Reality, Jan. 2023, doi: 10.1007/s10055- [16] V. Braun and V. Clarke, “Using thematic 023-00753-8. analysis in psychology,” Qualitative [8] J. Huang, M. S. Lucash, R. M. Scheller, and Research in Psychology, vol. 3, no. 2, pp. A. Klippel, “Walking through the forests of 77–101, Jan. 2006, doi: the future: using data-driven virtual reality 10.1191/1478088706qp063oa. to visualize forests under climate change,” [17] A. U. Batmaz, A. K. Mutasim, and W. International Journal of Geographical Stuerzlinger, “Precision vs. Power Grip: A Information Science, vol. 35, no. 6, pp. Comparison of Pen Grip Styles for Selection 1155–1178, Jun. 2021, doi: in Virtual Reality,” in 2020 IEEE 10.1080/13658816.2020.1830997. Conference on Virtual Reality and 3D User [9] F. Pallavicini and A. Pepe, “Comparing Interfaces Abstracts and Workshops (VRW), Player Experience in Video Games Played Mar. 2020, pp. 23–28. doi: in Virtual Reality or on Desktop Displays: 10.1109/VRW50115.2020.00012. Immersion, Flow, and Positive Emotions,” [18] J. Schomaker, “Unexplored territory: in Extended Abstracts of the Annual Beneficial effects of novelty on memory,” Symposium on Computer-Human Neurobiology of Learning and Memory, vol. Interaction in Play Companion Extended 161, pp. 46–50, May 2019, doi: Abstracts, New York, NY, USA, Oct. 2019, 10.1016/j.nlm.2019.03.005. pp. 195–210. doi: [19] N. Talukdar and S. Yu, “Breaking the 10.1145/3341215.3355736. psychological distance: the effect of immersive virtual reality on perceived 174 novelty and user satisfaction,” Journal of Strategic Marketing, vol. 0, no. 0, pp. 1–25, Aug. 2021, doi: 10.1080/0965254X.2021.1967428. [20] W. Huang, “Investigating the Novelty Effect in Virtual Reality on STEM Learning,” Ph.D., Arizona State University, United States -- Arizona, 2020. Accessed: Dec. 01, 2022. [Online]. Available: https://keep.lib.asu.edu/items/158443 175