With the increasing prevalence of virtual reality, games for this medium have also been increasing in number lately. Several factors affect the balancing of video games. However, effects of environmental clutter and motion on game design for virtual reality have not been well explored yet. Environmental clutter and motion are important in making virtual environments more interesting and closer to real life since real world environments usually include some form of clutter or motion. Total exclusion of clutter and motion may make the virtual environment drab and dull. On the other hand, these components may affect the user performance, making the tasks and the game in general more difficult. This study aims at exploring the effects of clutter and motion on task performance in a virtual reality searching game. A user study was performed with 15 participants. Results indicated that clutter effected user performance negatively, made the users feel more restricted inside the virtual environment and made it more difficult to concentrate. No effect of motion on user performance was observed.
H.5.1. Information interfaces and presentation (e.g., HCI): Multimedia Information Systems—Artificial, augmented, and virtual realities.
Video games intend to provide entertaining, fulfilling
and meaningful experiences for the users. Several
components effect user experience such as game
mechanics, narrative and interface [
In the last few years, with the release of the second
generation virtual reality headsets such as Oculus [
Effects of clutter and movement on task performance in VR games has not been well studied yet. It might be thought that less clutter and movement may simplify the task and lead to better user performance. However, it may also make the games too simple and boring, and degrade the quality of training offered by serious games. The motivation behind this study is to give insight into future VR games for better balancing (entertainment games) and more effective training (serious games). This study tries to emphasize the importance of environmental elements in game design since these elements may have a direct impact on the perceived difficulty of the task, hence the game balancing.
There have been very few studies that explored the
effects of clutter and movement on user performance
or experience in VR games. Ragan et al. studied the
effects of visual complexity and field of view on training
effectiveness for visual scanning tasks in virtual reality
systems [
A virtual reality searching game was designed and implemented to evaluate the effects of clutter and motion on user performance. The goal of the user was to find and mark the boxes with the matching labels with the one that was shown on the virtual display. The boxes were positioned on six virtual shelves surrounding the user (Figure 1). As the user touched a box with their hands, the color of the box changed and got highlighted in pink to provide real time feedback. The users could walk on the tracked area to select the virtual boxes (Figure 2). The virtual shelves were placed to fit inside the tracked area so that the users didn’t need to step out of the tracked area. If the users selected a box unintentionally, they could deselect it by touching again. When deselected, the boxes returned to their original color. The labels of the boxes were always facing the user so that they did not need to move around the shelves to see the other faces of the boxes. There was a time limit of 25 seconds that was imposed to be able to explore the effects of different conditions on user performance. Each shelf had three levels. Each level of each shelf had three boxes and each shelf had twelve boxes in total. For clutter, there were two conditions: no-clutter and clutter (Figure 3). For motion, there were also two conditions: no-motion and motion. Moving objects were as follows: a forklift, five small fans on the shelves, a big fan on the ceiling, flowing text on six displays attached to shelves, three blinking reflectors, two smoking bins and two swinging machine arms. Clutter and motion were not only in the background, but also in the workspace.
Hardware
12 Opti Track V100R2 FLEX cameras were used for real
time motion tracking. The size of the tracked area was
8ft by 8ft. A VR2200 head mounted display (HMD) was
used for viewing [
Two by two within subjects experiment was performed with the independent variables of clutter and motion. Both independent variables had two levels: no clutter clutter and no motion - motion, making four levels in total. The level of these two variables were varied within subjects in four conditions: no clutter-no motion, clutter-no motion, no clutter-motion, and cluttermotion. Each participant completed three instances with each condition. The configurations were assigned to each user in a random order with counterbalancing. In each instance, a different box label was presented to the users so that no two instances of the total twelve had the same two labels. The users needed to find as many boxes with the requested label as they could within 25 seconds. This duration was found by in-house testing as the duration in which a user familiar with VR could select all of the boxes. We avoided giving the users too much time to be able to detect the differences between different conditions. Each trial contained 10 boxes with the requested label that were distributed roughly evenly among the six shelves. The users were not informed on the number of boxes in the scene with the requested labels to make them keep searching. Box distributions and the labels were assigned randomly to the instances to eliminate any possible learning effect. The score was calculated as follows: Number of hits - number of misses + (0.5)*number of corrections. Hits were the boxes that were correctly selected (matching with the requested label). Misses were the boxes that were incorrectly selected (not matching with the requested label). Corrections were the deselections of the misses.
Our study aims to answer the following research question: What are the effects of clutter and motion on user performance in virtual reality games? We developed the following two hypotheses: (H1) Clutter will effect user performance negatively. (H2) Motion will effect user performance negatively.
Automated data was collected for the following: box distribution, number of hits, misses, unintentional touches and corrections with their time logs. After the users completed three repetitions of a condition, a survey was given to them that had questions on the perceived difficulty of the task, frustration, ease of finding the boxes, distraction, feeling of being restricted, ease of concentration, presence and motion sickness as well as user comments.
Participants 15 adult individuals participated in the study (N = 15) who were recruited via e-mail announcements and word of mouth. All participants were undergraduate or graduate university students from several different majors. Participants were aged between 21 and 33 (µ = 25.80, SD = 3.05). Gender distribution was 5 female and 10 male. 13 participants had no prior virtual reality experience, 2 participants had minimal prior virtual reality experience. The user study was conducted under the IRB Pro00013008.
Participants arrived at the laboratory, read and signed the consent form and filled out the demographics questionnaire. The research staff briefly explained the VR equipment and the user’s objective in the game. 6.96 6.82
5.91 5.56
No Clutter
Clutter
Motion
No Motion
The research staff helped the users to wear the head mounted display and the hand bands, and a familiarization session began. The aim of the familiarization session was to make the users comfortable with the VR system and the game mechanics. The familiarization session included one display and one shelf with the no clutter - no motion condition. The familiarization session ended when the user stated that they were comfortable with the VR system, which took 53 seconds on average. The experiment then began. The users were randomly assigned with a condition and then presented with three instances of that condition. After the users completed one condition’s three instances, they were given a survey. Then, they were assigned with another condition and completed three instances with that condition that was followed by a survey. After the users completed the four condition’s three instances (12 instances in total), the experiment ended and the research staff helped the users to take off the worn equipment. The labels, the boxes in the scene and the distribution of the boxes on the shelves were assigned randomly for each instance to eliminate any bias and learning effect.
Average scores for the four conditions are presented in Figure 4. As we analyzed the data for the effects of conditions on score using two way ANOVA with repeated measures with α = 0.05 and Bonferroni correction; for clutter/no-clutter variable there was statistically significant difference (F(1, 14) = 14.259, p = 0.002), for motion/no-motion variable there was no statistically significant difference (F(1, 14) = 0.177, p = 0.680). As paired t-tests were performed, the results presented in Table 1 were obtained. Effect of clutter on user performance was statistically significant when there was no motion. Plot of the means of scores for the clutter and the motion variables are presented in Figure 5. As it can be observed in Figure 5, change in the clutter variable creates a significant change in the mean score. Presence of clutter worsens the score. However, no significant effect of motion on the score was observed.
At the end of each condition’s trial of three instances,
we asked each user to give a score for the following
variables: the perceived difficulty of the task, level of
frustration, ease of finding the boxes, level of
distraction, feeling of being limited (restricted), ease of
concentration, presence and motion sickness, within a
Likert scale of 5 to 1 (5: very much, 1: not at all). The
presence questions were from the Witmer and Singer’s
questionnaire [
p 0.048 2.168 0.048 0.546 0.499
p 0.207 0.751 0.334 0.619 0.695 t-stat -1.323 -0.323 -1.000 -2.168 0.048
= 0.05 and Bonferroni correction resulted in statistical significance only for the following variables: clutter variable for feeling of being limited (F(1, 14) = 7.977, p = 0.014) and clutter variable for ease of concentration (F(1, 14) = 5.091, p = 0.041). As paired t-tests were performed, results in Table 2 and Table 3 were obtained respectively. For the limitation, effect of clutter was statistically significant in both the presence and the absence of motion. For the ease of concentration, effect of clutter was statistically significant only for the absence of motion.
In the surveys, there were open ended questions about what the users liked the most and the least about the game, and if they had any additional comments or suggestions. There were a lot of positive comments about how fun and realistic the VR game was. User 1: “Very realistic. Boxes seemed like they were there.” User 13: “It is fun and interactive. Also going against the time makes it challenging.” User 18: “Overall, I thought this was a lot of fun. I liked how real it felt and the difficulty of it.” A lot of users stated positive comments about the clutter. For the clutter - no motion condition, the following comments were made: User 5: “I think this one has more decorations on the shelf. I feel this makes it look more real.” User 10: “I liked the mess.” User 17: “I liked the extra materials added.” User 18: “I liked how there were other things around the room making it more difficult to find the boxes.” For the clutter - motion condition, the following comments were made: User 2: “I liked all the detail.” User 5: “The scene looks very real.” User 20: “The boxes were usually easy to spot and the environment seemed like one I would find in the real world.” A few users stated negative comments about the distraction caused by the clutter: User 1: “Clutter (tools etc.) made the task difficult to focus on the boxes. The distractions made the task significantly more difficult.” User 13: “The mess was the most distracting thing.
Difficulty of the Task
Frustration
Ease of Finding
Distraction
Limitation
Ease of Concentration
Presence
Motion Sickness Clutter Motion
Clutter NoMotion
NoClutter Motion
NoClutter NoMotion
The user study results and the statistical analysis
indicated that clutter effected user performance
negatively, supporting H1. This aligns with the findings
in [
A few users stated that they perceived that the shelves were farther away in the conditions with no-clutter. Although the layout was the same between all four conditions, lack of clutter made the virtual environment seem larger. This might be incorporated into game design when the scene is desired to look spacious. On a different note, some users stated that when the label to search for was a brand they liked or they were familiar with, the task was more fun to perform. In light of these results, when task performance isn’t important in the designed game, extra elements may be added in the scene to give the users a more realistic and more interesting game environment. However, in serious games where the task performance is important, it might be a better practice to keep the virtual environment empty at first, eliminating any distractions and then gradually inducing clutter and dynamism as the user gets trained. This way, the users would be better prepared for real world conditions without getting overwhelmed.
This study focuses on virtual reality games. The results may or may not be applicable for games for other mediums such as computer or console. It should also be noted that this study was based on a searching task, which shaped the results and their interpretation. Age of the participants may be another limiting factor in applying the conclusions of the study for the children or elderly population groups.
In this study, effects of clutter and motion on user performance in a virtual reality searching game was examined in the search of more effective virtual environment design. A two by two user study was performed with 15 college aged participants. Results indicated that clutter effected user performance negatively, made the users feel more limited and made it more difficult to concentrate. No effect of motion on user performance was observed. We believe these results will provide insight into the design of future virtual environments that will lead to more user friendly everyday VR experiences.
Future work may include evaluating different levels of clutter and dynamism, or other environmental properties on user experience in virtual environments.
The authors thank the Florida Department of Education for funding the VR4VR project, under which this study was performed.
1. Felipe Bacim, Eric Ragan, Siroberto Scerbo, Nicholas F. Polys, Mehdi Setareh, and Brett D. Jones. 2013. The effects of display fidelity, visual complexity, and task scope on spatial understanding of 3D graphs. In Proceedings of
Information Processing Society, Toronto, Ont., Canada, 25-32.