ion of objects is considered: from geometric bodies (balls and parallelepipeds) to cartoon fruits, and photo realistic objects after then. For these conditions, a classical two-factor experiment is carried out, in which the first factor is the visibility of the control beam, and the second is the visibility of the highlighting of the captured object. The influence of one of these factors or their combination on the success of the movement of the captured object is expected. The success is determined by the time of movement (the less time, the more successful) and the optimality of the trajectory. The formula is proposed that determines the optimality of the trajectory. It is surprising that the movement of an already captured object is not afected by any of the above factors. Apparently, it is necessary to carefully study the capture process itself, not including the post-capture movement in the calculation.
2D relative positioning device (mouse) + keyboard; interfaces based on absolute positioning within the screen (touch / multi-touch interfaces). In contrast, the VR environment is still quite young in terms of hardware and software interfaces.
Using the “classic” input interfaces for VR doesn’t seem to be eficient [ 1, 2]. They do not allow the additional degrees of freedom which is available in virtual space (2DoF vs 6DoF). On the hardware side, this is solved by creating positioning devices that track the position of a physical device in the real world and translate its position in the virtual world. This can be a joystick in hand [3], a reflective marker [ 4, 5], or a full [6, 7] or partial [8, 9, 10] pose recognition and tracking system. The interfaces based on other methods of interacting with virtual objects, including brain-computer, sound and speech recognition, and eye-tracing, will be left outside the current research for now.
When using virtual reality environments, the question of choosing between direct and indirect interaction inevitably arises. On the one hand, in fact, the interaction will always be mediated by software and hardware tools, since objects in a virtual environment are virtual by default, that is, they do not exist in reality and therefore are not available for direct interaction. On the other hand, the main advantage of a human-computer interface in a virtual environment is the ability to create the illusion of direct interaction (presence), as opposed to interacting with the desktop. It seems obvious that the less mediated user actions are better.
In this study we analyze user interaction eficiency at the interaction with virtual objects at various levels of mediation of interaction. The mediation of interaction in this case is considered as a combination of the visibility level of the interaction tool and the visual response of the interaction object. The maximum level of mediation looks like the movement of virtual objects using a visible beam and the captured virtual object is highlighting. Minimal mediation means that the participant of the experiment does not see the beam that captures the object, and the object itself is not highlighted during capture either.
The hypothesis of this study is that the visibility of the tool will make the task of moving the object more eficient.
The purpose of this study is to establish the influence of the visual mediation of the tool on the eficiency of the task of moving objects in virtual reality.
The experiment takes place in the virtual scene, represented by the room containing familiar pieces of furniture. In addition to static objects, dynamic objects appear in the room, with which the subject should interact. An example of the room view is shown in Figure 1
The experiment is split into several sessions. In the first session, the objects are geometric bodies (balls and parallelepipeds), in the second they are cartoon images of fruits, in the third they are realistic books. This was done to exclude the influence of the cognitive factor on the research results.
From time to time, an area in the virtual room lights up. The user’s task is to take any item and move it to the specified point. Then the target area changes position. The location of these targets changes randomly.
The environment uses the VR HTC Vive headset. It connects with the computer via cables. Apart from a VR headset, the Vive system is equipped with the special hand-operated controllers and with two infrared cameras for tracking a person in the environment. In order to implement the system, the Unity 3D development environment was used, along with the C# programming language and SteamVR plugin. The most elaborated and suitable for the experiment set of visual resources was chosen at the design stage in order to create the efect of the presence and the immersion into the virtual environment.
The pilot study involved 5 participants (male) aged from 21 to 33 years, the average age was 23.8 years. All of them are specialists in the IT field. Two have a little experience of interacting with VR, two have no experience of interacting with VR, one is a regular VR user. In this research, the participants in the experiment manipulated objects using a controller. From the controller came a beam that captures objects.
The participants were ofered sessions with the following difering conditions (Figure 2): 1. The beam has a visual representation in virtual space.
a) The beam has a visual representation in the virtual space and the captured object is highlighted (Figure 2a).
When a participant aims the beam at an object, the object is highlighted and remains highlighted until the participant capture. After capturing the object can be moved in the space and released anywhere. b) The beam has a visual representation in the virtual space, but the objects do not change their appearance regardless of the aiming of the controller (Figure 2b). The capture is visually determined only by the fact that the object is moving through the virtual space. The participant can also release the object anywhere. 2. The beam has no visual representation in the virtual space.
a) The beam has no visual representation in the virtual space, but the selected objects are highlighted (Figure 2c).
When a participant aims the controller at an object, the object is highlighted and remains highlighted until the participant captures. After capturing the object can be moved in the space and released anywhere. b) The beam has no visual representation in the virtual space and the objects do not change their appearance when the controller is aiming (Figure 2d).
The capture is visually determined only by the fact that the object is moving through the virtual space. The participant can also release the object anywhere. (a) The beam has a visual representation in the vir- (b) The beam has a visual representation in the virtual space and the captured object is highlighted tual space, but the objects do not change their appearance regardless of the aiming of the controller (c) The beam has no visual representation in the (d) The beam has no visual representation in the virtual space, but the selected objects are high- virtual space and the objects do not change their lighted appearance when the controller is aiming
The participant’s task was to capture the object and release it in the target area. In each session, each participant made 20 attempts.
Thus, the study investigated the efect of two factors: the visibility of the capture beam and the presence of visual highlighting of the captured object.
The efectiveness of the interaction is determined by two criteria: 1. The time it took the participant to move the object from the capture location to the target area. 2. The optimality of the trajectory, which was calculated as the correspondence between the resulting trajectory and the straight line between the location of the capture of the object and the target area.
To estimate the optimal trajectories seen as a set of provisions (points in space) of the captured object in increments of 0.04 seconds by time. The optimality estimate was calculated as follows: = 1 ︃( + ,
)︃ 2 Here • is the length of the “optimal” trajectory in the space,
√︁ = ( − )2 + ( − )2 + ( − )2.
In this experiment we used = 1.175. This value has been used to account for possible avoidance of obstacles (such as a virtual couch or chair) and is empirical. In the virtual scenes without furniture, it is recommended to use = 1. { , , } and {, , } this is the initial coordinates of the virtual object and the target in the virtual space. • is the polyline length approximating the path of the virtual object in space. The polyline is constructed as follows: every 0.04 seconds, the current position of the virtual object in the space is recorded and the point is added to the current polyline. • and are the number of points in the optimal and real trajectory, respectively. In this case, is constructing on the assumption that the object moves along the trajectory with the “optimal” velocity (the average velocity for the current participant and the current virtual scene).
The two-way ANOVA method [11] was used to process the results.
3. Results and discussions
400 trials were completed in each seance. In total 1200 trials were completed. For each
combination of conditions (control beam and highlighting of chosen object) 80 trials per subject were
presented in each seance. The means were chosen as a measure of the central tendency. The
(
To test the influence of the mediated control on the efectiveness of the participants’ movement
of objects, a 2 (control beam: visible vs. invisible) × 2 (highlighting of chosen object: visible
vs. invisible) two-way ANOVA on the trials’ time and the optimality of trajectory of objects’
movement was applied in each seance. But it didn’t reveale a significant efect of the mediated
control, see Table 1.
• (
As shown in table 1, the factor of the individual diferences between participants has the strongest efect in each seance, but this factor also isn’t significant. In this method, if the empirical value of a factor is more than the critical value of a factor, the null hypothesis is rejected. If the opposite is true, the null hypothesis is accepted. As we see, all of the empirical values of the factors significantly less than the critical values of a factor. Thus, we accept the null hypotheses: all of the factors don’t efect to an optimality of trajectory and the time of moving object. It can be assumed neither the visibility of the control beam nor the highlighting of the chosen objects nor the individual diferences between the participants nor the common factor of the visibility of the control beam and the highlighting of chosen object afect to the efectiveness of the participants’ movement of the objects after taking. You can use any way to taking object and to checking whether you take it but the efectiveness of the objects’ movement doesn’t change.
During the experiment posed hypothesis was refuted. The efectiveness of manipulating objects in the virtual reality is not afected by the visibility or the invisibility of the tool. From this, we can conclude that the speed and the optimality of moving objects in the virtual reality is not related to what tools were used for this.
This is most likely due to the fact that after the object has already been captured, the task of moving it no longer depends on the visibility of the tool. In the course of the experiment, the trajectory of the already captured object and the time of its movement after the capture were measured.
It is necessary to continue researching the influence of the instrument’s mediation on the capture of the object itself. This will help to move closer to understanding the optimal way of the mediated interaction in the virtual reality environment.
The authors are grateful to the student team of the Ural Federal University “ThreeXyNine” (D. Gmyra, M. Gashkov, N. Bugrov) for technical assistance in carrying out the experiment.
We thank the participants in the experiment. [8] M. G. Jacob, J. P. Wachs, Context-based hand gesture recognition for the operating room,
Pattern Recognition Letters 36 (2014) 196–203. [9] M. Jacob, Y.-T. Li, J. P. Wachs, A gesture driven robotic scrub nurse, in: Systems, Man, and Cybernetics (SMC), 2011 IEEE International Conference on, 2011, pp. 2039–2044. doi:10.1109/ICSMC.2011.6083972. [10] P. O. Kristensson, T. Nicholson, A. Quigley, Continuous Recognition of One-handed and Two-handed Gestures Using 3D Full-body Motion Tracking Sensors, in: Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, IUI ’12, ACM, New York, NY, USA, 2012, pp. 89–92. [11] E. Sidorenko, Methods of mathematical processing in psychology, LLC Speech, St. Petersburg, 2001.