=Paper=
{{Paper
|id=Vol-2744/short5
|storemode=property
|title=Research Methods in the Field of Computer Visualization Based on Virtual Reality (short paper)
|pdfUrl=https://ceur-ws.org/Vol-2744/short5.pdf
|volume=Vol-2744
|authors=Natalya Averbukh,Vladimir Averbukh
}}
==Research Methods in the Field of Computer Visualization Based on Virtual Reality (short paper)==
https://ceur-ws.org/Vol-2744/short5.pdf
Research Methods in the Field of Computer
Visualization Based on Virtual Reality
Natalya Averbukh1[0000000282326711] and Vladimir Averbukh1,2[0000000243791450]
1
Ural Federal University Yekaterinburg, Russia natalya averbukh@mail.ru
2
Krasovskii Institute of Mathematics and Mechanics, Yekaterinburg, Russia
averbukh@imm.uran.ru
Abstract. The paper examines research methods in the field of computer visu-
alization based on virtual reality. Developing such systems involves not only the
issues of software implementation, modelling or adequate equipment choice, but
also the tasks connected with cognitive processes occurring in both users and
developers of visualization systems. Featured is a brief review of papers deal-
ing with research in human interaction with virtual reality. The scope of tasks in
developing visualization systems based on virtual reality is determined. The con-
clusion discusses several research issues, the solutions to which will increase the
efficiency of interaction of users with virtual environments.
Keywords: virtual reality · computer visualization · mental model · cognitive
processes.
1 Introduction
Virtual reality environments were initially used for training simulation, but as far back
as in the late 1980s scientific visualization systems based on virtual reality occurred.
Those were the pilot versions of virtual test stands that were used in developing reusable
space shuttles [6]. Currently, virtual reality environments have been actively used in
scientific visualization and software visualization systems.
Developing such systems involves not only the issues of software implementation,
modelling or adequate equipment choice, but also the tasks connected with cognitive
processes occurring in both users and developers of visualization systems. There also
occurs the problem of perception of spatial characteristics: depth and distance, abso-
lute and relative dimensions of objects, distance to them. This problem occurs due to
the fact that when navigating in virtual reality, especially when using the so called nat-
ural interfaces and ‘direct interaction’ with displayed objects, a user, naturally, should
understand the distance to them and between them. Apart from that, the issue of percep-
tion of spatial characteristics can be related to the information, transmitted by the ob-
ject’s characteristics, if any abstract parameters are coded with object dimensions, their
correlation, distance between them or from them to a certain point. Thus, developing
computer visualization systems demands research in the field of computer psychology.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons Li-
cense Attribution 4.0 International (CC BY 4.0).
2 N. Averbukh and V. Averbukh
2 Related works
A virtual environment can act both as an instrument of research, in the field of psychol-
ogy in particular, and as an object of research. To be more precise, the interaction of
a person with virtual reality is an object of research, alongside with those special as-
pects that manifest themselves in a person’s behavior in a virtual environment. Another
important field is the research into the potential of using virtual environments as the
basis for computer visualization systems. A whole range of papers are devoted to these
issues. The papers of Slater’s group discuss both the issues of measuring states specific
to virtual reality [16], [20], [17], [18] and the experience of measuring a person’s notion
of their own body in the process of using an avatar in virtual reality [19], [9], [4], [10],
[8], which can be applied in such a direction as immersive journalism, when a person is
offered to ‘personally experience’ the events covered by the mass media[21].
In our country, the interconnection between the state of presence, specific to virtual
reality, and cognitive control (i.e. a metacognitive function controlling the accuracy of
fulfilling a certain task) is established [22].
There are also tasks connected with the perception of depth in virtual reality, see
further [12] and [15].
Research into the potential of virtual reality is also featured in the papers devoted to
evaluating the efficiency of 3D software visualization systems based on virtual reality,
when using a city metaphor, such as [7]. The efficiency of visualization is evaluated in
terms of performance, a range of impressions and user experience.
Two types of visualization are typically discussed in academic research: illustrative
visualization and cognitive visualization. Illustrative visualization demonstrates the al-
ready known phenomena, while cognitive visualization facilitates cogitation by show-
ing brand new effects that provide new knowledge [23]. Apart from that, one can dis-
tinguish demonstrative visualization, which shows the results of quantitative research,
thus proving (or refuting) certain ideas of a mathematician (or any other expert working
with the visualization system). Three types of imagery are used in this case: natural im-
agery, reflecting real features of the model; imagery traditional to the given application
domain (physics, chemistry, biology, mathematics etc.); and abstract imagery, which
should be invented for any specific case, perhaps, based on visualization metaphors –
basic perception ideas for visual objects [3].
3 Determining the tasks for the development of visualization
systems based on virtual reality
It is worth noting that experts setting the tasks of visualizing certain objects, processes
or phenomena have specific ideas of their structure that can be called mental models.
Situations may occur when an analyst does not know what the visualization should
look like, but he or she knows exactly what it does not look like. At the same time, he
or she is not always able to describe it clearly. A visualization developer should also
have the idea of a mental model of the object, process or phenomenon. That is why
he or she should have sufficient knowledge in the specific field and should grasp the
essence of the simulation task thoroughly. Nevertheless, an analyst often sees much
Research Methods in the Field of Computer Visualization Based on Virtual Reality 3
more in the resulting ‘picture’, than the developer has expected them to. For example,
when visualizing one of the tasks of mathematical physics dealing with air condition
around a flying object an analyst ‘saw’ much more characteristics than the developers
had expected. It turned out that, by using color shades to express pressure, temperature
etc., they managed to convey the intermediate values, which gave additional information
to the expert.
The work of a client and a visualization system user is largely explained in Brush-
linskiy’s papers. According to him, the unknown is not some kind of an ‘absolute void’,
which cannot be operated with. It exists in a certain system of interrelations linking it
with what is already given in the problem. It is in the course of disclosing these interre-
lations that something new can be extracted [5]. Brushlinskiy describes cogitation as a
process where the solution is anticipated at the first stage, gradually unfolding at further
stages of the cognitive process. Thus, a visual analyst already foresees the way visual
objects reflecting the model should look at the discussion stage.
The users of visualization systems based on virtual reality environments get in-
formation additional to that acquired in the process of using traditional visualization
methods. Moving around a virtual space allows the user to see the inside of complex
objects and their interrelation.
When using virtual reality, in order to ensure its advantages in visualization one
should develop special views that enable factoring these advantages in. In the case of
software visualization, various visualization metaphors exist, which can be used to cre-
ate such views; for instance, a popular city / landscape metaphor.
In the case of scientific visualization, one should consider each situation separately,
even if traditional methods of data representation should be used; for example, in the
case of computational grids visualization, when it is clear what to show but not entirely
clear how to do it in order to ensure the advantages of virtual reality environments or
the expert’s sustainable work process. Therefore, one should take into consideration the
distortions that occur during interaction with virtual reality.
Developed views should comply with the models of phenomena, processes and ob-
jects, as well as with users’ mental models. They should ensure those advantages that
virtual reality offers.
As noted above, a developer should study the application domain, work in close
cooperation with analysts/users in order to take into consideration their mental mod-
els and carefully examine the features of implementing visualization systems based on
virtual reality and interface interaction in such systems.
When developing views for systems based on virtual reality one should also take
into consideration the role of interfaces. Two types of interfaces can be distinguished in
the systems based on virtual reality: one for controlling and adjusting the system itself
and the virtual reality, and one for working directly with the model. These interfaces
can be of radically different types. For instance, one interface can be natural, with the
use of gestures or voice commands [11]; while the other one can use various devices
such as a keyboard, a joystick etc.
In the process of researching human interaction with virtual reality systems, an is-
sue of studying the human factor inevitably arises. Firstly, one can distinguish the afore-
mentioned issue of distortions in the perception of space, depth and distances in virtual
4 N. Averbukh and V. Averbukh
reality. Interesting work has been done before in the field of depth perception in vir-
tual reality environments, which involved demonstration of visual feedback value when
reaching a virtual object [12]. The role of a familiar dimension of the object the user
was offered to reach was also examined when tackling the same task [15]. Apart from
that, [15] shows that a task is better fulfilled if the object that the user is offered to reach
is specific, not an abstract one. Although an abstract object rather means a ‘simple box’,
while in visualization systems, this typically refers to complex structured objects with
high semantic charge. However, it is possible that the aforementioned principle will also
relate to them, as even such objects are unfamiliar to the user in a trivial way, are not
present in their experience and, therefore, do not have a familiar dimension.
The issue of accuracy in reaching a virtual object may turn out critical in using
natural interfaces. Another issue, connected with spatial characteristics, is the issue of
transmitting certain features of the simulated object by means of length, width, height
and depth, as well as overall volume or area of certain surfaces. Research is required
into whether it is possible to adequately perceive dimensional features of virtual objects
by interacting with them.
Of importance are also aspects connected with cognitive processes and aspects deal-
ing with the psychomotor system, which is inevitably involved when interacting with
a visualization system based on virtual reality. One should not forget that the issue of
proprioceptive feedback in virtual reality remains unsolved, that is why all movements
in virtual reality are perceived differently in comparison with the physical world.
As is shown by individual examples in [2], in virtual reality, interaction with the
controller can substitute natural interaction, and a person may reflexively push the but-
ton in situations where he or she would clench their fists or grab something in the real
world.
As demonstrated in [2], being in a virtual reality requires simultaneous perception of
both the real space and the virtual space from the person switching between them [17].
This may influence the interaction with the model or the decisions made by the person.
Apart from the issues mentioned above, the questions relating to the choice between
realism (which is more easily perceived) and abstraction (which may be more adequate
for the model) also arise when developing visualization systems based on virtual real-
ity. This issue also includes the problem of developing interfaces used in visualization
systems based on virtual reality. As mentioned before, interaction with realistic objects,
which have familiar dimensions, is more convenient for the user because they can cal-
culate their movements more easily than in the case of interaction with abstract objects.
Hence, the choice of visualization metaphors and interface.
4 Conclusion. Setting research problems
Developing visualization systems based on virtual reality sets a whole range of research
tasks connected with the influence of spatial perception on the interaction with models
in visualization systems or dealing with the psychomotor system.
These could be tasks relating to object dimension evaluation. The difference from
conventional spatial perception experiments is in the fact that only at the first stage
can the experimental task involve purely evaluating the dimensions. The main part of
Research Methods in the Field of Computer Visualization Based on Virtual Reality 5
such research should involve evaluation of parameters displayed by means of linear
dimensions, surface area or volumes, both absolute and relative.
Another issue related to space deals with a person’s ability to find their way in
a virtual world. Experience shows that, in some cases, a person moving around in a
complex virtual scene, which is impossible to scan at once, does not memorize the
route they have ‘travelled’. This may turn out critical in using a landscape metaphor or
a city metaphor.
On the other hand, we should examine the efficiency of various interface principles
(the natural ones and the ‘device’ ones), their convenience for users and their applica-
bility for various tasks occurring in visualization systems based on virtual reality. For
instance, the papers [14] and [13] offer interfaces that enable the use of a manipulator
glove detecting spatial gestures of the person [14] or interaction with tangible exhibits
of a paleontology museum [13].
The issues of evaluating the potential of solving complex intellectual tasks in virtual
reality also remain relevant. Such issues have been offered solutions, for example, in [1],
but the topic requires further elaboration.
References
1. Averbukh, N., Shcherbinin, A.: The presence phenomenon and its influence upon intellectual
task performance within virtual reality settings. Psychology. Journal of the Higher School of
Economics 8(4), 102–119 (2011), in Russian
2. Averbukh, N.V.: On the issue of perception of space when experiencing the phenomenon of
presence in virtual reality. In: Ershova, R.V. (ed.) Digital society as a cultural and historical
context of human development: a collection of scientific articles. pp. 11–15. State social and
humanitarian University, Kolomna (2020), in Russian
3. Averbukh, V.: Visualization metaphors. Programming and Computer Software 27(5), 227–
237 (September 2001)
4. Banakou, D., Groten, R., Slater, M.: Illusory ownership of a virtual child body causes overes-
timation of object sizes and implicit attitude changes. Proceedings of the National Academy
of Sciences of the United States of America 110(31), 12846–12851 (2013)
5. Brushlinskiy, A.: Actor, Thinking, Learning, Imagination. MPSI, Moscow (2003), in Russian
6. Bryson, S.: Virtual environments in scientific visualization. In: VRST ’94 Proceedings of the
conference on Virtual reality software and technology. pp. 201–220 (1994)
7. Merino, L., Fuchs, J., Blumenschein, M., Anslow, C., Ghafari, M., Nierstrasz, O., Behrisch,
M., Keim, D.: On the impact of the medium in the effectiveness of 3d software visualizations.
In: Proceedings on 2017 IEEE Working Conference on Software Visualization (VISSOFT).
pp. 11–21 (2017)
8. Neyret, S., Bellido Rivas, A., Navarro, X., Slater, M.: Which body would you
like to have? the impact of embodied perspective on body perception and body
evaluation in immersive virtual reality. Frontiers in Robotics and AI 7(31) (2020).
https://doi.org/10.3389/frobt.2020.00031
9. Normand, J.M., Giannopoulos, E., Spanlang, B., Slater, M., Giurfa, M.: Multisensory stimu-
lation can induce an illusion of larger belly size in immersive virtual reality. PLoS ONE 6(1),
e16128 (2010). https://doi.org/110.1371/journal.pone.0016128
10. Peck, T., Seinfeld, S., Aglioti, M., Slater, M.: Putting yourself in the skin of a black avatar
reduces implicit racial bias. Consciousness and Cognition 22(3), 779–787 (September 2013)
6 N. Averbukh and V. Averbukh
11. Pestova, M., Starodubtsev, I.: Development of specialized gesture interfaces for scientific
visualization systems. In: GraphiCon 2016: Proceedings of the International scientific con-
ference. pp. 369–373. Nizhny Novgorod (September 19-23 2016), in Russian
12. Renner, R.S., Steindecker, E., MüLler, M., Velichkovsky, B.M., Stelzer, R., Pannasch, S.,
Helmert, J.R.: The influence of the stereo base on blind and sighted reaches in a virtual
environment. ACM Trans. Appl. Percept. 12(2) (2015). https://doi.org/10.1145/2724716,
https://doi.org/10.1145/2724716
13. Ryabinin, K., Akhtamzyan, A., Kolesnik, M., Sudarikova, E.: Tangible in-
terfaces for the virtual reconstructions of museum exhibits. In: GraphiCon
2019: Proceedings of the International scientific conference. vol. 1, pp. 87–
92. Bryansk, Russia (2020). https://doi.org/10.30987/graphicon-2019-1-87-92,
https://vestnik.astu.org/ru/nauka/conference article/3665/view, in Russian
14. Ryabinin, K., Belousov, K., Chuprina, S., Zelyanskaya, N.: Perceptive-
cognitive user interface for visual analytics systems. In: GraphiCon 2019:
Proceedings of the International scientific conference. vol. 1, pp. 93–98.
Bryansk, Russia (2020). https://doi.org/10.30987/graphicon-2019-1-93-98,
https://vestnik.astu.org/ru/nauka/conference article/3666/view, in Russian
15. Schubert, R.S., Müller, M., Pannasch, S., Helmert, J.R.: Depth information from binocular
disparity and familiar size is combined when reaching towards virtual objects. In: VRST ’16
(2016)
16. Slater, M.: Measuring presence: A response to the witmer and singer presence questionnaire.
Presence, Teleoperators and Virtual Environments 8, 560–565 (October 1999)
17. Slater, M.: Presence and the sixth sense. Presence 11(4), 435–439 (August 2002)
18. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual
environments. Philosophical Transaction of Royal Society B: Biological Sciences 364, 3549–
3557 (2009)
19. Slater, M., Spanlang, B., Sanchez-Vives, M.V., Blanke, O.: First person experi-
ence of body transfer in virtual reality. PLoS ONE 5(5), e10564–e10564 (2010).
https://doi.org/10.1371/journal.pone.0010564
20. Slater, M., Steed, A.: A virtual presence counter. Presence, Teleoperators and Virtual Envi-
ronments 9(5), 413–434 (2000)
21. Steed, A., Pan, Y., Watson, Z., Slater, M.: “we wait”—the impact of character responsive-
ness and self embodiment on presence and interest in an immersive news experience. Front.
Robot. AI (2018)
22. Velichkovsky, B., Gusev, A., Kremlev, A., Grigorovich, S.: Cognitive control influences the
sense of presence in virtual environments with different immersion levels. Lecture Notes in
Computer Science 10324, 3–16 (2017)
23. Zenkin, A.: Cognitive Computer Graphics. Applications in Classical Theory of Natural Num-
bers. (Synopsis). “NAUKA’, Moscow (1991), in Russian