=Paper=
{{Paper
|id=Vol-2525/paper8
|storemode=property
|title=Information system development using augmented reality tools
|pdfUrl=https://ceur-ws.org/Vol-2525/ITTCS-19_paper_16.pdf
|volume=Vol-2525
|authors=Rostislav Grigorev,Olga Medvedeva,Bahrom Valijanov,Svetlana Mustafina
|dblpUrl=https://dblp.org/rec/conf/ittcs/GrigorevMVM19
}}
==Information system development using augmented reality tools==
https://ceur-ws.org/Vol-2525/ITTCS-19_paper_16.pdf
Information system development using augmented reality tools*
Grigorev Rostislav Aleksandrovich Valijanov Bahrom Adhamdzon-ugli
Kazan Federal University Kazan Federal University
Kazan, Russia Kazan, Russia
irlirion@protonmail.com valijanov1103@mail.ru
Medvedeva Olga Anatolievna Mustafina Svetlana Anatol’evna
Kazan Federal University Bashkir State University
Kazan, Russia Ufa, Russia
OAMedvedeva@kpfu.ru Mustafina_SA@mail.ru
Abstract
This work is devoted to the issues of visualization and
information processing, in particular, to improving the
visualization of 3D objects using augmented reality
technology. The concept of augmented reality offers a
more advanced user interface for visualization due to a
combination of natural ways to control the change of
angle of an object and visualization in a real context. In
the process of performing the work, computer graphics,
algorithms, and modeling methods were used. The
experimental part of the work was carried out using a
set of development tools for tracking Vuforia and Unity
development tools.
Introduction
Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the
real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory
modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that
fulfills three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D
registration of virtual and real objects. The overlaid sensory information can be constructive (i.e. additive to the
natural environment), or destructive (i.e. masking of the natural environment). This experience is seamlessly
interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this
way, augmented reality alters one's ongoing perception of a real-world environment, whereas virtual reality
completely replaces the user's real-world environment with a simulated one. Augmented reality is related to two
largely synonymous terms: mixed reality and computer-mediated reality.
Main part
To implement augmented reality technology, two main software components are needed: tracking and visualization.
At the moment, researchers and specialists have developed an extensive theoretical and algorithmic base for their
implementation in the form of various individual components, as well as in the form of integrated programs and sets
of augmented reality development tools. Individual components include libraries and frameworks of computer
vision, three-dimensional graphics engines and other solutions.
The structure of the visualization system by means of augmented reality implies the presence of the following basic
components (figure 1):
*
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
1. Tracking subsystem that provides correct integration of a virtual object into a real environment.
2. Storage of three-dimensional models and other information on objects.
3. Visualization subsystem that provides drawing of objects by means of computer graphics.
4. A graphical interface that provides user interaction.
Figure 1. Key elements of augmented reality visualization system.
An overview of the technologies
Virtual and augmented reality (VR and AR) are modern and rapidly developing technologies. Their goal is to
expand the physical space of human life objects created with the help of digital devices and programs, and having
the nature of the image (figure 2).
Figure 2a shows the image that the user sees through special virtual reality glasses. The image is divided into two
separate pictures for each eye and specially distorted to create the illusion of three-dimensional space for the eyes. If
a person moves or simply turns his head, the program automatically rearranges the image, which creates a sense of
real physical presence. With the help of controllers (joysticks, etc.), the user can interact with the surrounding
objects, for example, he can pick up a stone and throw it from a mountain - the built-in physical model that
calculates the flight of the stone, which will further create the illusion of real space.
Figure 2b shows an application using augmented reality technology. In this application, you can place images of
furniture on the image from the phone's camera, but due to their deformations, the user gets the impression that he
sees a real object located in the room. The important thing is that in this case, the reality (room) is complemented by
a virtual chair, and the corresponding technology will be called augmented reality. Creation of augmented reality is
possible not only with the help of smartphones, but also other technical means, for example, by means of special
glasses. In this case, the virtual image is completed on the surface of the lenses of the glasses.
a) b)
Figure 2. Examples of virtual technology (a) and augmented reality (b).
Hardware that is used: virtual and augmented reality glasses, controllers, headphones, smartphones and tablets.
These devices allow a person to see and hear digital objects (figure 3). In the near future, it is expected the
appearance of the gloves with real-time feedback that allow a person to feel digital objects (figure 4).
a) b) c)
Figure 3. VR and AR devices: glasses with headphones (a), controllers (b), smartphones and tablets (c).
Figure 4. The glove prototype with real-time feedback.
Programs are created, as a rule, on the same platforms on which computer games are developed (Unity, Unreal
Engine, etc.), with the help of various tools for developing virtual and augmented reality programs (Steam VR,
Google VR, Oculus, Windows Mixed Reality, Google ARCore, Apple ARkit, Google Tango, Vuforia, etc.).
Device prototypes and the first uses of the terms VR and AR existed as early as the mid-20th century, but modern
terminology was formed in the early 90s. For VR in the work of Jaron Lanier (Jaron Lanier), for AR in the work of
authors Caudell, Thomas and Mizell (Caudell, Thomas P., and David W. Mizell).
Current application experience of VR / AR in education
In the last decade, due to the decrease in the cost of devices, technologies have become more accessible to a wide
range of users. This, in turn, led to an increase in the number of programs (applications) on various topics. For AR
apps to change users' faces, measure distances of real world objects, various puzzles, and tutorials (mainly on
anatomy and astronomy).
If we talk about the application in education, for virtual reality it is the study of nature, laboratory work in physics,
the study of dinosaurs, travelling to the planets, astronomy and much more. For AR it is the study of anatomy,
chemistry, astronomy.
VR and AR technologies are often mentioned in immersive education programs. Such programs include the use of
modern information technology in the learning process, which takes place inside various virtual worlds and
simulations, often in a playful way. This type of training contributes to increased engagement, communication
between students and interest in the subject.
The main problems faced by teachers are the extra time spent on downloading applications, training students to
work with them, poor work of geolocation, sometimes poor quality of response models, difficulties for students with
work in AR format. In General, all the problems are related to the lack of experience in working with AR and still
imperfect technology. In the future, with the development of technology, these problems will be eliminated.
Let's look at some ideas of how AR technologies can be used in education.
Visualization of algebraic surfaces, both second and higher orders. In figure 5 shows algebraic surfaces of order 2
when they are displayed using AR technology. The student will have the opportunity to qualitatively study the
surface as a real object in front of him, and not on the computer screen and, especially, books, as well as change the
parameters in real time and see the result. All this should contribute to a better understanding of the structure of
equations (interactive change of parameters) and the three-dimensional shape of surfaces.
Figure 5. Algebraic surfaces of the second orders.
Similar visualizations can be created for higher-order surfaces (figure 6).
a) b) c)
Figure 6. Algebraic surfaces of order greater than the second:
(a) - Clebsch diagonal cubic surface, (b) - Mobius strip, (с) - Klein bottle.
The main direction for application in physics is visualization of equations of mathematical physics. This shows the
solution as a physical process. The learner will be able to dynamically change the parameters of the equation and see
the impact of this change on the result.
Visualization of phase diagrams, in particular pvt-diagram (phase diagram) of water (figure 7). In the diagram it is
possible to display physical processes: Isobaric, isochoric, isothermal, adiabatic and polytropic processes. The
student will see a complete picture of the process, not projections on certain planes, interactively change the start
and end points of the process, see additional information about the process (released / absorbed energy, parameters
at the beginning and end).
Figure 7. Phase diagram of water.
In chemistry, the mapping of atomic orbitals (figure 8) will help to better understand and remember their structure.
Visualization of the structure of molecules (figure 9), allows you to see different chemical bonds in space.
Figure 8. The shape and spatial arrangement of atomic s-, p-, d- and f-orbitals.
Figure 9. Caffeine molecule 3d vector image.
In mechanical engineering, visualization of equipment models with the ability to play an animation showing the
principle of their operation. For pumps and turbines, you can place a phase diagram of the medium with the physical
process plotted on it next to it. In figure 10 shows a picture from the AR application, which shows a nuclear power
plant (nuclear power plant) with a reactor. The application displays the main structures, equipment and animated
motion environment.
Figure 10. Application of AR in nuclear power plant with reactor.
Experimental part
This article implements the information system “Painting in a Museum” with additional functionality using AR. The
experimental part of the work was carried out using a set of development tools for tracking Vuforia and Unity
development tools.
Development phase:
1. The study of the possibility of using augmented reality in the context of the Museum of fine arts.
2. The design of the user interface.
3. Create 3D models and animations.
4. Loading target labels on Vuforia.
5. Create a project in Unity.
6. Testing.
7. Release.
Results
1. With AR, visitors will get a deeper experience. Elements of augmented reality will make familiarization with the
exposition easier, will present the studied subjects more clearly. Visitors will be able to get more information in an
unobtrusive form, to feel like researchers discovering secrets hidden from the ordinary eye in the exhibits. Adding
game elements (such as a quest, a quiz, a charade, or a puzzle) to a Museum tour will increase your involvement in
the world the exhibition is dedicated to.
2. The user interface should help the user understand what to do and not interfere with the perception of information.
This is quite a painstaking process. In the initial stages, you can do without it to test the health of label tracking.
3. This is the most important and difficult stage. To create 3d models and animations, you need skills in 3d modeling
programs. We use Blender as it is free.
4. On the official website of Vuforia you need to register, download and install the plugin for Unity. To obtain a
developer key. You can now create a new app on the site, download the target tags, download the database, and add
it to the Unity project.
5. Create new projects. Add AR camera from Vuforia. Add 3d model and create hierarchy from Image Target and
3d object. Add an interface.
6. Check the performance on different devices and catch bugs.
7. Collect project.
Conclusion
The implementation of augmented reality in consumer products requires considering the design of the applications
and the related constraints of the technology platform. Since AR systems rely heavily on the immersion of the user
and the interaction between the user and the system, design can facilitate the adoption of virtuality. The technologies
underlying augmented reality headsets and software are advancing all the time. They’re getting smaller, lighter,
more detailed, and more credible by the day. Augmented reality looks set to revolutionize on-the-job training, and
enhance the abilities of designers, artists, retailers, game developers, and office workers.
Acknowledgements
The authors express their gratitude and appreciation to the organizers of the conference of the Ural Federal
University and Innopolis University for their help and support in preparing the article.
References
1. Bay H., Tuytelaars T., Van Gool L. Surf: Speeded up robust features // Computer vision – ECCV 2006. –
Springer Berlin Heidelberg, 2006. – Pp. 404-417.
2. Bimber O., Raskar R. Modern approaches to augmented reality //ACM SIGGRAPH 2006 Courses. – ACM, 2006.
– Pp. 101-120.
3. Bowman D. A. et al. New Directions in 3D User Interfaces // IJVR. – 2006. – Т. 5. – №. 2. – Pp. 3-14.
4. Handheld Augmented Reality [Electronic resource] // Christian Doppler Laboratory on Handheld Augmented
URL: http://handheldar.icg.tugraz.at/ Date of access: 20.06.2015.
5. Kurz D., Benhimane S. Gravity-aware handheld augmented reality // Mixed and Augmented Reality (ISMAR),
2011 10th IEEE International Symposium on. – IEEE, 2011. – Pp. 111-120.
6. Lee K. Augmented reality in education and training // TechTrends. – 2012. – Т. 56. – №. 2. – Pp. 13-21.
7. Yasumuro Y. et al. Consistent Presentation of Interactive Virtual Objects in Real Space with 3D Markers–
Interactive Virtual Interior Design // Digital Image Computing: Techniques and Applications, Proceedings of the
VIIth Biennial Australian Pattern Recognition Society Conference. – 2003. – Pp. 653-662.