=Paper=
{{Paper
|id=Vol-3373/paper22
|storemode=property
|title=Technology and Scenarios for Objects 3D Models visualization using Augmented Reality
|pdfUrl=https://ceur-ws.org/Vol-3373/paper22.pdf
|volume=Vol-3373
|authors=Olga Pavlova,Andriy Bashta,Mariia Kostiuk,Houda El Bouhissi
|dblpUrl=https://dblp.org/rec/conf/intelitsis/PavlovaBKB23
}}
==Technology and Scenarios for Objects 3D Models visualization using Augmented Reality==
https://ceur-ws.org/Vol-3373/paper22.pdf
Technology and Scenarios for Objects 3D Models visualization
using Augmented Reality
Olga Pavlovaa, Andriy Bashtaa, Mariia Kostiuka and Houda El Bouhissi b
a
Khmelnytskyi National University, Institutska str., 11, Khmelnytskyi, 29016, Ukraine
b
LIMED Laboratory, Faculty of Exact Sciences,University of Bejaia, 06000, Bejaia, Algeria
Abstract
Current trends and prospects of using augmented reality technology for various industries are
considered. Analysis of publications and the latest commercial developments that use AR
showed that this direction is promising both from the scientific and business points of view. In
this study, the concept of information technology for 3D models visualization in augmented
reality is proposed. A mathematical model of object representation in three-dimensional space,
a method and algorithm for objects 3D models visualization in augmented reality, as well as a
structural diagram and principle of operation of the proposed information technology for
objects 3D models visualization in augmented reality have been developed. A test three-
dimensional model of the object was also developed and experiments were conducted on its
visualization in AR using the proposed information technology for objects 3D models
visualization in augmented reality.
Keywords 1
Augmented reality (AR), Information Technology (IT), 3D model, visualization, Mobile
Application
1. Introduction
Today, the issue of using augmented reality in various areas of activity is of great importance both
from a scientific and a practical point of view. According to the latest research on the market [1]
augmented reality is an effective tool for business, as it allows to make purchases, namely trying on
clothes, shoes, interior items and furniture without physically visiting the store. This is currently
extremely relevant in a world facing the consequences of the global Covid-19 pandemic, where
businesses have had to look for tools to be able to operate and communicate with customers remotely.
It is also extremely relevant now, when Ukraine is at a state of war.
Augmented reality provides visualizing objects and things that may not be physically nearby, but a
person will see them in real time and in life size. This is useful in the development of virtual training
equipment and simulators, for example, for the medical and military industries, since real equipment is
expensive and in most cases quite large for the equipment of training centers.
There is no doubt that augmented reality (AR) is already being integrated into almost all areas
of industry. This technology is already available on almost every smartphone/tablet, and according to
Statist's forecasts (Fig. 1), the industry will only grow (capitalization is expected to be 1.73 billion
dollars by 2024).
A lot of industrial areas (including retail, automotive, healthcare, education, entertainment and
others) are implementing AR, there has never been a greater demand for AR-based applications
development services than there is today [1].
IntelITSIS’2023: 4th International Workshop on Intelligent Information Technologies and Systems of Information Security, March 22–24,
2023, Khmelnytskyi, Ukraine
EMAIL: olya1607pavlova@gmail.com (O. Pavlova); andreybashta@gmail.com (A. Bashta); maria@khnu.km.ua (M.Kostiuk)
ORCID: 0000-0003-2905-0215 (O. Pavlova); 0000-0002-0775-1347(A. Bashta); 0000-0002-9559-4109(M.Kostiuk)
© 2023 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
Figure 1: Number of mobile AR active users worldwide from 2019 to 2024 according to [1]
According to the analysis, the topics of the most frequently published scientific research works in
augmented reality domain are devoted to AR application for educational purposes, medicine, E-
commerce and advertising area and AR/VR-based user interfaces for different purposes. The results of
the quantitative analysis of the publications number in each domain are presented in Fig.2.
Figure 2: Analysis of industries where the most research work is conducted in the direction of
augmented reality application
Considering the abovementioned, development of a multipurpose information technology that could
provide the visualization of any kind of object`s three-dimensional model in augmented reality in
natural size and in real time currently would be relevant.
Since smartphones or tablets are the most frequently used tools to operate with AR-based
applications, the most convenient form of the proposed information technology implementation would
be cross-platform mobile application. It would make it available for using on both Android and iOS
based devices.
The areas of such information technology application can also be various - learning purposes (for
example, an atom or molecule structure visualization in Chemistry), in museums or archeology domain
to visualize the historic artefacts, in medicine for internal organs structure visualization etc.
2. Related works
In the course of the study, an analysis of the latest scientific publications devoted to the application
of augmented reality for three-dimensional objects visualization was carried out.
In [2] the method, algorithm and software-based approach of route paving using 3D markers and
augmented reality technology are proposed.
The authors of [3] propose three relatively nontechnical methods to produce 3D AR objects for
chemistry courses and demonstrate their use as both quick in-lecture activities and as part of an extended
laboratory.
Thе paper [4] proposes the integration of photogrammetric reconstruction, 3D modelling and
augmented reality application in order to achieve the complete visualization of a stone sculpture even
if highly damaged or fragmentary.
The paper [5] provides two demonstrations of how Augmented Reality (AR), which is the projection
of virtual information onto a real-world object, can be applied in the classroom and in the special
laboratory.
The paper [6] applies Geographic Information Technologies (GIT) to the field of Cultural Heritage
(CH), aiming to analyze patrimonial valuation through Digital Representations (DR). It approaches the
main topics of 3D data acquisition, modelling and visualization, as well as cultural heritage applications,
and presents 3D and Augmented Reality (AR) trials, together with their main results and discussion.
In [7] a general review of using augmented reality as an efficient tool for software publications is
given. However, any practical case of using AR is not highlighted.
The authors of [8] present DesignAR, an augmented design workstation for creating 3D models.
The authors of [9] provide radiological image visualization using virtual and augmented reality for
better planning and monitoring of surgeries.
In paper [10] reviews and advances existing literature concerning immersive employee experiences
in the metaverse. In this research, previous findings were cumulated showing that virtual work
environments as computer-generated reality spaces integrate digital twinning, remote collaboration
tools, productivity software, and wearable self-tracking devices.
The authors of [11] give an example of the application of augmented reality in medical education
would be an augmented reality T-shirt that allows students to examine the inside of the human body as
an anatomy lab.
Also, since IT industry of Ukraine is developing rapidly and following the world IT trends, it was
decided to conduct an analysis of Ukrainian scientists` research works in augmented reality domain.
Thus, the authors of [12] conduct an analysis of the current state and prospects for the development of
augmented reality in Ukraine in business and education.
In [13] augmented reality is proposed to be used in university education of future IT specialists. The
authors of [14] propose using AR technology for interactive Chemistry learning. Overall Ukrainian
scientists mostly consider AR implementation for educational purposes.
The conducted review of the literature sources showed that none of the known solutions are aimed
for multi-purpose three-dimensional objects visualization in augmented reality. Also the analysis of
already known software solutions which have AR-component and provide visualization of 3D models
of objects has been conducted. The advantages and disadvantages of the reviewed tools are also given.
The results of the analysis are presented in Table 1.
Table 1
Review of the ready-made solution that use augmented reality for objects visualization
Name Presenta- Operating Brief Advantages Disadvantages
tion form System Description
Fectar Mobile Android, IOS In app-AR- -variety of AR- -low quality of
application model models AR-models
gallery, -large users because of
model can community moderation lack
be added by -does not -overloaded
user require device interface relative
with AR to the
support application`s
functionality
Assemblr Mobile Android, IOS An AR - laconic design -access to certain
application model - a wide AR models is paid
gallery app selection of AR
that allows models
users to - convenient
create and search for AR
share AR models by
models dividing them
from into categories
mockups - a large
number of AR
models divided
by categories
- the possibility
of using ready-
made 3D
models
UniteAR Mobile Android, IOS An - a large - access to the
application application number of AR application is
designed to models divided granted only after
create, by categories granting access to
distribute -the possibility the camera
and display of using ready- - low quality AR
AR models made 3d models
models - no model
preview
AR- Mobile Android, iOS The part of - detailed and - the number and
component application the animated AR variety of AR
of Google application models models is limited
application designed to - availability of - users cannot
display an a preview upload their own
AR model before models
related to a switching to
search AR mode
query
TeamViewer Cross- Android, iOS, Designed - the product - paid application
Assist AR platform Windows, for remote has a clear
application macOs assistance practical - a small number
by application of pointers in the
transmitting form of AR
a video models
signal from
the device
and
overlaying
auxiliary
symbols in
the form of
AR models
on top of
the video.
ARt Website Cross- A website - preview for - minimalistic
platform aimed at AR models interface
rendering - expandable - AR support is
AR models list of AR required from a
for exterior models device
design.
ARvid Mobile iOS An AR- -over 500 -paid application
application based realistic -in app purchases
mobile augmented
application reality 3D
with 3D - most of the
models to AR models are
place and animated and
control. have 4K
textures.
Currently, the only completely ready free of charge tool for visualizing three-dimensional models
of objects is the AR component of Google mobile application [15].
Among the advantages of this product is that the applications are available for Android and iOS
operating systems, and the models for visualization are animated. Among the disadvantages - the
database contains only 32 models, these are mostly animals and birds, the function of uploading own
models is absent.
Taking into account the relevance of this issue, it was decided to develop an augmented-reality-
based information system for objects three-dimensional models visualization in the form of cross-
platform mobile application.
Therefore, the aim of this study is:
1) to create a method and algorithm of objects 3D models visualization in augmented reality;
2) to develop the client-based part in the form of cross-platform mobile application which provides
3D models visualization;
3) to create the test models for visualization in AR and conduct the experiment on objects 3D
models visualization in augmented reality in Khmelnytskyi National University campus.
3. Representing of a three-dimensional object in space. Mathematical
model
To present a 3D model of object in real world, it is necessary to take into account that we need
to work in three-dimensional space.
Since the ultimate goal is to develop a tool for visualizing models in the form of a mobile
application, we will use the smartphone camera with AR function support as a tool to operate with
images in augmented reality.
First, we need to consider the coordinates of the external place, where the camera of mobile
phone focuses.
The phone itself will also conditionally be located in the three-dimensional Cartesian coordinate
system at the intersection of the X, Y and Z axes.
Figure 3 shows a scene with an object model image in the form of a polygon with a camera (a)
and a view from the device's camera (b).
а) b)
Figure 3: The model of the object in the three-dimensional Cartesian coordinate system (a) the
entire scene with the camera b) the view from the device's camera
To develop an algorithm for representing an object in augmented reality, it is necessary to calculate
the distance from the device's camera to the model of the object, which we will see on the smartphone
screen according to formula (1):
𝑅𝑅
𝑙𝑙 = 𝛼𝛼𝛼𝛼 , (1)
sin
360
where l is a distance from the device camera to the AR model;
R is a radius of the virtual sphere attached to the camera;
𝛼𝛼 is an angle of view.
The following formulae (2-4) are used to convert the coordinates of the represented model from the
Cartesian coordinate system to the spherical one:
𝑟𝑟 = �𝑥𝑥 2 + 𝑦𝑦 2 + 𝑧𝑧 2 , (2)
where r is a distance from the origin of the coordinates to the point.
�𝑥𝑥 2 +𝑦𝑦 2
𝜃𝜃 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 � 𝑧𝑧
�, (3)
where 𝜃𝜃 is a polar angle.
𝑦𝑦
𝜑𝜑 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 � �, (4)
𝑥𝑥
where 𝜑𝜑 is an azimuthal angle.
Also we need to calculate the orientation of the model along the Y axis, around the center of rotation
(formula (5)) and the distance between points in three-dimensional space (formula (6) :
𝑦𝑦 = 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎(−𝑥𝑥, −𝑧𝑧) − 𝜃𝜃. (5)
𝑙𝑙 = �(𝑥𝑥2 − 𝑥𝑥1 )2 + (𝑦𝑦2 − 𝑦𝑦1 )2 + (𝑧𝑧2 − 𝑧𝑧1 )2 , (6)
where L is the distance between points or beacons in three-dimensional space.
4. Method of objects 3D models visualization in Augmented Reality
To develop a proposed tool for 3D models visualization, first it is necessary to develop a method of
its operation, taking into account the abovementioned mathematical model. The graphic representation
of the Method of objects 3D models visualization in augmented reality is presented in Fig.4.
The proposed method consists of the following steps:
1. Development of software for model visualization in augmented reality;
2. Construction of test models of 3D models of objects
3. Downloading test models to a software and technical tool, which is most convenient to present in
the form of a mobile phone application;
4. Life-size, real-time augmented reality 3D models of objects in the environment using a
smartphone camera.
5. Conducting experiments on the successful location of the object in space.
Figure 4: Method of 3D objects models visualization in augmented reality
As it can be seen from the Figure, software and technical toolkit will contain 3D models in its data
storage, making them available for visualization using the smartphone camera. User also can upload
own models in application`s data storage throuhg user interface.
When developing applications with complex functionality, special attention should be paid to taking
into account all possible bottlenecks at the early stages of the life cycle, that is, when designing the
architecture of the software tool.
That is why a special attention should also be paid to UX development.
According to [16] user interface should follow the main Gestalt principles — Proximity, Similarity,
Continuation, Closure, and Figure/Ground, since AR-based user interface design is not only about the
aesthetics, it is about intuitiveness and comprehensibility of functionality in appearance, which is more
a psychological component of user interface design.
Such interface should be user friendly and clear. Also a special attention should be paid to 3D
graphics design. The models for representing in AR must be of a high quality.
Based on the method, an algorithm for objects 3D models visualization in augmented reality was
developed, which is presented in Figure 5.
As an environment for models visualisation any view from the device camera can be chosen. the
image of the model appears on device screen in preview mode.
When user selects the 3D real-time display option, the image is superimposed over the image of the
surrounding space (environment) in the smartphone camera, allowing the user to see the model in the
field of view of the device's camera.
The user can move and scale the model. The principle of operation of the proposed in this work
information technology for 3D objects models visualization in augmented reality is shown in Figure 6.
Figure 5: Algorithm of 3D models of objects rendering in augmented reality
Figure 6: The principle of the software and technical tool operation for the reproduction of
objects 3D models in augmented reality.
The proposed information technology for 3D objects models visualization in augmented reality
consists of Data Storage and Software System. Data Storage is the part where the models are stored.
Users can upload their own models there.
Software System is divided by two parts - a user side part – i.e Frontend part where user can interact
with program interface, choose and preview the model for visualization; and server-side part – i.e
Backend part which is responsible for saving a new uploaded by user model into the data storage,
validation of the chosen by user model in AR and its visualization using device camera. The structure
of the proposed information technology for 3D objects models visualization in augmented reality is
presented in Fig.7.
Figure 7: The structure of the proposed information technology for 3D objects models
visualization in augmented reality
5. Results & Discussion
For conducting the experiments a test 3D model of “Solar Tree construction” was developed. For
development Blender 3D modelling Environment was used. The model is presented in Figure 8.1. Also
a script for visualization of the test model was developed. We can upload the model in software database
in .glb or .obj extension and have it available for the visualization. The script for the model visualization
was transferred to Android-based smartphone with AR function support for conducting the experiment.
The experiment on the proposed model visualization in real time in augmented reality has been
conducted. As an environment for the visualization of outdoor art object the Botanical Garden of
Khmelnytskyi National University was chosen. During the experiment the test model was visualized in
AR through smartphone camera in the external environment and a photo from camera view was taken.
The photo with the experiment results is presented in Fig.8.2.
As the experiment proved, the developed software works quite well and performs its functions. The
test model was displayed on the smartphone screen and was superimposed over the image of the
surrounding space using smartphone camera.
The further efforts of the authors will be aimed at:
1) creating more test models and conduct the experiments on visualization 3D models for different
purposes and in various environments;
2) developing user interface of the proposed information system for objects 3D models
visualization in augmented reality in the form of cross-platform mobile application.
3) application of the developed tool for scientific and real-life needs, such as digitization and
visualization of museum exhibits and archaeological artifacts.
Figure 8.1: 3D model for Figure 8.2: The results of the experiments on 3D objects models
visualization in AR visualization in augmented reality
6. Conclusions
The current trends and the prospective of Augmented Reality technology application for various
purposes are considered. The literature analysis and the analysis of the related works showed that the
interest to AR technology is present both from scientific and business sides. The industries where
augmented reality is currently being implemented the fastest and in the largest volume are science,
medicine and e-commerce/advertising.
The literature review provided the conclusion that currently there are no technical means that provide
visualization of three-dimensional models of objects in augmented reality with the ability of the user to
add their own models. Therefore, creating of such information technology is an urgent task.
AR using trends research has shown that the most effective form for the proposed information
technology implementation is the form of a cross-platform mobile application which is available both
Android and iOS users.
In this study, a mathematical model for visualizing an object in three-dimensional space, a method
and algorithm for visualizing an object model in augmented reality, as well as a structural diagram and
principle of operation of the proposed information technology were developed. A test three-dimensional
model was also developed and experiments were conducted on its visualization in augmented reality
using the proposed information technology. Experiments have shown that the prototype of the
developed information technology works quite well. Therefore, directions for further research are
developing user interface of the proposed information system for objects 3D models visualization in
augmented reality in the form of cross-platform mobile application and its application for scientific and
real-life needs, such as digitization and visualization of museum exhibits and archaeological artifacts.
7. References
[1] Mobile augmented reality (AR) market revenue worldwide from 2021 to 2026 URL:
https://www.statista.com/statistics/282453/mobile-augmented-reality-market-size/ (Last accessed
11.02.2023).
[2] Pavlova O., Bashta A., Kravchuk S., Hnatchuk Y., Bouhissi H.E. Augmented Reality Based
Technology and Scenarios for Route Planning and Visualization. CEUR Workshop Proceedings,
2022, 3156, pp. 613–623
[3] Sanii B. Creating Augmented Reality USDZ Files to Visualize 3D Objects on Student Phones in
the Classroom. Journal of Chemical Education 2020 97 (1), 253-257 DOI:
10.1021/acs.jchemed.9b00577
[4] Gherardini F. 3D Virtual Reconstruction and Augmented Reality Visualization of Damaged Stone
Sculptures. IOP Conference Series: Materials Science and Engineering, Volume 364, Florence
Heri-Tech - The Future of Heritage Science and Technologies 16–18 May 2018, Florence, Italy
DOI: 10.1088/1757-899X/364/1/012018
[5] Plunkett, K. N. A Simple and Practical Method for Incorporating Augmented Reality into the
Classroom and Laboratory. Journal of Chemical Education 2019 DOI:
10.1021/acs.jchemed.9b00607
[6] Marques L. et al. Cultural Heritage 3D Modelling and visualisation within an Augmented Reality
Environment, based on Geographic Information Technologies and mobile platforms. ACE:
Architecture, City and Environment, 11 (33): 117-136, 2017. DOI:10.5821/ace.11.33.4686. ISSN:
1886-4805.
[7] Wolle P., Müller P., Rauh D. Augmented Reality in Scientific Publications—Taking the
Visualization of 3D Structures to the Next Level. ACS Chemical Biology 2018 13 (3), 496-499
DOI: 10.1021/acschembio.8b00153
[8] Reipschläger P. and Dachselt R. 2019 DesignAR: Immersive 3D-Modeling Combining
Augmented Reality with Interactive Displays. In Proceedings of the 2019 ACM International
Conference on Interactive Surfaces and Spaces (ISS '19). Association for Computing Machinery,
New York, NY, USA, pp. 29–41. https://doi.org/10.1145/3343055.3359718
[9] González I., Méndez J.A., Palomera R. et al. Applications of Virtual and Augmented Reality in
Biomedical Imaging. Journal of Medical Systems 43, 102 (2019). https://doi.org/10.1007/s10916-
019-1239-z
[10] Carter D. Immersive Employee Experiences in the Metaverse: Virtual Work Environments,
Augmented Analytics Tools, and Sensory and Tracking Technologies. Psychosociological Issues
in Human Resource Management; Woodside10 (1), 2022, pp. 35-49.
DOI:10.22381/pihrm10120223
[11] Kye1 B., Han N., Kim E., Park Y., Jo S. Educational applications of Metaverse: possibilities and
limitations. J Educ Eval Health Prof 2021, 18(32). DOI: https://doi.org/10.3352/jeehp.2021.18.32
[12] Mitii I., & Soloviov V. (2018). Augmented Reality: Ukrainian Present Business and Future
Education. Educational Dimension, 51, pp. 290-296.
[13] Babkin V., et al. Using augmented reality in university education for future IT specialists:
educational process and student research work. CEUR Workshop Proceedings, 2021.
[14] Nechypurenko P., Stoliarenko V., Starova T., Selivanova T., Markova O., Modlo Y., & Shmeltser
E. (2020). Development and implementation of educational resources in chemistry with elements
of augmented reality.
[15] Google AR/VR Components. URL:https://arvr.google.com/ (Last accessed 14.12.2022).
[16] Hovorushchenko T., Pavlova O., & Kobel K. Method of Evaluating the User Interface of Software
Systems for Compliance with the Gestalt Principles. In 2019 IEEE 14th International Conference
on Computer Sciences and Information Technologies (CSIT) Vol. 2, pp. 138-141
[17] Augmented Reality Trends to Redefine Business Growth in 2023. URL:
https://www.intelivita.com/blog/augmented-reality-trends/ (Last accessed 14.12.2022).
[18] Augmented Reality Trends of 2023: New Milestones in Immersive Technology. URL:
https://mobidev.biz/blog/augmented-reality-trends-future-ar-technologies (Last accessed
14.02.2022).
[19] Sasaki R., Yamamoto K. A Sightseeing Support System Using Augmented Reality and Pictograms
within Urban Tourist Areas in Japan. ISPRS Int. J. Geo-Inf. 2019. 8, 381.
https://doi.org/10.3390/ijgi8090381
[20] C. C. Ho, M. -C. Ho and C. -Y. Chang. Markerless Indoor/Outdoor Augmented Reality Navigation
Device Based on ORB-Visual-Odometry Positioning Estimation and Wall-Floor-Boundary Image
Registration. 2019 Twelfth International Conference on Ubi-Media Computing (Ubi-Media). Bali,
Indonesia. 2019. pp. 199-204. doi: 10.1109/Ubi-Media.2019.00046