=Paper=
{{Paper
|id=Vol-2547/paper17
|storemode=property
|title=Augmented Reality-based historical guide for classes and tourists
|pdfUrl=https://ceur-ws.org/Vol-2547/paper17.pdf
|volume=Vol-2547
|authors=Yaroslav M. Krainyk,Anzhela P. Boiko,Dmytro A. Poltavskyi,Vladimir I. Zaselskiy
|dblpUrl=https://dblp.org/rec/conf/aredu/KrainykBPZ19
}}
==Augmented Reality-based historical guide for classes and tourists==
https://ceur-ws.org/Vol-2547/paper17.pdf
241
Augmented Reality-based historical guide for classes and
tourists
Yaroslav M. Krainyk[0000-0002-7924-3878], Anzhela P. Boiko[0000-0002-3449-0453],
Dmytro A. Poltavskyi
Petro Mohyla Black Sea National University, 10, 68 Desantnykiv Str., Mykolaiv, 54000,
Ukraine
yaroslav.krainyk@chmnu.edu.ua, anzhela.boiko@chmnu.edu.ua,
di77929@gmail.com
Vladimir I. Zaselskiy
Kryvyi Rih Metallurgical Institute of the National Metallurgical Academy of Ukraine,
5, Stepana Tilhy Str., Kryvyi Rih, 50006, Ukraine
zaselskiy52@gmail.com
Abstract. In this paper, development of historical guide based on Augmented
Reality (AR) technology is considered. The developed guide application it
targeted to be used in different scenarios, in particular, during history learning
classes, for guidance of the tourists to exhibits both indoor and outdoor. Common
features of all these scenarios are generalized and according to them main
information and objects model for forming scene are identified. This part is
followed by detailed description of objects and scene representation, markers
usage, employment of additional services, etc. Finally, the developed historical
guide application has been introduced. It harnesses A-Frame library for
processing of models and their representation. The application is able to work
with different markers so that it can be extended easily. In addition, one of the
main benefits of the developed application is support of multiple platforms
because it works from web-browser and does not require installation of additional
software. The developed application can be effectively used for all provided
scenarios and has potential for further extension.
Keywords: augmented reality, historical guide, class, model, application,
information.
1 Introduction
Augmented Reality (AR) technology has managed to become mainstream technology
in a short period. Many of new devices on the market are powerful enough to support
the technology. There are also huge demands and expectations from this technology at
the customers’ side. One of the most promising fields of use for AR is educational
technology. Possibilities to enrich learning experience with AR seem to be almost
___________________
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
242
unlimited. It relates to all age categories and all fields of knowledge. The major
attractive point of AR is high level of interactivity that allows communicating with AR-
objects.
The great advantage of AR is a huge visual experience for users. It is especially
effective for the subjects with massive amount of visual data for presentation. From the
student’s point of view, visual data is much more involving and comprehensible than
other types of information. Moreover, AR allows student to participate actively in the
educational process.
One of the good candidates for application of AR is historical content and all
historical-related activities. It can be learning history at university, automated guide
over historical places, interactive application for museums, etc.
2 Overview of scientific sources and AR-tools
It can be alleged that AR has a huge potential in educational sphere [20]. Modern
researches emphasize development of complex learning environment based on AR
[10]. However, another approach supposes that each learning subject has its own
peculiarities and they should be treated in context of the subject. This is the reason why
specialized tools appear for learning chemistry [12; 13], physics [8] etc. In fact, they
are easier from the design point of view because they do not require to cover multiple
learning scenarios and can limit student to usage of only a few tools within educational
framework [6; 9; 11; 14; 15; 16; 17; 19; 22]. It has advantage for both students and
lectures as clearly identifies activities that include AR. Hence, in the proposed work we
design AR-application for historical education with precise cases for its usage.
Many new libraries for AR-development has appeared in the recent time. Let us
analyze most popular among them.
ARKit [2] is an AR-platform designed specifically for Apple devices. It procures
great user experience for AR-applications. ARKit support various objects for tracking
(posters, images, etc.) which means that requirements for the marker view is much
softer. However, the obvious drawback is that ARKit available only on vendor’s
devices and cannot be used on other platforms.
At the other side, ARCore [5] and Vuforia [23] software development kits (SDKs)
are much more flexible and support deployment on most popular mobile platforms.
They are a great choice for developers who are going to implement application for
multiple platforms. In terms of services, Vuforia is advanced SDK as it has integration
with cloud. However, Vuforia inserts watermarks in the content if you use it for free
and has limitation on performed requests, while ARCore is an open-source and provides
free license.
All previously mentioned libraries have one feature in common. Their work is based
on some native application programming interface (API). Therefore, application for
one platform cannot be installed directly on the other one. More universal approach
leverages capabilities of web-browsers installed in all platforms. Code executed in the
browser is instantly available to any device. Developer needs to guarantee correct
deployment on the server and qualitative implementation of client-side functions.
243
A-Frame [1] is an open-source library for Virtual Reality (VR) and AR (by including
side-project library). This is a JavaScript library and it works out of the browser. Thus,
A-Frame is not bound to the specific platform and can be launched on almost any device
that has necessary computational ability. Because A-Frame is open-source and requires
no fee payment from developers, it is a perfect choice for educational projects that
leverage AR in their functionality.
So, originally, A-Frame is supposed to work with VR-facilities. To map
functionality of the library to AR, it is necessary to include one additional library
available from [4]. It enables markers, processing image from camera, and other basic
features of AR.
One more library works behind the curtain of A-Frame. three.js [21] library serves
as a basic software for A-Frame. three.js is fully-featured library in terms of VR-
operations. However, it heavily relies on scripting and requires that all functions have
to be declared and implemented in code. That detail can make code developed with
three.js cumbersome and demands more time to implement all tasks from specification.
In comparison with three.js, A-Frame provides many new components (in form of html-
tags) that greatly simplify development process and allow organizing all elements on
the scene in a structural way. Therefore, A-Frame can be considered as a wrapper
around basic library but it greatly extends initial version in various terms. It clearly
decouples declaration of element from implementation of its actions. Additionally, it
binds default functionality set to the object even when developer wrote no script code
for actions.
In this work, we develop AR-application that can be used as a tool in learning history
and as a tourist guide. We identify differences for these cases in context of application
usage and AR-involvement. A-Frame library has been chosen as a base platform for
application implementation due to its free license, automatic support of multiple
platforms, and rich set of components and functions that speed up and simplify
development process.
3 Results and discussion
It can be assumed that application under development has three categories of users:
1. Students of historical classes.
2. Tourists who visit indoor exhibitions (e.g. in museums).
3. Tourists who visit outdoor locations and exhibits possibly located on a notable
distance from each other.
All these groups should be treated differently by application in terms of content
presentation, location tracking, and selection of models to form the scene, etc. Students
are going to work in group while tourist will use application individually in most cases.
Estimated time that student spends with application is bigger than tourist does.
Consequently, we suggest that for classes’ purposes scene should contain more
activities and provide more factual information.
244
In general, models of objects should be close to the outer view of the original objects.
However, approaches for content presentation are quite different. For the historical
classes, historical facts must be emphasized and established for learning. Two
remaining options may contain some entertaining content to seize attention of the user
to the current item or maybe to the other items from the list. But it does not prohibit
entertaining entities (e.g. animated character) to be present in the educational content
as long as they support it and enrich user experience.
For the first group, they always stay within the same classroom. In the second case,
application users move from one room to another. Thus, indoor navigation will be
sufficient for this task. The last group of users can be tracked via the means of Global
Positioning System (GPS).
AR-based historical guide performs the following functions:
1. Provides visual representation of historical object, dynamic scenes, etc.
2. Reaction to user input actions and provides dynamic behavior of the object.
3. Facilitates additional information about the presented entity to the user.
4. Switching between different objects in catalogue.
5. Location of the object in the museum, park, etc. on a map or by other means
(optional).
6. Animation of the object in context of one or several scenes (optional).
7. Guides user to the object in interactive manner (optional).
The function list implies permanent presence or temporarily appearance of the
following object types in the scene:
historical object itself;
textual description of the object;
graphical information about object location.
As A-Frame library requires markers for its work, it is necessary to provide markers
and bring them into the scene. Historical guide is able to display different objects;
therefore, two options can be used. The first one is to provide multiple markers and
switch among them manually. Booklet with set of markers seems a feasible solution for
this case because it is compact but still provides information to the application. The
second option is to implement switching directly in the application. It implies presence
of additional object in the application, menu. The user can select object he or she is
interested in and processing engine will treat marker as a starting point for different
object.
For the development of complex scenes, presence of multiple markers is required.
They must be captured and tracked simultaneously to provide better user experience.
Hence, placement of the markers on the surface and scenario for user actions should be
worked through carefully. Multiple objects on the scene make it more dynamic.
However, in context of learning history, those objects have to relate to each other, have
some comparable feature to analyze, so the overall scene will have sense and represent
part of historical knowledge.
245
Textual information that accompanies the object on the scene has an aim to provide
brief description about object shown. Its location on the screen depends on the current
point of view selected by the user. It should stay readable regardless of user actions
specifically connected with object rotation. From the implementation perspective, such
textual model can be represented as text object in the A-Frame library. It is also a good
practice to give user choice to turn off visibility of the description object.
Location information in the AR-historical guide depends on what institution actually
uses it. If it is application for tourists, it shows path to attractions via card service (e.g.
Google Maps). On the other side, application for museum does not need such
functionality and in-building navigation is preferable in this case. User can select
exhibit he/she interested in and make use of markers placed inside the museum.
Depending on the selected item, arrows that show direction or next step on the route
will be shown. Thus, route to the exhibit is constructed dynamically and guides user to
the necessary location inside the building. Finally, for the history classes at school or
university, routing to the object is not necessary and brief static information about
object location is enough. The location block should be placed beside the main object
on the scene. However, we can employ different approach to represent place of the
object by using interchangeable surrounding assets. For instance, if object is located in
snow region, the object itself is surrounded by snow. In opposite, if object relates to the
warm region, it might be demonstrated in the scene with desert environment and
appropriate landscape.
One of the most indispensable features of AR-application is active participation of
the user and ability to respond to user actions. In general, we can suppose that device
with deployed program contains touch screen and sensors to react on user’s movements
and gestures. By default, the user observes front side of the object. The user is able to
change perspective in two ways:
1. By gestures sensed by inertial sensors (accelerometer, gyroscope).
2. By gestures received from touch screen.
Both of them can be considered as primary actions to control the scene.
A-Frame supports several formats of object models. glTF format is recommended
by A-Frame documentation as format for WebVR. Nonetheless, during development
of historical guide application, it is possible to convert format of the file to the required
one using modeling environments such as Blender. The main issue concerned with
models is to obtain models of historical objects. Complex models may contain as much
as several hundred thousand vertices and go with multiple textures. Design of precise
copies of objects is very time-consuming task that will take much more time than
application development. Therefore, it would be a better choice to find freely available
models on services like Sketchfab [18] or order models from designer. Downloading
models from the specialized websites puts additional constraints because you may not
find the object you are looking for and will include some similar object or have to refuse
from the demonstration of the object.
In general, A-Frame proposes traditional web-site architecture for the purposes of
AR-application. User sends request to retrieve corresponding resource from server and
works with it. It is preferably to direct user to single page and do not switch from this
246
page as it requires more time. It is obligatory for the user to grant access for camera
usage for the page. When page receives this access level, it can act. The next step to do
for the user is to provide marker(s) into the scene and direct camera at the marker(s).
After that moment, all previously mentioned scenarios and tools take place to
demonstrate scene.
The following part of the page markup is used in description of the scene:
...
The outer element defines the scene and sets support for AR-capabilities via web-
camera-support. Inner elements of the scene are list of assets (only the first one is shown
while the rest are omitted), marker, and camera. The models are stored in the models
directory and for performance reasons models in glTF-format are applied for
demonstration application.
Let us provide demonstration of software developed according to the proposed
model background and functional requirements for history learning AR-environment.
For demonstration purposes, the application has been deployed on the local machine.
In Fig. 1, starting scene with no selected model is shown.
The menu appears over top of the image captured from camera and is always
available to change the model. As one of the option from the list is selected,
corresponding model appears at the marker place. In this case, the first model
corresponds to pearl monument as well as second model is represented by castle object
as demonstrated in Fig. 2.
Besides presence of the model, textual description for the model is provided (in this
case, text does not contain information about the model, it just performs placeholder
function).
Once again, this fictional model does not correspond to any known architectural
form. However, it proves that even such large-scale buildings like castles are suitable
for demonstration in AR-guide even on mobile device.
247
Fig. 1. Screen with open option menu and camera directed to the marker
Fig. 2. Demonstration of the first model
Despite demonstration figures have been retrieved in the local infrastructure, the
application code is available from Github repository [7]. It is also deployed at the
248
github.io [3] for fast acquaintance with site and can be launched from every mobile
device with camera that provides resolution sufficient for marker recognition and has
internet access. We also make available for use and immediate testing marker for the
application that is a Fig. 3.
Fig. 3. Marker for application
During the testing stage, several artifacts have been observed in the application work.
First, it is required for the camera image provide distinguishable image of the marker.
Otherwise, the model disappears. Secondly, the speed of processing for browser
application even in the local network is lower than for native ones. We also experienced
problems with launching site directly from the deployed location. Hence, possible
solution for this problem is to pull source code from repository and run it from local
environment.
The further development of the project is concerned with addition of new model into
the application and improvements from the interactivity point of view.
4 Conclusions
In the presented paper, we established AR-based application that can be used in context
of effective presentation of historical materials and history-related content. The
application is meant to be used for historical classes and as a mobile tourist guide. We
have provided detailed description of the system and all of its components. The entities
used in application have been thoroughly inspected and functionality of each of them
is identified. The developed application is based on the A-Frame JavaScript library that
works from the browser. Hence, application is available to almost all device types that
have web-browser installed and support capturing image from camera.
249
References
1. A-Frame – Make WebVR. https://aframe.io (2019). Accessed 21 Mar 2019
2. Apple: Augmented Reality – Apple Developer. https://developer.apple.com/augmented-
reality (2020). Accessed 17 Jan 2020
3. AR. https://dmitriypoltavskiy.github.io/AR-project (2019). Accessed 14 Feb 2019
4. AR.js - Augmented Reality for the Web. https://jeromeetienne.github.io/AR.js (2019).
Accessed 28 Nov 2019
5. ARCore – Google Developers. https://developers.google.com/ar (2019). Accessed 21 Mar
2019
6. Buzko, V.L., Bonk, A.V., Tron, V.V.: Implementation of Gamification and Elements of
Augmented Reality During the Binary Lessons in a Secondary School. In: Kiv, A.E.,
Soloviev, V.N. (eds.) Proceedings of the 1st International Workshop on Augmented Reality
in Education (AREdu 2018), Kryvyi Rih, Ukraine, October 2, 2018. CEUR Workshop
Proceedings 2257, 53–60. http://ceur-ws.org/Vol-2257/paper06.pdf (2018). Accessed 30
Nov 2018
7. DmitriyPoltavskiy/AR-project. https://github.com/DmitriyPoltavskiy/AR-project (2019).
Accessed 12 Feb 2019
8. Hruntova, T., Yechkalo, Y., Striuk, A., Pikilnyak, A.: Augmented Reality Tools in Physics
Hruntova, T.V., Yechkalo, Yu.V., Striuk, A.M., Pikilnyak, A.V.: Augmented Reality Tools
in Physics Training at Higher Technical Educational Institutions. In: Kiv, A.E., Soloviev,
V.N. (eds.) Proceedings of the 1st International Workshop on Augmented Reality in
Education (AREdu 2018), Kryvyi Rih, Ukraine, October 2, 2018. CEUR Workshop
Proceedings 2257, 33–40. http://ceur-ws.org/Vol-2257/paper04.pdf (2018). Accessed 30
Nov 2018
9. Kolomoiets, T.H., Kassim, D.A.: Using the Augmented Reality to Teach of Global Reading
of Preschoolers with Autism Spectrum Disorders. In: Kiv, A.E., Soloviev, V.N. (eds.)
Proceedings of the 1st International Workshop on Augmented Reality in Education (AREdu
2018), Kryvyi Rih, Ukraine, October 2, 2018. CEUR Workshop Proceedings 2257, 237–
246. http://ceur-ws.org/Vol-2257/paper24.pdf (2018). Accessed 30 Nov 2018
10. Lytridis, C., Tsinakos, A.: Evaluation of the ARTutor augmented reality educational
platform in tertiary education. Smart Learning Environments 5, 6 (2018).
doi:10.1186/s40561-018-0058-x
11. Merzlykin, O.V., Topolova, I.Yu., Tron, V.V.: Developing of Key Competencies by Means
of Augmented Reality at CLIL Lessons. In: Kiv, A.E., Soloviev, V.N. (eds.) Proceedings
of the 1st International Workshop on Augmented Reality in Education (AREdu 2018),
Kryvyi Rih, Ukraine, October 2, 2018. CEUR Workshop Proceedings 2257, 41–52.
http://ceur-ws.org/Vol-2257/paper05.pdf (2018). Accessed 30 Nov 2018
12. Nechypurenko, P.P., Starova, T.V., Selivanova, T.V., Tomilina, A.O., Uchitel, A.D.: Use
of Augmented Reality in Chemistry Education. In: Kiv, A.E., Soloviev, V.N. (eds.)
Proceedings of the 1st International Workshop on Augmented Reality in Education (AREdu
2018), Kryvyi Rih, Ukraine, October 2, 2018. CEUR Workshop Proceedings 2257, 15–23.
http://ceur-ws.org/Vol-2257/paper02.pdf (2018). Accessed 30 Nov 2018
13. Nechypurenko, P.P., Stoliarenko, V.G., Starova, T.V., Selivanova, T.V., Markova, O.M.,
Modlo, Ye.O., Shmeltser, E.O.: Development and implementation of educational resources
in chemistry with elements of augmented reality. In: Kiv, A.E., Shyshkina, M.P. (eds.)
Proceedings of the 2nd International Workshop on Augmented Reality in Education
(AREdu 2019), Kryvyi Rih, Ukraine, March 22, 2019, CEUR-WS.org, online (2020, in
press)
250
14. Pinchuk, O.P., Sokolyuk, O.M., Burov, O.Yu., Shyshkina, M.P.: Digital transformation of
learning environment: aspect of cognitive activity of students. In: Kiv, A.E., Soloviev, V.N.
(eds.) Proceedings of the 6th Workshop on Cloud Technologies in Education (CTE 2018),
Kryvyi Rih, Ukraine, December 21, 2018. CEUR Workshop Proceedings 2433, 90–101.
http://ceur-ws.org/Vol-2433/paper05.pdf (2019). Accessed 10 Sep 2019
15. Popel, M.V., Shyshkina, M.P.: The Cloud Technologies and Augmented Reality: the
Prospects of Use. In: Kiv, A.E., Soloviev, V.N. (eds.) Proceedings of the 1st International
Workshop on Augmented Reality in Education (AREdu 2018), Kryvyi Rih, Ukraine,
October 2, 2018. CEUR Workshop Proceedings 2257, 232–236. http://ceur-ws.org/Vol-
2257/paper23.pdf (2018). Accessed 30 Nov 2018
16. Rashevska, N.V., Soloviev, V.N.: Augmented Reality and the Prospects for Applying Its in
the Training of Future Engineers. In: Kiv, A.E., Soloviev, V.N. (eds.) Proceedings of the
1st International Workshop on Augmented Reality in Education (AREdu 2018), Kryvyi
Rih, Ukraine, October 2, 2018. CEUR Workshop Proceedings 2257, 192–197. http://ceur-
ws.org/Vol-2257/paper18.pdf (2018). Accessed 30 Nov 2018
17. Shapovalov, Ye.B., Bilyk, Zh.I., Atamas, A.I., Shapovalov, V.B., Uchitel, A.D.: The
Potential of Using Google Expeditions and Google Lens Tools under STEM-education in
Ukraine. In: Kiv, A.E., Soloviev, V.N. (eds.) Proceedings of the 1st International Workshop
on Augmented Reality in Education (AREdu 2018), Kryvyi Rih, Ukraine, October 2, 2018.
CEUR Workshop Proceedings 2257, 66–74. http://ceur-ws.org/Vol-2257/paper08.pdf
(2018). Accessed 30 Nov 2018
18. Sketchfab - Publish & find 3D models online. https://sketchfab.com (2020). Accessed 17
Jan 2020
19. Striuk, A.M., Rassovytska, M.V., Shokaliuk, S.V.: Using Blippar Augmented Reality
Browser in the Practical Training of Mechanical Engineers. In: Ermolayev, V., Suárez-
Figueroa, M.C., Yakovyna, V., Kharchenko, V., Kobets, V., Kravtsov, H., Peschanenko,
V., Prytula, Ya., Nikitchenko, M., Spivakovsky A. (eds.) Proceedings of the 14th
International Conference on ICT in Education, Research and Industrial Applications.
Integration, Harmonization and Knowledge Transfer (ICTERI, 2018), Kyiv, Ukraine, 14-
17 May 2018, vol. II: Workshops. CEUR Workshop Proceedings 2104, 412–419.
http://ceur-ws.org/Vol-2104/paper_223.pdf (2018). Accessed 30 Nov 2018
20. Syrovatskyi, O.V., Semerikov, S.O., Modlo, Ye.O., Yechkalo, Yu.V., Zelinska, S.O.:
Augmented reality software design for educational purposes. In: Kiv, A.E., Semerikov,
S.O., Soloviev, V.N., Striuk, A.M. (eds.) Proceedings of the 1st Student Workshop on
Computer Science & Software Engineering (CS&SE@SW 2018), Kryvyi Rih, Ukraine,
November 30, 2018. CEUR Workshop Proceedings 2292, 193–225. http://ceur-ws.org/Vol-
2292/paper20.pdf (2018). Accessed 21 Mar 2019
21. three.js – JavaScript 3D library. https://threejs.org (2019). Accessed 28 Nov 2019
22. Tkachuk, V.V., Yechkalo, Yu.V., Markova, O.M.: Augmented reality in education of
students with special educational needs. In: Semerikov, S.O., Shyshkina, M.P. (eds.)
Proceedings of the 5th Workshop on Cloud Technologies in Education (CTE 2017), Kryvyi
Rih, Ukraine, April 28, 2017. CEUR Workshop Proceedings 2168, 66–71. http://ceur-
ws.org/Vol-2168/paper9.pdf (2018). Accessed 21 Mar 2019
23. Vuforia: Market-Leading Enterprise AR | PTC.
https://www.ptc.com/en/products/augmented-reality/vuforia (2020). Accessed 17 Jan 2020