=Paper=
{{Paper
|id=Vol-3865/08_paper
|storemode=property
|title=Digital Augmented Reality App for Historic Architecture (short paper)
|pdfUrl=https://ceur-ws.org/Vol-3865/08_paper.pdf
|volume=Vol-3865
|authors=Olena Shlyakhetko,Iryna Ivanochko,Natalia Kryvinska
|dblpUrl=https://dblp.org/rec/conf/aiia/ShlyakhetkoIK24
}}
==Digital Augmented Reality App for Historic Architecture (short paper)==
https://ceur-ws.org/Vol-3865/08_paper.pdf
Digital Augmented Reality App for Historic Architecture⋆
Olena Shlyakhetko1,†, Iryna Ivanochko1,2,∗,† and Natalia Kryvinska1, †
1
Comenius University Bratislava, 82005 Bratislava, Slovakia
2
Lviv Polytechnic National University, 79000 Lviv, Ukraine
Abstract
This paper aims to create an augmented reality application to reconstruct damaged historical buildings. We
have developed a mobile Augmented Reality (AR) application for Android devices that offers a new way to
experience historic architecture. The app uses QR code scanning and GPS localization to display the
reconstructed monument at the original location of the building. We have designed the app to cater to three
types of users: non-logged-in users, logged-in users, and administrators. Users can explore and rate
historical sites, providing valuable feedback to our development team. We have rigorously tested the app
and engaged with users, and we believe it will be an effective tool for historical education and a model for
future AR applications in different contexts. Our goal is to make the app user-friendly and easy to navigate,
even for individuals who are not familiar with smartphones. Additionally, we plan to introduce a feature
that locks the user's phone while the app is in use, allowing museums to offer phone rentals for visitors
who do not have smartphones.
Keywords
Augmented Reality, virtual reality, mobile application, 3D reconstruction, cultural heritage, QR code 1
1. Introduction
Regarding technology, both virtual Reality (VR) and Augmented Reality (AR) have their merits. VR
creates a digital version of a real-world setting, while AR overlays virtual objects onto the real world
[1]. After careful consideration, we have determined that AR is the superior choice for our
application. Its more straightforward implementation, lower system costs, and minimal need for
additional hardware make it a more practical solution. Furthermore, the potential discomfort and
balance issues that some users, particularly the elderly, may experience with VR [2] could pose safety
concerns for our application. By choosing AR, we can ensure a more inclusive and user-friendly
experience. Unlike virtual reality, augmented reality enriches the real world with virtual objects,
which creates a wholly digital environment around the user. While older AR systems require bulky
hardware and software, the evolution of mobile technology has made it possible to experience
augmented reality on the go. This shift has significantly increased the accessibility of AR, making it
a viable and user-friendly option for our application. In this thesis, we present the design of a mobile
AR application for Android devices. The app will allow users to view 3D reconstructions of
historically significant buildings that no longer exist. Users can see the reconstructed monument on
their mobile phone screens by scanning a QR code using the back camera. Additionally, we will
implement GPS to display the reconstructed monument at the original location of the building. This
feature will provide a more immersive experience for users. Unlike other attempts that require
additional hardware [3], our advantage is that users already have all the necessary hardware in their
pockets, making the project more cost-effective.
3rd Workshop on Artificial Intelligence for Cultural Heritage (AI4CH 2024, https://ai4ch.di.unito.it/), co-located with the 23nd
International Conference of the Italian Association for Artificial Intelligence (AIxIA 2024). 26-28 November 2024, Bolzano, Italy
∗
Corresponding author - Natalia Kryvinska.
†
These authors contributed equally.
olenaivanochko@gmail.com (O. Shlyakhetko); iryna.ivanochko@fm.uniba.sk (I. Ivanochko);
natalia.kryvinska@fm.uniba.sk (N. Kryvinska)
0009-0000-7156-4397 (O. Shlyakhetko); 0000-0002-1936-968X (I. Ivanochko); 0000-0003-3678-9229 (N. Kryvinska)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
A few applications serve similar purposes [4], but none allow the user to enter the projected building.
Our application will make this possible. This means our monument models must be more detailed
and have proper interiors. Another challenge is making GPS locations highly accurate because
people will move small distances inside the reconstruction [5]. We also want users to be able to rate
the places they visit and optionally add them to their favorites list. This will serve as feedback for us,
letting us know what people are most interested in and also letting other users know what is worth
seeing.
2. Model building
The application requires input from a QR code scanned by a phone camera and GPS localization [6].
Once the code is scanned, our application will display the corresponding model and sound of the
exhibition on mobile phones (Android/iOS). Depending on customer needs and resources, the
museum hosts or other servers will provide the application's server-side.
- System requirements
o For the client-side user will need an Android 4.0+ or an ios system phone
o For the server-side user will need to provide the HTTP server with models and
sounds
o Application will be available from Google Play and App Store
- Application requirements
o The application will need a phone with GPS and a camera
The class diagram illustrates the program's class layers and their relationships. These class layers
were created to adhere to Spring design principles. There are three primary layers: model, repository,
and controller, as well as a view layer. All repositories and controllers follow the same generic class
pattern.
Figure 1: Class Diagram
We have designed the application to accommodate three types of users: non-logged-in users, logged-
in users, and administrators. Each role has access to different actions. For example, an administrator
can check user accounts, objects, and personal accounts.
Figure 2: User use case diagram
Figure 3: Not logged user use case diagram
Figure 4: Admin use case diagram
3. Model testing practical part
3.1. Test cases
Test case 1. Reconstruction of the textile spinning machine
requirements:
‒ smartphone with working GPS module
‒ newest version of the application installed
scenario: The user opens the application and picks the „see reconstruction” button in the interface.
The phone's back camera will become active, and the user will be asked to scan the QR code on the
podium near the exhibition. After that, guidelines will appear to help the user finish reconstructing
the object. The application will use the GPS module to guide the user to the proper location and
where to point their device. After a brief moment, the application will present an image on the phone.
result: the 3D image of the reconstructed textile spinning machine appears on the screen.
Test case 2. Reconstruction of the watchtower
requirements:
‒ smartphone with working GPS module
‒ newest version of the application installed
scenario: The user opens the application and picks the „see reconstruction” button in the interface.
The phone's back camera will become active, and the user will be asked to scan the QR code on the
podium near the exhibition. After that, guidelines will appear to help the user finish reconstructing
the object. The application will use the GPS module to guide the user to the proper location and
where to point their device. After a brief moment, the application will present an image on the phone.
From this moment, the user can walk inside the watchtower.
result: 3D image of reconstructed watchtower, both interior and exterior. appears on the screen.
Test case 3. Reconstruct the old manor house
requirements:
‒ smartphone with working GPS module
‒ newest version of the application installed
scenario: The user should go to the nearest manor house. The user opens the application and picks
the “see reconstruction” button in the interface. Users will be asked to scan the QR code on the
podium near the manor house. The user shall move the camera to where he was told in the
application {reconstructed object appears on the screen, the animations with the walking
environment are displayed, and sound is played}.
Test case 4. Changing reconstruction objects
requirements:
‒ smartphone with working GPS module
‒ newest version of the application installed
scenario: The user opens the application and picks the “see reconstruction” button in the interface.
Users will be asked to scan the QR code on the podium near the exhibition. The user shall move the
camera to the place where he was told in the application {reconstructed object appears on the screen}.
The user shall go to the next exhibition and click the "see reconstruction" button again; as in previous
steps, the user should move the camera to the place where the application displays {Other
reconstructed objects appear on the screen}.
Test case 5. Changing localization while observing objects.
requirements:
‒ smartphone with working GPS module
‒ newest version of the application installed
‒ rooted phone with fake GPS application {android only}
scenario: The user opens the application and picks the “see reconstruction” button in the interface.
Users will be asked to scan the QR code on the podium near the exhibition. Users should use fake
GPS applications to change localization to other places a minimum of 50 meters. {User should be told
about localization change and render process should be aborted, the main screen with "reconstruct"
button should be displayed}.
3.2. Application visualization for tests
Figure 5: QR-Code scanning [src] Figure 6: Reconstructed scene [src]
The development of our mobile Augmented Reality (AR) application [7, 8] offers a transformative
approach to experiencing historical architecture that has long been lost. By leveraging widely
accessible mobile technology, users can engage with 3D reconstructions of significant buildings
through a seamless interface that utilizes QR code scanning and GPS localization. The advantages of
AR over Virtual Reality – such as ease of implementation, reduced costs, and minimal hardware
requirements – position our application as a user-friendly alternative, particularly for those facing
VR technology challenges. Our application enhances the educational value of museum exhibits and
fosters user interaction by allowing individuals to explore and rate historical sites, thus contributing
valuable feedback to our development team. The architectural fidelity of the models, coupled with
immersive audio experiences, aims to provide a rich and engaging environment that invites users to
delve deeper into their cultural heritage. Through rigorous testing and user engagement, we
anticipate this application will be an effective tool for historical education and a model for future AR
applications in various contexts. This project will significantly enhance how individuals experience
and connect with their historical surroundings.
4. Conclusions
The modern-day reliance on 2D media has made these exhibits less engaging, especially for young
people, which could lead to a loss of our cultural heritage. To address this issue, we propose bringing
reconstructed exhibits to life in 3D with added sound for a more immersive experience. Our research
shows that the application should be user-friendly and easy to navigate, even for individuals
unfamiliar with smartphones. We plan to automate the application as much as possible by utilizing
GPS modules and QR codes. The GPS will help us identify the user's location, while QR codes will
ensure the accurate display of relevant information. This approach will make the application simple,
consistent, and easy to use. By allowing visitors to move around exhibits while experiencing the
sights and sounds of the past, we aim to help them remember and connect with history on a deeper
level as we provide more sensory stimuli. Our main challenge lies in creating new 3D models that
are readily available. However, we hope that with the full approval of museums, we can integrate
the cost of making these models into their exhibit budgets. Looking ahead, we plan to introduce a
feature that locks the user's phone while the application is in use, allowing museums to offer phone
rentals for visitors without smartphones. Additionally, we will explore alternative input sources for
our localization mechanism, such as using Wi-Fi instead of QR code scanning, to streamline the
application further.
Acknowledgements
Funded by the EU NextGenerationEU through the Recovery and Resilience Plan for Slovakia
under the project No. 09I03-03-V01-000121
References
[1] J.-T. Hwang i T.-C. Chu, 3D Building Reconstruction By Multiview Images And The Integrated
Application With Augmented Reality, Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., t.
XLI-B1, s. 1235–1241, paź. 2016, doi: 10.5194/isprs-archives-XLI-B1-1235-2016.
[2] C. Regan, An investigation into nausea and other side-effects of head-coupled immersive virtual
reality, Virtual Real., t. 1, nr 1, s. 17–31, cze. 1995, doi: 10.1007/BF02009710.
[3] Y. Liu i Y. Wang, AR-View: An augmented reality device for digital reconstruction of
Yuangmingyuan, International Symposium on Mixed and Augmented Reality - Arts, Media and
Humanities, Orlando, USA, paź. 2009, 3–7. doi: 10.1109/ISMAR-AMH.2009.5336752.
[4] C. Panou, L. Ragia, D. Dimelli, i K. Mania, „Outdoors Mobile Augmented Reality Application
Visualizing 3D Reconstructed Historical Monuments”:, w Proceedings of the 4th International
Conference on Geographical Information Systems Theory, Applications and Management,
Funchal, Madeira, Portugal, 2018, s. 59–67. doi: 10.5220/0006701800590067.
[5] B. Jach, Ł. Dłużniewski, Ł. Radosz, K. Likus, AR reconstruction of historical buildings. Seminar
Work, 2021.
[6] M. Hincapié, C. Díaz, M. I. Zapata-Cárdenas, H. Rios, D. Valencia, & D. Güemes-Castorena,
Augmented reality mobile apps for cultural heritage reactivation. Computers & Electrical
Engineering, 93, 2021,107281. https://doi.org/10.3390/ijgi7120463
[7] Hincapié, M., Díaz, C., Zapata-Cárdenas, M. I., Rios, H. D. J. T., Valencia, D., & Güemes-
Castorena, D. Augmented reality mobile apps for cultural heritage reactivation. Computers &
Electrical Engineering, 93, 2021, 107281.
[8] Angelopoulou, A., et al. Mobile augmented reality for cultural heritage. In Mobile Wireless
Middleware, Operating Systems, and Applications: 4th International ICST Conference,
Mobilware 2011, London, UK, 2012, pp. 15-22.