=Paper=
{{Paper
|id=Vol-3243/paper7
|storemode=property
|title=Visual Cues for Cultural Heritage Urban Navigation with Smart Glasses
|pdfUrl=https://ceur-ws.org/Vol-3243/paper7.pdf
|volume=Vol-3243
|authors=Tsvi Kuflik,Peter Yagodin,Omer Sharon
|dblpUrl=https://dblp.org/rec/conf/avi/KuflikYS22
}}
==Visual Cues for Cultural Heritage Urban Navigation with Smart Glasses==
https://ceur-ws.org/Vol-3243/paper7.pdf
Visual Cues for Cultural Heritage Urban Navigation
with Smart Glasses
Tsvi Kuflik1 , Peter Yagodin1 and Omer Sharon1
1
Information Systems Department, The University of Haifa, Haifa, Israel
Abstract
Mobile augmented reality technology offers new ways of supporting visitors to cultural heritage sites.
The services may include delivering information, navigation support and even gaze-based intuitive
interaction. Significant progress has been made in exploring the potential of these technologies for indoor
museum visits, as well as for the use of such technology outdoors. When considering technological
support in a cultural heritage visit, one of the main challenges is supporting visitors’ navigation. This
paper presents an initial prototype of smart glasses-based navigation in an urban cultural heritage visit.
We present the idea and a prototype that has yet to be experimented for usability and acceptance by the
users.
Keywords
Smart glasses-based navigation, Urban navigation support, Outdoor cultural heritage
1. Introduction
In recent years, as mobile smart devices are becoming widely available. Traditional audio guides
are being replaced by smartphone apps and even smart glasses. Augmented reality becomes a
possible tool allowing visitors to interact in an engaging way with sites and artefacts in their
vicinity and elicit information about them. These novel mobile guides present location-based
information by audio and notify about objects of interest as they become visible to the user [1, 2].
Using mobile augmented reality in domains such as tourism and specifically in cultural heritage,
has the potential to enhance the visitor’s experience by providing easy and intuitive interaction
while simply bringing information into the visitor field of view when the visitor is looking at
an object [3]. These systems can reactively or proactively [4, 5] augment a visitor’s view with
relevant CH information without the devices themselves becoming the focus of attention. They
allow real-world information to be sent, received, and viewed without requiring the visitor
to carry and simultaneously view a hand-held guide. Moreover, they can overlay historical
images and bring to life historical events overlaying them on the current view of a landscape
[6, 7] This technology is gaining interest in recent years due to its attractiveness and ease of use
[8, 9]. Recently, Litvak and Kuflik [10] demonstrated the potential of using augmented reality
smart glasses as a mobile tourist guide outdoors and noted a few challenges facing the wide
use of such devices. Among the experimented features was navigation support within a small
archaeological site. Following the positive response to the navigation support provided by the
AVI-CH 2022 Workshop on Advanced Visual Interfaces and Interactions in Cultural Heritage. June 06, 2022. Rome, Italy
Envelope-Open tsvikak@is.haifa.ac.il (T. Kuflik); piter95555@gmail.com (P. Yagodin); sharonomeros@gmail.com (O. Sharon)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
Table 1
Specifications of smartphone and smart glasses (taken from Litvak and Kuflik [10])
Parameter Smartphone Smart glasses
Battery life Up to 16 hours Up to 8 hours
Weight 165g 98g
AR implementation Hand-held, Video Head-worn, Optical
. FOV 5.5” screen size 65” screen size (equivalent)
Interaction Touch screen Touchpad (tap & swipe)
Communication BLE 4.1, Wi-Fi, GPS, 4G LTE BLE 4.1, Wi-Fi, GPS
Sensors Accelerometer, Gyroscope, Accelerometer, Gyroscope,
Magnetometer Magnetometer
Figure 1: The EverySight “Raptor” smart glasses used — the glasses and an illustration of a user wearing
them.
augmented reality smart glasses, we decided to focus on this aspect, planning a larger scale
visiting path. The paper presents the design and implementation of the system and lessons
learned during the design and initial evaluation of the system as a navigation aid in an urban
setting.
2. System overview
In order to be able to experiment urban navigation in a realistic setting, a prototype system was
developed. The main component is the Everysight Raptor smart glasses1 (Figures 1,2 and 3),
powered by a Qualcomm® Snapdragon™ 410E with 32GB internal storage, 2GB SDRAM. The
device is the same device used for the study described in Litvak and Kuflik [10]. Table 1 shows
a comparison of the technical specifications of the smart glasses with a mobile phone (taken
from Litvak and Kuflik [10]).
Android Studio made it possible to build the application together with the Raptor SDK (RDK).
The RDK v0.2.1 was used to implement the AR layering of POI names and navigation aids.
1
https://everysight.github.io/rdk_docs/
Figure 2: A rear view of the monocular augmented reality smart glasses with the optical lens; the smart
glasses live display in action (left to right).
Figure 3: Touch-based interaction with the smart glasses.
Currently, the RDK repository is private and accessible by invitation only. Location tracking
was made possible using the smartphone’s GPS receiver and the Line of Sight (LOS) detection
capability provided by the smart glass sensors and RDK utilities. The GPS receiver and LOS
calculation helped us build a context-aware application that followed the visitor’s attention to
nearby objects.
For demonstrating the use of the system for supporting urban navigation in cultural heritage
scenarios, we selected the German Colony in downtown Haifa. The German Templar Colony is
a historic neighborhood in Haifa, at the foot of Mt. Carmel just below the Baha’i Gardens. It
was established in Ottoman Haifa in 1868 as a German Templar Colony in Palestine. Its main
street is about 600 meters long and a tour was planned along this street. Textual content about
historical buildings at the German colony of Haifa was prepared and linked to their geographic
position. A pre-visit simple interface enabled users to plan a visit between POIs in the German
colony. As the smart glasses are capable of superimposing virtual content such as text and
images onto the visitor’s FOV (Figure 3), the virtual navigation instructions between POIs was
made visible in a see-through optical display (as described by Ashkenazi and Shamir [11], “The
electro-optical unit includes a processor and a light projection unit. The processor is coupled with
the light projection unit. The light projection unit is configured to transmit light beams onto the
partially transmissive partially reflective lens”).
The navigation instructions included an arrow pointing in the direction of the next POI to
visit and a textual description provided the name of the next POI and the distance to it (partially
illustrated by Figure 2). Once a visitor arrived at a POI, a message announced the arrival at
the POI and if the visitor was interested in an explanation about the POI, s/he had to touch
the touch pad on the right side of the glasses (Figure 3). The information about the POI was
available only once the visitor got close enough and was notified and until the visitor moved
out of the pre-defined radius. If the visitor deviated from the planned tour path, the system
maintained the next POI to visit and continuously provided instructions about how to get there,
assuming that the visitor would like to continue the planned tour later on.
The navigation planning and instructions were based on using google maps API. The overall
system architecture is given by Figure 4: A web-based interface enables the user to plan a tour.
The tour is stored in a web-based database. The tour is downloaded to the user’s mobile phone
that used google maps to calculate the position continuously. The navigation instructions are
sent and presented on the smart glasses’ screen and if/when information is requested by the
user, the POI description is sent to the glasses and played.
The following sequence of activities is followed:
1. The user connects phone with Raptor glasses via in-app function.
2. The user selects path / creates path from saved locations.
3. After starting the navigation, phone uses the Google Maps API along with the device’s
GPS coordinates in order to get directions to the next station.
4. Directions are sent via the Bluetooth connection to the Raptor glasses.
5. At any point, the user can change the color of the displayed text (in the glasses), by
swiping left or right.
6. When the user reaches a station, a tap on the side of the glasses reads out a description of
that station.
7. The station automatically changes to the next in the path when reaching a certain radius
from it (configurable in-app), and plays a sound when you do so.
3. Lessons learned during development and initial trials
During the development and while experimenting with the smart glasses, we noticed first of all,
that it takes some time to get used to using the glasses, as the user has to focus on the screen in
order to notice the arrow and the instructions and to continuously change focus from looking
around to getting navigation information – this takes some time. Another issue is the color of
the text – it can be red, green or yellow, hence, depending upon the scene in the background,
sometimes it is hard to notice it. Moreover, it appeared that there may be personal preferences
Figure 4: System architecture.
with respect to the color of the text, hence color changing was made easily for the user during
navigation, simply by touching the touch pad. There were no problems with the directional
arrow with respect to color, but the arrow appeared to be quite jumpy, hence a sliding window
mechanism was introduced to relax a bit the “jumpiness”. Another aspect which seemed to be a
result of the navigation system was that when getting to roundabouts, as the user had to turn
aside to get to a cross walk, the system misinterpreted this as an erroneous change in direction.
Finally, the users’ pose had an impact as well — the user had to keep straight, as otherwise the
navigation system got confused.
To summarize, it seems that while the technology seems interesting and promising, we are
not sure that it is mature enough to be used on a daily basis, but this is something for a larger
users’ study.
Figure 5: An illustration of the textual description positional options and the directional arrow (left)
and color options (right).
4. Planned evaluation
Given the above, we are planning to evaluate the system in a user study at the German colony
in Haifa. Users will be asked to create a short tour of about 5 or 6 POIs along the street. They
will connect the glasses to their mobile phone, download the pre-planned tour and follow it.
During the tour they will be asked also to deviate from the planned tour, move to a nearby
street and get back to the tour.
We plan to evaluate the usability of the system with respect to the visualization of the
navigation cues (colors, how they are visible given different background) and to elicit feedback
and ideas for improving the system.
5. Summary and potential future work
While outdoors navigation support seems to be a solved problem, given GPS technology and
mobile devices, still, navigation using mobile devices requires the user either to look at the
screen of the mobile device, using a map-based visualization (with or without audio instruction)
or to use audio alone, if s/he wants to avoid looking at the screen. These limitations may be
addressed by the use of smart glasses that in addition to audio-based instructions may also
display visual cues in the user’s field of view, thus reliving the user from the need to look at the
screen and decipher the map visualization.
One important issue is the fact that it is impossible to take a screenshot of what the user
actually sees through the smart glasses hence we are unable to provide an accurate image.
However, an illustration is provided by Figure 5. While the current study focused on the urban
navigation function alone, for experimentation purpose (more for setting up the system) a user
interface for visit planning was developed. It allows the user to create a tour path and download
it to the mobile system (smartphone) and then the user can navigate while following the path
or deviating from it, at will. Information regarding POIs is delivered upon arrival at a POI, but
still, personalization and recommendation were out of scope for this study.
Acknowledgments
The smart glasses for the study were provided by Everysight company for free.
References
[1] N. Brancati, G. Caggianese, G. D. Pietro, M. Frucci, L. Gallo, P. Neroni, Usability evaluation
of a wearable augmented reality system for the enjoyment of the cultural heritage, in:
2015 11th International Conference on Signal-Image Technology & Internet-Based
Systems (SITIS), IEEE, 2015. doi:10.1109/sitis.2015.98 .
[2] A. Tomiuc, et al., Navigating culture. enhancing visitor museum experience through
mobile technologies. from smartphone to google glass., Journal of Media Research-Revista
de Studii Media 7 (2014) 33–46.
[3] M. tom Dieck, T. Jung, D.-I. Han, Mapping requirements for the wearable smart glasses
augmented reality museum application, Journal of Hospitality and Tourism Technology 7
(2016) 230–253. doi:10.1108/jhtt- 09- 2015- 0036 .
[4] M. Mokatren, T. Kuflik, I. Shimshoni, Exploring the potential of a mobile eye tracker as
an intuitive indoor pointing device: A case study in cultural heritage, Future generation
computer systems 81 (2018) 528–541. doi:10.1016/j.future.2017.07.007 .
[5] ????
[6] S. Jiang, B. Moyle, R. Yung, L. Tao, N. Scott, Augmented reality and the enhancement of
memorable tourism experiences at heritage sites, Current Issues in Tourism (2022) 1–16.
doi:10.1080/13683500.2022.2026303 .
[7] K. Kontopanagou, A. Tsipis, V. Komianos, A framework for exploring churches/-
monuments/museums of byzantine cultural influence exploiting immersive technolo-
gies in real-time networked environments, Technologies 9 (2021) 57. doi:10.3390/
technologies9030057 .
[8] A. Alkhafaji, S. Fallahkhair, E. Haig, A theoretical framework for designing smart and
ubiquitous learning environments for outdoor cultural heritage, Journal of Cultural
Heritage 46 (2020) 244–258. doi:10.1016/j.culher.2020.08.006 .
[9] D. Cisternino, L. Corchia, V. D. Luca, C. Gatto, S. Liaci, L. Scrivano, A. Trono, L. T. De Paolis,
Augmented reality applications to support the promotion of cultural heritage: The case of
the basilica of saint catherine of alexandria in galatina, J. Comput. Cult. Herit. 14 (2021).
URL: https://doi.org/10.1145/3460657. doi:10.1145/3460657 .
[10] E. Litvak, T. Kuflik, Enhancing cultural heritage outdoor experience with augmented-
reality smart glasses, Personal and ubiquitous computing 24 (2020) 873–886. doi:10.1007/
s00779- 020- 01366- 7 .
[11] A. Ashkenazi, H. Shamir, Wearable optical display system for unobstructed viewing, 2019.
US Patent 10,488,660.