In recent years, as mobile smart devices are becoming widely available. Traditional audio guides are being replaced by smartphone apps and even smart glasses. Augmented reality becomes a possible tool allowing visitors to interact in an engaging way with sites and artefacts in their vicinity and elicit information about them. These novel mobile guides present location-based information by audio and notify about objects of interest as they become visible to the user [1,2]. Using mobile augmented reality in domains such as tourism and specifically in cultural heritage, has the potential to enhance the visitor's experience by providing easy and intuitive interaction while simply bringing information into the visitor field of view when the visitor is looking at an object [3]. These systems can reactively or proactively [4,5] augment a visitor's view with relevant CH information without the devices themselves becoming the focus of attention. They allow real-world information to be sent, received, and viewed without requiring the visitor to carry and simultaneously view a hand-held guide. Moreover, they can overlay historical images and bring to life historical events overlaying them on the current view of a landscape [6,7] This technology is gaining interest in recent years due to its attractiveness and ease of use [8,9]. Recently, Litvak and Kuflik [10] demonstrated the potential of using augmented reality smart glasses as a mobile tourist guide outdoors and noted a few challenges facing the wide use of such devices. Among the experimented features was navigation support within a small archaeological site. Following the positive response to the navigation support provided by the augmented reality smart glasses, we decided to focus on this aspect, planning a larger scale visiting path. The paper presents the design and implementation of the system and lessons learned during the design and initial evaluation of the system as a navigation aid in an urban setting.
In order to be able to experiment urban navigation in a realistic setting, a prototype system was developed. The main component is the Everysight Raptor smart glasses1 (Figures 1,2 and 3), powered by a Qualcomm® Snapdragon™ 410E with 32GB internal storage, 2GB SDRAM. The device is the same device used for the study described in Litvak and Kuflik [10]. Table 1 shows a comparison of the technical specifications of the smart glasses with a mobile phone (taken from Litvak and Kuflik [10]). Android Studio made it possible to build the application together with the Raptor SDK (RDK). The RDK v0.2.1 was used to implement the AR layering of POI names and navigation aids. Currently, the RDK repository is private and accessible by invitation only. Location tracking was made possible using the smartphone's GPS receiver and the Line of Sight (LOS) detection capability provided by the smart glass sensors and RDK utilities. The GPS receiver and LOS calculation helped us build a context-aware application that followed the visitor's attention to nearby objects.
For demonstrating the use of the system for supporting urban navigation in cultural heritage scenarios, we selected the German Colony in downtown Haifa. The German Templar Colony is a historic neighborhood in Haifa, at the foot of Mt. Carmel just below the Baha'i Gardens. It was established in Ottoman Haifa in 1868 as a German Templar Colony in Palestine. Its main street is about 600 meters long and a tour was planned along this street. Textual content about historical buildings at the German colony of Haifa was prepared and linked to their geographic position. A pre-visit simple interface enabled users to plan a visit between POIs in the German colony. As the smart glasses are capable of superimposing virtual content such as text and images onto the visitor's FOV (Figure 3), the virtual navigation instructions between POIs was made visible in a see-through optical display (as described by Ashkenazi and Shamir [11], "The electro-optical unit includes a processor and a light projection unit. The processor is coupled with the light projection unit. The light projection unit is configured to transmit light beams onto the partially transmissive partially reflective lens").
The navigation instructions included an arrow pointing in the direction of the next POI to visit and a textual description provided the name of the next POI and the distance to it (partially illustrated by Figure 2). Once a visitor arrived at a POI, a message announced the arrival at the POI and if the visitor was interested in an explanation about the POI, s/he had to touch the touch pad on the right side of the glasses (Figure 3). The information about the POI was available only once the visitor got close enough and was notified and until the visitor moved out of the pre-defined radius. If the visitor deviated from the planned tour path, the system maintained the next POI to visit and continuously provided instructions about how to get there, assuming that the visitor would like to continue the planned tour later on.
The navigation planning and instructions were based on using google maps API. The overall system architecture is given by Figure 4: A web-based interface enables the user to plan a tour. The tour is stored in a web-based database. The tour is downloaded to the user's mobile phone that used google maps to calculate the position continuously. The navigation instructions are sent and presented on the smart glasses' screen and if/when information is requested by the user, the POI description is sent to the glasses and played.
The following sequence of activities is followed:
1. The user connects phone with Raptor glasses via in-app function.
2. The user selects path / creates path from saved locations. 3. After starting the navigation, phone uses the Google Maps API along with the device's GPS coordinates in order to get directions to the next station. 4. Directions are sent via the Bluetooth connection to the Raptor glasses. 5. At any point, the user can change the color of the displayed text (in the glasses), by swiping left or right. 6. When the user reaches a station, a tap on the side of the glasses reads out a description of that station. 7. The station automatically changes to the next in the path when reaching a certain radius from it (configurable in-app), and plays a sound when you do so.
During the development and while experimenting with the smart glasses, we noticed first of all, that it takes some time to get used to using the glasses, as the user has to focus on the screen in order to notice the arrow and the instructions and to continuously change focus from looking around to getting navigation information -this takes some time. Another issue is the color of the text -it can be red, green or yellow, hence, depending upon the scene in the background, sometimes it is hard to notice it. Moreover, it appeared that there may be personal preferences with respect to the color of the text, hence color changing was made easily for the user during navigation, simply by touching the touch pad. There were no problems with the directional arrow with respect to color, but the arrow appeared to be quite jumpy, hence a sliding window mechanism was introduced to relax a bit the "jumpiness". Another aspect which seemed to be a result of the navigation system was that when getting to roundabouts, as the user had to turn aside to get to a cross walk, the system misinterpreted this as an erroneous change in direction. Finally, the users' pose had an impact as well -the user had to keep straight, as otherwise the navigation system got confused. To summarize, it seems that while the technology seems interesting and promising, we are not sure that it is mature enough to be used on a daily basis, but this is something for a larger users' study.
Given the above, we are planning to evaluate the system in a user study at the German colony in Haifa. Users will be asked to create a short tour of about 5 or 6 POIs along the street. They will connect the glasses to their mobile phone, download the pre-planned tour and follow it. During the tour they will be asked also to deviate from the planned tour, move to a nearby street and get back to the tour.
We plan to evaluate the usability of the system with respect to the visualization of the navigation cues (colors, how they are visible given different background) and to elicit feedback and ideas for improving the system.
While outdoors navigation support seems to be a solved problem, given GPS technology and mobile devices, still, navigation using mobile devices requires the user either to look at the screen of the mobile device, using a map-based visualization (with or without audio instruction) or to use audio alone, if s/he wants to avoid looking at the screen. These limitations may be addressed by the use of smart glasses that in addition to audio-based instructions may also display visual cues in the user's field of view, thus reliving the user from the need to look at the screen and decipher the map visualization.
One important issue is the fact that it is impossible to take a screenshot of what the user actually sees through the smart glasses hence we are unable to provide an accurate image. However, an illustration is provided by Figure 5. While the current study focused on the urban navigation function alone, for experimentation purpose (more for setting up the system) a user interface for visit planning was developed. It allows the user to create a tour path and download it to the mobile system (smartphone) and then the user can navigate while following the path or deviating from it, at will. Information regarding POIs is delivered upon arrival at a POI, but still, personalization and recommendation were out of scope for this study.
The smart glasses for the study were provided by Everysight company for free.