It has become a challenge to protect the built heritage and conserve the local traditional and cultural values of communities for future generations. Heritage conservation education is important as it increases the local communities' awareness and appreciation of the value of the heritage and thus helps to achieve sustainable conservation. This paper presents the design of an intelligent mobile application for heritage conservation education. The mobile application has a collection of historical photos of local heritage in Hong Kong and provides a heritage map with markers for users to tour on heritages. Users can learn histories of heritages in a fun way by using its Augmented Reality function to compare the current and the past and by using its intelligent recognition function to retrieve historic photos relevant to the current camera view. A survey of 45 Hong Kong residents showed that the mobile app can serve as a fun way to learn history in Hong Kong and is a useful tool for heritage conservation education. Heritage conservation education, mobile application, augmented reality, image recognition
The balancing between heritage conservation and urban development is hard. With the rapid
growth of cities, governments open up lands for the essential needs and build supporting facilities. It
has become a challenge to protect the built heritage and conserve the local traditional and cultural
values of communities for future generations [
Heritage conservation in Hong Kong. Hong Kong was a British Colony with very limited land
and other natural resources. Since the change of its sovereignty from the British to China in 1997,
heritage conservation has become one of the most popular issues due to economic, social, political,
and environmental concerns [
2020 Copyright for this paper by its authors.
Our contribution. This paper presents the design of an intelligent mobile application for heritage conservation education in Hong Kong. The mobile app has a collection of historical photos of local heritage in Hong Kong and provides a map with markers pointing out the locations of heritages. Due to urban renewal, historical buildings and old streets were demolished and face-lifted. The location markers are supplemented with the mobile’s GPS information such that when a user visits a location, nearby heritage can be easily identified. Then, users can use Augmented Reality to augment the camera view with historical photos, which provides an interesting way for exploring a heritage. Machine learning is also used to identify shops and objects, e.g., public transports and restaurants, such that relevant historical photos can be shown to help the users understand the difference between the current and old living styles. A survey of 45 Hong Kong residents showed that the mobile app can serve as a fun way to learn history in Hong Kong and is a useful tool for heritage conservation education.
Organization. Section 2 reviews existing and related work on heritage conservation education and using augmented reality in education. Section 3 presents the design details of our mobile application. Section 4 gives the preliminary evaluation result. Section 5 discusses the preliminary evaluation result and concludes the paper.
Augmented Reality in Education. Augmented Reality (AR) [
Heritage conservation education in Hong Kong. The present solution of heritage conservation education in Hong Kong relies on the books and photo albums on history, museum exhibitions such as the “Hong Kong Story” permanent exhibition at the Hong Kong Museum of History [16], and guided tours on heritage such as the guide tours by Kaifong Tour [17]. The Hong Kong government has developed a mobile application “iM Guide” [18] to offer museum visitors a deeper understanding of the exhibits through text and multimedia content with indoor navigation (Figure 1).
Yet the non-interactive nature of books and photo albums make them not interesting and attractive enough for heritage conservation education; the museum exhibitions and guide tours also suffer from a lot of constraints including time and social distancing under the COVID-19 pandemic. Snappy [19] is a mobile application that collects old Hong Kong street view photos through Crowdsourcing (Figure 2). Everyone can view, edit, and upload photos. Yet due to crowdsourcing, some contents in the mobile application are incorrect and inconsistent.
AR experiential learning in heritage tours. Experiential learning is the approach of learning from experience or learning by doing, and it has been shown as one of the best learning approaches for learners [20]. Experiential learning has been applied in various contexts, e.g., Artificial Intelligence education [21]. In heritage tourism, it is often that visitors require considerable information and knowledge to appreciate the cultural heritage fully [22] and AR has been proposed as a good solution to supplement the information and knowledge required [23]. A recent study by Han et al. [24] further showed that the AR experience can positively affect the satisfaction and the experiential value of visitors in heritage tours. Thus, we believe that AR can act as an effective and attractive tool for heritage conservation education in heritage tours.
This section presents the detailed design of our intelligent mobile application for heritage conservation education in Hong Kong. There is a server for the storage of heritage information, including its text description, photos, and its location coordinates. The mobile application works as a client to retrieve the information for display, and it contains four components: a heritage map, heritage information pages, AR camera, and AI recognition. 3.1.
A heritage map is shown when the mobile application starts (Figure 3 left). The geographical coordinates of all the stored heritage locations are retrieved from the server database and displayed as pin markers at the locations on the Google Map. The upper right corner of the map shows a GPS button; when a user clicks on it, the map will jump to the user’s current location (Figure 3 middle). When a user clicks on a pin marker (Figure 3 right), the name of the heritage is shown. A second click on it will lead the user to its heritage information page. 3.2.
The heritage information page shows text descriptions and photos of a heritage (see Figure 4 left). It has also a slider, showing the latest and old photos of a heritage; users can slide left and right to compare the current and the past (Figure 4 middle). There are two buttons in the top navigation bar: one button is for voice guidance such that the users can listen to the speech of the text description when touring the heritage, saving the troubles of reading the text description on the mobile devices; the other button is for opening a Google map to guide navigation to the heritage (Figure 4 right). 3.3.
On the heritage map, when the camera button is chosen, a selection menu for the AR camera and the other component AI recognition is shown (Figure 5 left). If AR camera is selected, the normal camera view is shown (Figure 5 middle). If the user clicks on the screen, the mobile application uses the GPS location of the mobile device to determine the nearby heritage, and the users can select a heritage and its historic photos at different times such that the photo can augment suitably the normal view (Figure 5 right). The AR camera has also a manual mode for users to augment the camera view using photos of old lifestyles, e.g., children on street (Figure 6). The AR camera provides a fun way to recur the past on the existing camera view. 3.4.
The AI recognition function can be started from the selection menu previously introduced (Figure 5 left). To understand the culture of the tradition, a user can capture images of objects in some categories using the camera of the mobile device, e.g., restaurants and public transport, such that relevant historic photos in the corresponding category will be shown for the users to learn and compare the current and past. This function uses the cloud services [25] provided by the machine learning [26] platform IBM Watson, and more precisely, it uses the IBM Watson Image Recognition service to classify a camera image captured by the user into predefined categories.
Participants and setting. We invited 45 Hong Kong residents to participate in a preliminary evaluation. Due to the COVID-19 pandemic, a screen recording video demonstrating the functionality of our mobile application and a questionnaire were provided to the participants.
Result. The survey used a 5-point Likert scale (1: disagree, 2: partially disagree, 3: neutral, 4: partially agree, 5: agree). Table 1 shows the results of the survey.
Item (1) The AR camera brings the old life-style back back to the real life. (2) Our mobile application can motivate learners to learn the cultural values of local heritage. (3) It is fun to use our mobile application. (4) Our mobile application is an effective way to absorb new knowledge on heritage.
1 0.0% 0.0% 0.0%
In the preliminary evaluation, Table 1 shows that the majority of the participants agreed that the AR camera is an effective tool to recur the old lifestyle to reality. The majority of the participants also agreed that our mobile application provides a fun and motivating way for learners to learn the cultural values of local heritage. The majority of them also can gain new knowledge about the heritage through this application. Thus, our mobile application can serve as a fun way to learn history in Hong Kong and is a useful tool for heritage conservation education.
This paper presented the design of a mobile application for heritage conservation education in Hong Kong. We hope that it can provide insights for other researchers and practitioners on heritage conservation on how the mobile application and augmented reality can be used to increase the local communities’ awareness and appreciation of the value of the heritage so as to achieve sustainable conservation.
[15] B. M. Garrett, J. Anthony, C. Jackson, Using mobile augmented reality to enhance health professional practice education, Current Issues in Emerging eLearning 4.1 (2018) 10. [16] Hong Kong Museum of History, Audio guide for recreating a classic: The best features of the
Hong Kong story, 2021. URL: http://hk.history.museum/en_US/web/mh/exhibition/current.html. [17] Kai Fong Tour, Homepage, 2021. URL: https://www.kaifongtour.com. [18] SenMedia Limited, iM Guide: Museum multimedia information guide system, 2021. URL: https://www.senmedia.com.hk/index.php/portfolio/im-guide-museum-multimedia-informationguide-system. [19] Snappy, Homepage, 2021. URL: https://snappy.collaction.hk. [20] D. A. Kolb. Experiential learning: Experience as the source of learning and development. FT
Press, 2014. [21] S. F. Li, K. K. Ng, L. K. Lee, A study on the application of AI experiential learning in the architecture and design courses of a Taiwan university, in: R. Li, S. K. S. Cheung, C. Iwasaki, L. F. Kwok, M. Kageto (Eds), Blended Learning: Re-thinking and Re-defining the Learning Process. ICBL 2021, volume 128390 of Lecture Notes in Computer Science, Springer, Cham, 2021, pp. 103-115. doi:10.1007/978-3-030-80504-3_9. [22] D. L. Kerstetter, J. J. Confer, A. R. Graefe, An exploration of the specialization concept within the context of heritage tourism, Journal of Travel Research 39.3 (2001) 267-274. [23] M. C. tom Dieck, T. Jung, A theoretical model of mobile augmented reality acceptance in urban heritage tourism, Current Issues in Tourism 21.2 (2018) 154-174. [24] S. Han, J. H. Yoon, J. Kwon, Impact of experiential value of augmented reality: The context of heritage tourism, Sustainability 13.8 (2021) 4147. doi:10.3390/su13084147. [25] Wang, Z., Yang, Y., Liu, Y., Liu, X., Gupta, B. B., & Ma, J. (2020). Cloud-based federated boosting for mobile crowdsensing. arXiv preprint arXiv:2005.05304. [26] Gupta, B. B., Yamaguchi, S., & Agrawal, D. P. (2018). Advances in security and privacy of multimedia big data in mobile and cloud computing. Multimedia Tools and Applications, 77(7), 9203-9208.