“Take me Home”: AR to Connect Exhibits to Excavation Sites Angeliki Antoniou George Lepouras Apostolos Kastritsis Department of Archival, Library and Department of Informatics and Institute for Language and Speech Information Studies Telecommunications Processing University of West Attica University of the Peloponnese Athena Research and Innovation Aegaleo, Greece Tripolis, Greece Centre angelant@uniwaop.gr g.lepouras@uop.gr Athens, Greece kastritsis@gmail.com Jason Diakoumakos Yannis Aggelakos Nikos Platis Department of Informatics and Yannis Aggelakos Creative Services Department of Informatics and Telecommunications Tripolis, Greece Telecommunications University of the Peloponnese aggelakospnm@gmail.com University of the Peloponnese Tripolis, Greece Tripolis, Greece iasonasdi@uop.gr nplatis@uop.gr ABSTRACT and are exhibited in the central museum of the region. Regional mu- We present the design and implementation of an AR application that seums like the one in Tripolis are a very good and practical solution connects museum objects to their original locations in archaeologi- in collecting items from different locations and exhibiting them in cal sites; our aim is both to solve the museum de-contextualization one place; however, over the years it was realized that an important problem and to promote exploration of the archaeological sites in weakness of this practice was the de-contextualization of the experi- a region. The application projects the museum object upon the ence. Many researchers agree that a museum exhibit is not merely a archaeological site and its original location, bringing the landscape material object that needs to be protected and displayed, but mostly into the museum experience. We outline technical challenges en- a synthesis of practices, values, beliefs, traditions, memories, etc. countered and solutions adopted. Since this is a work in progress, [4, 12]. Thus, removing the object from its original location and further user testing remains to evaluate our approach and the app. bringing it in the museum may solve many practical problems, but gives rise to important de-contextualization issues regarding the CCS CONCEPTS visitor’s experience with the object, which need to be addressed by re-contextualizing the object, also addressing visitor needs for • Information systems → Information systems applications; • Ap- “making sense” of the presented content and their relation to their plied computing; • Human-centered computing → Systems original locations [1]. and tools for interaction design; In this light, the famous Norwegian painter J.C. Dahl (1788–1857) criticized the de-contextualizing character of museums and pro- KEYWORDS posed the landscape as a museum in itself, stressing the importance Augmented reality, Cultural Heritage, Museum, Contextualization of the landscape in the museum experience: “This conflation of ACM Reference Format: museum skepticism and the launching of the landscape as museum Angeliki Antoniou, George Lepouras, Apostolos Kastritsis, Jason Diak- constitutes an interesting and overlooked contribution to a famil- oumakos, Yannis Aggelakos, and Nikos Platis. 2020. “Take me Home”: AR iar debate, namely the century-old critique of the museum as an to Connect Exhibits to Excavation Sites. In Proceedings of 𝐴𝑉 𝐼 2𝐶𝐻 2020: instrument of deadening de-contextualization.” [8]. Workshop on Advanced Visual Interfaces and Interactions in Cultural Heritage Another consequence of exhibiting items collectively in regional (𝐴𝑉 𝐼 2𝐶𝐻 2020). ACM, New York, NY, USA, 4 pages. museums, which are usually located in the central, most well- known city of the region, is that visitors are amassed there. In 1 INTRODUCTION recent years, this tendency of mass tourism to collect in specific The Archaeological Museum of Tripolis (Greece) is located in the locations has led to serious sustainability issues of cultural heritage. prefecture of Arcadia (around 160 km southwest of Athens). It Thus, many local authorities, especially in areas not so well-known houses a very important collection of Neolithic (10000 BC–7000 to tourists, like Arcadia, are trying to find ways to redirect visi- BC), Early Hellenic (200 BC–2000 BC), Mycenaean (1600 BC–1100 tors to further locations within their region; this would expand BC), Geometric (1100 BC–800 BC), Archaic (800 BC–500 BC), and cultural tourism and boost local economies by capitalizing on dif- Roman (146 BC–330 AD) ancient items. Like in most regional muse- ferent tourist routes and alternative cultural experiences [10]. Re- ums, the items come from different excavation sites around Arcadia contextualizing the objects in museums has, in this respect, the potential to raise awareness of the respective cultural destinations. The present study is a work in progress, aiming at connecting 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy © Copyright 2020 for this paper by its authors. Use permitted under Creative Commons specific museum objects to their original excavation site in an License Attribution 4.0 International (CC BY 4.0). attempt to 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Angeliki Antoniou et al. • provide important contextual information, has to repeat the complete procedure from scratch. The aforemen- • use landscape as an important contextual element, tioned method has the indisputable advantage of very high quality • highlight and enrich artifacts, panoramic images. However, the photographer has to carry bulky • highlight historical places and archaeological sites in the equipment, often by foot at remote areas, like some of the excava- wider region of Arcadia, tion sites in Arcadia. In order to assemble the set-up, considerable • direct visitors to less well-known locations and motivate time is needed, which is invaluable in situations where lighting them to visit the archaeological sites around the prefecture conditions require fast response from the operator (like sunrise or of Arcadia. the “golden hour” before sunset). The equipment is susceptible to adverse weather conditions, like extreme heat (often experienced 2 LITERATURE REVIEW in Greek outdoor settings), that could melt soft camera and tripod Connecting the museum’s objects to excavation sites can be ben- parts, or gusts of wind that could cause the tripod to fall and the eficial, since it would connect the artifacts to the places of their camera and lens to be damaged. Working with a laptop on-site to origin, thus providing important contextual information. Past ef- acquire the huge datasets produced and stitch them together takes forts showed the importance of contextual experiences in Cultural a lot of time, making a re-shot impossible in certain cases. For these reasons, we followed a hybrid approach. While a tradi- Heritage (CH) in enhancing the visitor experience, by adding con- tional setup, as described above, was used for each archaeological textual displays and contextual installations in different venues [1]. Different approaches have been tried, from targeted narratives that site, backup 360° equirectangular panoramic images were captured vividly describe the place of origin of the exhibit and directly ask using the Google Street View application on a current high-end the user to visit nearby locations [7], to installations and applica- Android smartphone. The application uses the device’s accelerom- tions that allow sensory interactions involving smell, touch and eter to understand the orientation of the device. A visual guide sound, with visual and aural feedback [2]. assists the operator at capturing all the different angles required for Augmented Reality (AR) techniques can enrich the experience a full 360° panoramic image. After capturing is complete, the soft- and increase the interest of the visitors, as well as enhance the learn- ware automatically stitches together all the captured images and ing experience [15] regarding the artifacts presented, also showing the photographer can preview the result in almost real time, like clear cognitive gains like memory enhancement and curiosity [9]. a user would view the result on his/her smartphone. As one real- izes, the process is much more straightforward than the traditional Well-designed AR at museums can significantly affect the quality method. The smartphone used weighs just about 150g, while the of experience [13] and the intentions of visitors to revisit the venue [5]. traditional 360° photography equipment weighs at least 1.5kg, even AR has been used successfully to alleviate the issues of museum with modern mirrorless cameras. Carrying a laptop or transferring de-contextualization. In the world-famous Acropolis Museum, AR the images to a workstation is not required and the automation was incorporated in narratives, in an attempt to place the exhibits that the application provides at stitching and presenting the final to their original location, but also to provide virtual reconstructions, panorama to the user in nearly real time is indispensable. The pho- to highlight details, etc. [6]. Other studies have shown that positive tographer can instantly decide if the result is satisfactory or not, emotions and learning benefits were found when AR was used to and quickly proceed on re-shooting. It should also be noted that contextualize museum exhibits [3], and that the effective commu- modern smartphones possess high resolution imaging sensors with HDR (high dynamic range) capabilities and are able to produce nication of the original context through AR increased motivation very high quality pictures assisted by computational photography and curiosity [11]. processing techniques. In our captures, some image stitching errors occurred either with 3 CREATION OF A 3D WORLD USING the specialized software or with the smartphone application and 360-DEGREE PANORAMAS were easily corrected using Adobe Photoshop. The final images The AR application that we envisaged for the museum of Tripolis where resized and compressed so that the experience would be required the virtual artifacts to be placed on the excavation sites at seamless even on modestly powerful smartphones their current state today. For this purpose, it was decided that 360° immersive panoramas was the preferred way of transferring the visitor to the site within a virtual environment. 4 IMPLEMENTATION OF THE AR APP The usual method to create such panoramas involves a compli- Our AR app can be used on Android smartphones. Visitors point cated technique for acquiring the panoramic images needed. The the camera of their smartphone at AR-enabled exhibits and they photographer must have a modern digital single lens reflex cam- can explore the place that the exhibit was actually found in, with era (or dSLR), an ultra-wide angle (also called a fisheye) lens, and their phone as a window to the archaeological site (Figures 1, 2). a sturdy tripod with a special panoramic head. She then has to Regarding the recognition of AR-enabled exhibits by the app, position herself and the camera at the approximate center of the at first we created a cloud database with various pictures of the site, and start taking photos at specific angle increments in order exhibits using the Vuforia framework and linked it to Unity, the to fully capture the surrounding space in a circular fashion; then real time graphics engine that we used for our implementation. the camera is tilted at a specific angle and another set of photos is Unfortunately, tests of the image recognition algorithms in situ did recorded. Afterwards, photos must be stitched together via dedi- not yield satisfactory results. Our experimentation revealed that the cated software. If the results are not satisfactory, the photographer algorithms were adversely affected by changing lighting conditions. “Take me Home”: AR to Connect Exhibits to Excavation Sites 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy the social circumstances in ancient times, and AR provided a view of these landscapes as they are today. In addition, the museum experience was accompanied by maps that would provide directions to the visitor to reach different archaeological sites in the wider region. Since technological interference can interrupt the cultural ex- perience happening in the museum, we decided that the AR app should not be accessible during one’s initial visit to the museum but only during a second walk. The visitors, holding their smartphone, can then follow a different route, looking for the exhibits that have useful information to be triggered. 6 PATHS FOR FURTHER WORK Being a work in progress, the present study can be completed and extended on several axes. First and foremost, we believe that such novel methods need to be a part of the greater framework and design of the museum experience. Their outcomes can be maximized when Figure 1: Archaic head of kore originally found at Man- they are incorporated in the design of the exhibition and integrated tineia. into the curatorial plan. In our deployment at the Archaeological Museum of Tripolis we made a substantial effort to this end, as outlined above. In the general case, aspects to study further are: • how to decide which objects will be augmented, balancing the coverage of the museum collection and the number of promoted archaeological sites with the induced costs and complexity for producing the required AR material; • how to show to the visitor which exhibits are augmented in the museum, taking into account the technologies available and any other factors that may affect their applicability (in our case, for example, we had to use QR codes instead of the Figure 2: Marble stele originally found at the ancient site of more elegant direct recognition); Orchomenos. • what augmentation material to provide, and in what form (text, image, audio, video); • how to support the cultural experience without affecting it These are, however, unavoidable in the Archaeological Museum adversely in any way (for example, avoiding visitor distrac- of Tripolis, since it depends mostly on natural lighting and direct tion); sunlight comes through. Therefore, we decided to not use image • what are the visitor expectations out of such technological recognition and, instead, assign a QR code to every exhibit that we approaches, in regards to a cost effective app; wanted to augment. Building the cloud database with the QR codes • how to evaluate the outcomes of such technological ap- and their respective information, testing and integrating with the proaches. graphics engine produced the desirable results. We plan on conducting broader testing and summative evalua- As described in the previous section, for each exhibit we acquired tion of our effort in order to assess usability issues of our app with 360° panoramic photographs of the place it was found in. Then, in the wider public, user satisfaction, and effectiveness of the appli- Unity, we built a 3D world as a spherical structure with inverted cation in regards to its objectives (contextualization, promotion of normals and applied these panoramic images as textures over it. We close-by sites, etc.). This evaluation will hopefully demonstrate the provided a first-person controller view using the mobile phone’s potential of AR in contextualizing museum experiences. accelerometer and gyrometer to add interaction and control the Regarding the app itself, the virtual world implemented and viewpoint towards the mixed environment, thus giving the ability the use of interactive real time graphics on commodity devices to explore the simulated physical site via one’s phone screen. Fi- can provide a starting platform for gamification efforts and related nally, we added a minimalistic GUI to our app, providing additional research; gamification by itself has also been shown to have positive information about each archaeological site (location, directions, effects on museum experiences [14]. etc.). 7 CONCLUSIONS 5 PILOT APPLICATION Augmented Reality is a powerful tool that can enrich visitor expe- In the Archaeological Museum of Tripolis, our AR app was carefully riences, increase learning and attract visitors, but also a tool that coupled with targeted narratives designed for the various exhibits. can support contextualization of museum objects and contribute to The narratives asked visitors to imagine different landscapes and directing visitors to less well-known venues and sites. The present 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Angeliki Antoniou et al. work investigated issues of museum de-contextualization, and in- Using Augmented Reality. Studies in Digital Heritage 2, 1 (Sept. 2018), 42–67. troduced the design and implementation of an AR app to bring https://doi.org/10.14434/sdh.v2i1.24500 [12] Alda Rodrigues. 2016. People and Things: Questions Museums Make us Ask the original landscape in the museum experience and promote ar- and Answer. Royal Institute of Philosophy Supplement 79 (Oct. 2016), 199–216. chaeological places in the wider Arcadian region. Technical aspects https://doi.org/10.1017/s135824611600014x [13] Anne Bationo Tillon, Eric Marchand, Jean Laneurit, Fabien Servant, Isabelle of the complete process were described, and relevant challenges Marchal, and Pascal Houlier. 2010. A day at the museum: An augmented fine-art were explained together with the solutions applied. Finally, our exhibit. In 2010 IEEE International Symposium on Mixed and Augmented Reality roadmap to complete the present work was outlined, by evaluating - Arts, Media, and Humanities. IEEE. https://doi.org/10.1109/ismar-amh.2010. 5643290 its approach as well as its effectiveness and by extending it towards [14] Yuchen Weng, Tao Shen, Sihuang Chen, and Bing Xiao. 2019. Gamification in further interesting directions. Local Intangible Cultural Heritage Museums for Children: A Case Design. In Our future works will focus on testing alternative ways of in- Lecture Notes in Computer Science. Springer International Publishing, 233–245. https://doi.org/10.1007/978-3-030-23538-3_18 corporating the AR experience in the museum visit by minimizing [15] Susan A. Yoon, Karen Elinich, Joyce Wang, Christopher Steinmeier, and Sean possible intrusive elements. We will also work with reconstructions Tucker. 2012. Using augmented reality and knowledge-building scaffolds to im- prove learning in a science museum. International Journal of Computer-Supported of the original sites and show the objects not only in the landscape, Collaborative Learning 7, 4 (Aug. 2012), 519–541. https://doi.org/10.1007/s11412- as it is today but also project them in the reconstructions of the sites. 012-9156-x In addition, we are already working with a tool for archaeologists and visitors to show the exact position and location of the object as it was found during the excavation. This can be available to visitors of both the sites and the museum. ACKNOWLEDGEMENTS This work has been realized within the project “CrossCult” funded by the European Union’s Horizon 2020 research and innovation program, Grant #693150. We are also grateful to the Ephorate of Antiquities of Arcadia and the Archaeological Museum of Tripolis (Greece) for their cooperation. REFERENCES [1] Michael Conforti. 1995. Museums past and museums present: Some thoughts on institutional survival. Museum Management and Curatorship 14, 4 (Jan. 1995), 339–355. https://doi.org/10.1080/09647779509515454 [2] Daniel Harley, Melanie McBride, Jean Ho Chu, Jamie Kwan, Jason Nolan, and Ali Mazalek. 2016. Sensing context: Reflexive design principles for intersensory museum interactions. MW2016: Museums and the Web, Los Angeles, CA, USA (2016). [3] Jason M. Harley, Eric G. Poitras, Amanda Jarrell, Melissa C. Duffy, and Susanne P. Lajoie. 2016. Comparing virtual and location-based augmented reality mobile learning: emotions and learning outcomes. Educational Technology Research and Development 64, 3 (Jan. 2016), 359–388. https://doi.org/10.1007/s11423-015-9420- 7 [4] Martin Heidegger. 1963. The Origin of the Work of Art. (1963). [5] Timothy Jung, M. Claudia tom Dieck, Hyunae Lee, and Namho Chung. 2016. Effects of Virtual Reality and Augmented Reality on Visitor Experiences in Mu- seum. In Information and Communication Technologies in Tourism 2016. Springer International Publishing, 621–635. https://doi.org/10.1007/978-3-319-28231-2_45 [6] Jens Keil, Laia Pujol, Maria Roussou, Timo Engelke, Michael Schmitt, Ulrich Bockholt, and Stamatia Eleftheratou. 2013. A digital look at physical museum exhibits: Designing personalized stories with handheld Augmented Reality in museums. In 2013 Digital Heritage International Congress (DigitalHeritage). IEEE. https://doi.org/10.1109/digitalheritage.2013.6744836 [7] Kalliopi Kontiza, Angeliki Antoniou, Abdullah Daif, Susana Reboreda-Morillo, Maddalena Bassani, Silvia González-Soutelo, Ioanna Lykourentzou, Cather- ine Emma Jones, Joseph Padfield, and Martín López-Nores. 2020. On How Technology-Powered Storytelling Can Contribute to Cultural Heritage Sustain- ability across Multiple Venues—Evidence from the CrossCult H2020 Project. Sustainability 12, 4 (Feb. 2020), 1666. https://doi.org/10.3390/su12041666 [8] Mari Lending. 2009. Landscape versus Museum: J. C. Dahl and the Preservation of Norwegian Burial Mounds. Future Anterior 6, 1 (2009), xi–17. https://doi.org/ 10.1353/fta.0.0029 [9] Weiquan Lu, Linh-Chi Nguyen, Teong Leong Chuah, and Ellen Yi-Luen Do. 2014. Effects of mobile AR-enabled interactions on retention and transfer for learning in art museum contexts. In 2014 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design (IMSAR- MASH'D). IEEE. https://doi.org/10.1109/ismar-amh.2014.6935432 [10] Duncan Marson. 2011. From Mass Tourism to Niche Tourism. Research Themes for Tourism (2011), 1–11. [11] Eslam Nofal, Ahmed Magdy Elhanafi, Hendrik Hameeuw, and Andrew Vande Moere. 2018. Architectural Contextualization of Heritage Museum Artifacts