Cultural Heritage and Social Experiences in the Times of COVID 19 Maria Vayanou Akrivi Katifori Athens Research Center & Department of Informatics and Athens Research Center & Department of Informatics and Telecommunications, University of Athens Telecommunications, University of Athens Athens, Greece Athens, Greece vayanou@di.uoa.gr vivi@di.uoa.gr Angeliki Chrysanthi Angeliki Antoniou Department of Cultural Technology and Communication Department of Archival, Library and Information Studies University of the Aegean University of West Attica Mytilene, Greece Aegaleo, Greece a.chrysanthi@aegean.gr angelant@uniwaop.gr ABSTRACT alternative ways to reach their audience and even expand their clien- During the COVID 19 pandemic, cultural institutions were severely tele by organizing cultural activities online. For example, theatres affected. To this end, they made an effort to enhance their online organized online performances releasing numerous plays online presence and online activity became the main way of approaching (e.g. Greek National Theatre, https://www.n-t.gr), music festivals the public. Although the social aspects of a physical visit are well were broadcasting live on the Internet (e.g. Prague Spring Festival, acknowledged and valued today, they nevertheless seemed to be https://festival.cz), operas streaming on social media (e.g. Metropol- largely neglected in the virtual museum world. In this position itan Opera, https://www.metopera.org), visual arts openings were paper, we discuss current practices during the pandemic period and organized through ZOOM calls (e.g. https://www.acg.edu/the-kids- also explore challenges for the after-COVID era. are-asking), all wishing to continue with their cultural activity and maintain operation as normal as possible. CCS CONCEPTS Especially museums increased their online presence by 80% dur- ing the lockdown, offering more social media interactions, broad- • Information systems → Collaborative and social computing sys- casts, virtual tours and online exhibitions, and observed more tems and tools; • Human-centered computing → Collaborative than 40% increase in online visits [17]. Online museum presence and social computing theory, concepts and paradigms. may take several forms, ranging from traditional online catalogues and digital exhibitions that include narratives and audio-visual KEYWORDS content to virtual representations of the museum’s physical en- Museums, Virtual experiences, Social experiences, Social distancing, vironment, in which online visitors may move and closely ob- COVID 19 serve the exhibits on display, thus simulating the actual visiting ACM Reference Format: experience (see for instance the Google Art Project, where sev- Maria Vayanou, Akrivi Katifori, Angeliki Chrysanthi, and Angeliki Anto- eral museum spaces have been represented, or custom applica- niou. 2020. Cultural Heritage and Social Experiences in the Times of COVID tions such as the VR Museum of Fine Art, available on steam 19. In Proceedings of 𝐴𝑉 𝐼 2𝐶𝐻 2020: Workshop on Advanced Visual Interfaces - https://store.steampowered.com/app/515020/The_VR_Museum_ and Interactions in Cultural Heritage (𝐴𝑉 𝐼 2𝐶𝐻 2020). ACM, New York, NY, of_Fine_Art), or “Hold the World”, with David Attenborough virtu- USA, 4 pages. ally guiding remote visitors at the collections of London’s Natural History Museum (https://www.factory42.uk/holdtheworld), among 1 MUSEUM ACTIVITY DURING THE others). LOCKDOWN Despite the wealth of cultural resources that are currently acces- sible online and the remarkable diversity in the adopted experience With the outbreak of COVID 19 and the world pandemic, many design approaches, a close look at the most popular and most visited countries globally proceeded with population movement control online museums during the lockdown (https://www.theguardian. practices or even complete lockdowns in an attempt to restrict com/travel/2020/mar/23/10-of-the-worlds-best-virtual-museum-and- the spreading of the virus. Cultural institutions were severely af- art-gallery-tours) shows that cultural online visits are commonly fected since many had to stop their activity or significantly alter designed as private, single-user experiences. However, museum it while trying to cope with the new reality that the virus has im- studies have repeatedly highlighted the importance of the social posed in our lives. In this climate, cultural institutions had to find context [10], suggesting that social interactions are key elements in the design of engaging cultural visits. 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy In this paper, we posit that the digital empowerment of social- © Copyright 2020 for this paper by its authors. Use permitted under Creative Commons ity is an important and pertinent issue not only in the museums’ License Attribution 4.0 International (CC BY 4.0). physical space but also in the virtual museum world. Moreover, 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Maria Vayanou, Akrivi Katifori, Angeliki Chrysanthi, and Angeliki Antoniou we claim that virtual visits offer an incredible opportunity to cope social perception of the corresponding artworks and enable a so- with the physical constraints that may be posed to one or more cially derived retrieval mechanism for accessing online collections. members of a social group, enabling them to experience a joint, In all the above approaches, visitor interactions are indirect; they synchronous, social activity without requiring to be co-located in are accomplished by providing shared access to objects and applica- the same physical place - a need that highly increased due to the tions, which occur at different times and without prescribing direct recent pandemic lockdown. We thus, report practices that promote communication between the parties involved. social interactions and discuss important challenges. We believe that synchronous interaction is a key element that notably fosters the social context of cultural experiences. Previous work on the topic has experimented with systems that support 2 FROM PHYSICAL TO DIGITAL museum co-visits between onsite and remote visitors suggesting The advent of online leisure and edutainment at the beginning of that social presence is closely related to engagement and enjoyment the new millennium created a fierce competition for onsite museum and thus, an improved remote experience [14, 18]. When it comes experiences and brought about a significant response by museum to entirely online strategies, besides offering free access to digital professionals and researchers who rallied to highlight the advan- resources, some institutions have opted for broadcasting at specific tages of physical visits and interactions with the curated content. times. Leaving aside the proprietary issues that may have motivated As is evident in the museum research literature, apart from the this approach, there is also a social side effect of the broadcasting unique and powerful features that make physical objects and places scheme. But how will the users be aware of their common activ- springboards for learning, engagement and authentic experiences ity? In the broadcasting scheme, there is no shared environment [15], one of the most valued aspects of the museum experience is of interaction between the spectators, yet we believe that its so- related to the socio-cultural characteristics of a visit [10]. Social cial dimension may be notably increased by digitally mediating interaction is pervasive in museums and cultural heritage sites and the co-presence of other visitors: from indications of who or how enhances learning, even in cases lacking such a design intention many people are watching the same cultural content at the same or mandate [2]. In this light, the first digital interactive applica- time, to personalized notifications about acquaintances (obtained tions that attempted to provide more rich and engaging visitor from social networking profiles) who are currently watching or interpretation were heavily criticized for prioritizing the individual now joining the same activity. The use of awareness supporting user while overthrowing the inherent social affordances of exhibits technologies that are traditionally applied in collaborative systems for co-participation and collaboration as processes of knowledge (see [16] for a recent systematic literature review) may also benefit construction [12]. This led to the growth of a promising research the design of cultural experiences to foster their social dimension, area looking at ways to enhance digitally aided social experiences particularly when experienced remotely. in the context of museums and cultural heritage sites [13, 19]. Delving into the matter of spectators’ representation, a novel scheme of “live performances in Virtual Reality worlds” was re- cently proposed [http://hamletvr.org]: To Be With Hamlet is a live 3 SYNCHRONOUS AND ASYNCHRONOUS theater performance that enables collocated groups of visitors to SOCIAL INTERACTIONS navigate in the virtual 3D story world using VIVE equipment, in Sociality may be examined through several perspectives. We first which they may see their fellow audience members alongside the classify visitor interactions as synchronous or asynchronous and live performance of actors (captured with motion-tracking tech- highlight the social affordances offered in each case, while focusing nologies). Although this example is targeted towards collocated on the latter category. experiences, we believe that it paves the way for a new form of cul- Asynchronous interactions are typically accomplished through tural productions, showcasing how virtual 3D environments may “content sharing”. Realizing the value of visitor participation, as serve as the stage of live performances where audience members insightfully elaborated in the definition of the “Participatory Mu- are explicitly represented through avatars and can socially interact seum”, several works have aspired to empower active visitor in- with their companions. volvement by supporting the creation and public access to user- During the pandemic outbreak, several conferences took place generated content (e.g. comments or tags, personal stories, emo- in the virtual world like IEEE VR, http://ieeevr.org/2020, (offered tions, opinions etc), or even the authoring of personal digital collec- a desktop VR option through mozilla hubs, slack and a teleconfer- tions and cultural exhibitions. These systems cultivate the concept encing system, with presentations happening in VR according to of “museums as social places” and aim to promote social awareness, the participant’s decision). Due to the increased need, ACM also showcasing the presence of prior museum visitors and visualizing published a guide for virtual conferences with multiple platforms their contributions. This objective is addressed both in the scope and tools [1]. The first objective of using these platforms was to of onsite and remote visits. Some systems deployed for museum provide a shared multimedia presentation environment that would environments include Imprints [3], ArtLinks [8], MobiTags [7], and support question-answering interaction between the presenter and the Room of Opinion at the Hunt Museum [11]. In some cases, the audience. In the cultural experiences offered by the Natural the user-generated content is also leveraged for improving public History Museum (London), the same goal was addressed through a access and engagement with on-line cultural collections. See for different scheme, i.e. by live broadcasting scientists’ talks in which instance the steve.museum project [5] where online visitors are question-answering was accomplished using social media platforms, prompted to annotate the displayed artworks with free-form tags. an approach that is nowadays often used alongside physical confer- The collected data is used to create folksonomies that indicate the ences as well, taking more and more ground. In this case, the focus Cultural Heritage and Social Experiences in the Times of COVID 19 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy of the experience design is to serve the communication between or obsolete endeavor. Fully immersive spaces of interpretation and the presenter and the audience, rather than the social interaction co-located social interaction such as Caves, Panoramic Cylinders between audience members (although such instances often occur and iCinemas may also witness a severe decline because they are as well). But was this objective reached? Did these platforms foster very costly to create while the chances of revenue due to social social interactions between the attendees? Despite the significant distancing measures are significantly reduced. Nevertheless, in the technological advances in related hardware and software compo- case of families and dyads, it is important to build on previous work nents, we observe that the promotion of social interactions within that enhances sociability through conversation [13, 19]. virtual worlds continues to raise significant challenges, echoed also In terms of providing a shared environment for online interac- by prior research in the cultural heritage field. tions, virtual museums need to first support direct voice or/and text-based communication between visitors. To avoid attention trap- ping on separate interface elements, “bubble-based” approaches 4 CHALLENGES AND GUIDELINES FOR THE are often employed for text-based interaction - as for instance in FUTURE the virtual multi-user museum of Piet Mondrian by MUSEUM3D COVID 19 found cultural institutions unprepared, without an es- (https://museum3d.eu) or in the virtual exhibition implemented by tablished and effective virtual presence for something other than [20]. dissemination of their on-site activities. Those few, on the other Moreover, the representation of the virtual self and others in- hand, offering digital products possibly lacked the approach to dis- cludes several complementary aspects, such as avatar form, gaze, seminate them under the right context and in the proper channels. facial expressions, gestures and posture, which continue to pose One would argue that this critique is harsh. As discussed in [21], important challenges in the virtual reality community. Comparing many institutions, going beyond a basic web presence with general to collocated interactions that unfold in physical museum environ- information, offer different types of digitized content, available af- ments, social encounters in virtual worlds often need to cope with ter the organized investment on the digitization of the past decades. the absence of behavioral social cues (particularly when experi- Many museum websites offer access to hundreds of digitized arti- enced with desktop-based interfaces), which however provide an facts through high-quality images and their relevant information. important, indirect way of communication. We expect that in the These offerings have, in theory, “opened up the museums to wider virtual museum context these issues will be sufficiently addressed and more diverse user groups” and have led to an increase in the to enforce the social dimension of cultural visits. number of on-line visitors. A great percentage of these visitors, how- We observe that only a few systems have addressed the “hybrid ever, leave the website after looking at one or two pages, generally visit scenario”, enabling local and remote visitors to share their in less than 10 seconds [6]. experience in real-time and communicate with each other [4, 20]. In As regards onsite visits, during the times of COVID 19 museums the after COVID 19 era, hybrid visits may potentially provide a very are likely to face significant challenges that will influence social promising use-case, offering important socialization opportunities interaction design in the following areas: a) visitor strategy and for vulnerable groups, who are currently facing the danger of being management policies, b) spatial configurations and exhibition de- left out. sign and, c) movement flow based on social distancing parameters. Finally, it seems that the technology is there to support social Considering the above, will we witness a return to individual user experiences in cultural heritage, either onsite or online. The tech- interfaces? if so, will we still continue to design for social interac- nology has been extensively tested and validated in many contexts, tions? or is this something that due to the necessity of staying safe, and for many years, from gaming applications to social media. keeping the numbers down and the curve flat, will be completely The main challenge lies in the successful application of these ap- abandoned? proaches in defining concrete and relevant objectives and realizing Overall, cultural institutions need to re-invent the way they ap- them through an informed experience concept design. Cultural proach both their audience and technology, they need to transform institutions are complex ecosystems and social experiences are im- their narratives from object-centric to people-centric. “Stories are portant parts. The during and post-COVID museum experience about people, not things”, as discussed in [9] and stories have al- should include carefully designed social interactions, respecting ways been the most basic method we communicated experiences, the social distancing requirements of different groups of visitors oral traditions and knowledge, always in a social context where and users. the storyteller and their audience are involved in a dynamic and interactive process and often even exchange roles. This simple and REFERENCES fundamental truth, if applied in a museum context, would have [1] ACM Presidential Task Force. 2020. Virtual conferences: A guide to best the potential to create connection and engagement. Technology practices. Report on What Conferences Can Do to Replace Face-to-Face and the vast possibilities for digital experiences, on the other hand, Meetings. https://people.clarkson.edu/~jmatthew/acm/VirtualConferences_ GuideToBestPractices_CURRENT.pdf should be put at the service of this principle instead of being an [2] Sue Allen. 2003. Looking for learning in visitor talk: A methodological exploration. objective by themselves. We need to design for sociality, either on- In Learning conversations in museums. Routledge, 265–309. [3] Kirsten Boehner, Jennifer Thom-Santelli, Angela Zoss, Geri Gay, Justin S. Hall, site or in virtual spaces, and we need to place affective connection, and Tucker Barrett. 2005. Imprints of Place: Creative Expressions of the Museum perspective-taking and empathy at the center of this design. Experience. In CHI ’05 Extended Abstracts on Human Factors in Computing Sys- As regards collaborative interfaces, designing for intra-group tems (CHI EA ’05). Association for Computing Machinery, New York, NY, USA, 1220–1223. https://doi.org/10.1145/1056808.1056881 social interactions between strangers has been a persistent research [4] Barry Brown, Ian MacColl, Matthew Chalmers, Areti Galani, Cliff Randell, and challenge, however in this climate, it may well become an irrelevant Anthony Steed. 2003. Lessons from the Lighthouse: Collaboration in a Shared 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Maria Vayanou, Akrivi Katifori, Angeliki Chrysanthi, and Angeliki Antoniou Mixed Reality System. In Proceedings of the SIGCHI Conference on Human Factors https://doi.org/10.1145/3382773 in Computing Systems (CHI ’03). Association for Computing Machinery, New [14] Galena Kostoska, Marcos Baez, Florian Daniel, and Fabio Casati. 2015. Virtual, York, NY, USA, 577–584. https://doi.org/10.1145/642611.642711 remote participation in museum visits by older adults: a feasibility study. In 8th [5] Susan Chun, Rich Cherry, Doug Hiwiller, Jennifer Trant, and Bruce Wyman. International Workshop on Personalized Access to Cultural Heritage (PATCH 2015), 2006. Steve. museum: an ongoing experiment in social tagging, folksonomy, and ACM IUI 2015. CEUR-WS.org, Aachen, DE, 1–4. museums. In Museums and the Web. 22–25. [15] Gaea Leinhardt and Kevin Crowley. 2002. Objects of learning, objects of talk: [6] CIBER Research. 2013. Europeana 2012-2013: usage and performance update: Changing minds in museums. Perspectives on object-centered learning in museums Tech. rep., CIBER Research. http://ciber-research.eu/ (2002), 301–324. http://www.museumlearning.org/leinhardtcrowley.pdf [7] Dan Cosley, Jonathan Baxter, Soyoung Lee, Brian Alson, Saeko Nomura, Phil [16] Gustavo Lopez and Luis A. Guerrero. 2017. Awareness Supporting Technologies Adams, Chethan Sarabu, and Geri Gay. 2009. A Tag in the Hand: Supporting Used in Collaborative Systems: A Systematic Literature Review. In Proceedings Semantic, Social, and Spatial Navigation in Museums. In Proceedings of the SIGCHI of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Conference on Human Factors in Computing Systems (CHI ’09). Association for Computing (CSCW ’17). Association for Computing Machinery, New York, NY, Computing Machinery, New York, NY, USA, 1953–1962. https://doi.org/10.1145/ USA, 808––820. https://doi.org/10.1145/2998181.2998281 1518701.1518999 [17] Network of European Museum Organisations. 2020. NEMO report on the impact [8] Dan Cosley, Joel Lewenstein, Andrew Herman, Jenna Holloway, Jonathan Baxter, of COVID-19 on museums in Europe. https://www.ne-mo.org/fileadmin/Dateien/ Saeko Nomura, Kirsten Boehner, and Geri Gay. 2008. ArtLinks: Fostering Social public/NEMO_documents/NEMO_COVID19_Report_12.05.2020.pdf Awareness and Reflection in Museums. In Proceedings of the SIGCHI Conference [18] Galena Pisoni, Florian Daniel, Fabio Casati, Charles Callaway, and Oliviero Stock. on Human Factors in Computing Systems (CHI ’08). Association for Computing 2019. Interactive Remote Museum Visits for Older Adults: An Evaluation of Machinery, New York, NY, USA, 403––412. https://doi.org/10.1145/1357054. Feelings of Presence, Social Closeness, Engagement, and Enjoyment in an Social 1357121 Visit. In 2019 IEEE International Symposium on Multimedia (ISM). IEEE. https: [9] Chris Crawford. 2012. Chris Crawford on interactive storytelling. New Riders. //doi.org/10.1109/ism46123.2019.00023 [10] John H. Falk and Lynn D. Dierking. 2000. Learning from museums: Visitor [19] Maria Vayanou, Akrivi Katifori, Angeliki Antoniou, and Angeliki Chrysanthi. experiences and the making of meaning. 2016. Collocated Interaction in Cultural Storytelling Experiences: How to Coor- [11] Kieran Ferris, Liam Bannon, Luigina Ciolfi, Paul Gallagher, Tony Hall, and Marilyn dinate Visitor Actions. In proceedings of the workshop Collocated Interaction: New Lennon. 2004. Shaping Experiences in the Hunt Museum: A Design Case Study. In Challenges in’Same Time, Same Place’Research, CSCW Conference. Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Prac- [20] Spyros Vosinakis and Ioannis Xenakis. 2011. A Virtual World Installation in tices, Methods, and Techniques (DIS ’04). Association for Computing Machinery, an Art Exhibition: Providing a Shared Interaction Space for Local and Remote New York, NY, USA, 205–214. https://doi.org/10.1145/1013115.1013144 Visitors. Rethinking Technology in Museums (2011). [12] Christian Heath and Dirk vom Lehn. 2008. Configuring 'Interactivity'. Social Stud- [21] David Walsh, Mark M. Hall, Paul Clough, and Jonathan Foster. 2018. Charac- ies of Science 38, 1 (Feb. 2008), 63–91. https://doi.org/10.1177/0306312707084152 terising Online Museum Users: A Study of the National Museums Liverpool [13] Akrivi Katifori, Sara Perry, Maria Vayanou, Angeliki Antoniou, Ioannis- Museum Website. International Journal on Digital Libraries 21, 1 (July 2018), Panagiotis Ioannidis, Sierra McKinney, Angeliki Chrysanthi, and Yannis Ioannidis. 75–87. https://doi.org/10.1007/s00799-018-0248-8 2020. “Let Them Talk!”: Exploring Guided Group Interaction in Digital Story- telling Experiences. J. Comput. Cult. Herit. 13, 3, Article 21 (Aug. 2020), 30 pages.