Cultural Heritage and Social Experiences in the Times of COVID 19

Abstract

During the COVID 19 pandemic, cultural institutions were severely afected. To this end, they made an efort to enhance their online presence and online activity became the main way of approaching the public. Although the social aspects of a physical visit are well acknowledged and valued today, they nevertheless seemed to be largely neglected in the virtual museum world. In this position paper, we discuss current practices during the pandemic period and also explore challenges for the after-COVID era.

CCS CONCEPTS

• Information systems → Collaborative and social computing systems and tools; • Human-centered computing → Collaborative and social computing theory, concepts and paradigms.

MUSEUM ACTIVITY DURING THE

LOCKDOWN

With the outbreak of COVID 19 and the world pandemic, many countries globally proceeded with population movement control practices or even complete lockdowns in an attempt to restrict the spreading of the virus. Cultural institutions were severely affected since many had to stop their activity or significantly alter it while trying to cope with the new reality that the virus has imposed in our lives. In this climate, cultural institutions had to find alternative ways to reach their audience and even expand their clientele by organizing cultural activities online. For example, theatres organized online performances releasing numerous plays online (e.g. Greek National Theatre, https://www.n-t.gr), music festivals were broadcasting live on the Internet (e.g. Prague Spring Festival, https://festival.cz), operas streaming on social media (e.g. Metropolitan Opera, https://www.metopera.org), visual arts openings were organized through ZOOM calls (e.g. https://www.acg.edu/the-kidsare-asking), all wishing to continue with their cultural activity and maintain operation as normal as possible.

Especially museums increased their online presence by 80% during the lockdown, ofering more social media interactions, broadcasts, virtual tours and online exhibitions, and observed more than 40% increase in online visits [ 17 ]. Online museum presence may take several forms, ranging from traditional online catalogues and digital exhibitions that include narratives and audio-visual content to virtual representations of the museum’s physical environment, in which online visitors may move and closely observe the exhibits on display, thus simulating the actual visiting experience (see for instance the Google Art Project, where several museum spaces have been represented, or custom applications such as the VR Museum of Fine Art, available on steam - https://store.steampowered.com/app/515020/The_VR_Museum_ of_Fine_Art), or “Hold the World”, with David Attenborough virtually guiding remote visitors at the collections of London’s Natural History Museum (https://www.factory42.uk/holdtheworld), among others).

Despite the wealth of cultural resources that are currently accessible online and the remarkable diversity in the adopted experience design approaches, a close look at the most popular and most visited online museums during the lockdown (https://www.theguardian. com/travel/2020/mar/23/10-of-the-worlds-best-virtual-museum-andart-gallery-tours) shows that cultural online visits are commonly designed as private, single-user experiences. However, museum studies have repeatedly highlighted the importance of the social context [ 10 ], suggesting that social interactions are key elements in the design of engaging cultural visits.

In this paper, we posit that the digital empowerment of sociality is an important and pertinent issue not only in the museums’ physical space but also in the virtual museum world. Moreover, we claim that virtual visits ofer an incredible opportunity to cope with the physical constraints that may be posed to one or more members of a social group, enabling them to experience a joint, synchronous, social activity without requiring to be co-located in the same physical place - a need that highly increased due to the recent pandemic lockdown. We thus, report practices that promote social interactions and discuss important challenges. 2

FROM PHYSICAL TO DIGITAL

The advent of online leisure and edutainment at the beginning of the new millennium created a fierce competition for onsite museum experiences and brought about a signicfiant response by museum professionals and researchers who rallied to highlight the advantages of physical visits and interactions with the curated content. As is evident in the museum research literature, apart from the unique and powerful features that make physical objects and places springboards for learning, engagement and authentic experiences [ 15 ], one of the most valued aspects of the museum experience is related to the socio-cultural characteristics of a visit [ 10 ]. Social interaction is pervasive in museums and cultural heritage sites and enhances learning, even in cases lacking such a design intention or mandate [ 2 ]. In this light, the first digital interactive applications that attempted to provide more rich and engaging visitor interpretation were heavily criticized for prioritizing the individual user while overthrowing the inherent social afordances of exhibits for co-participation and collaboration as processes of knowledge construction [ 12 ]. This led to the growth of a promising research area looking at ways to enhance digitally aided social experiences in the context of museums and cultural heritage sites [ 13, 19 ]. 3

SYNCHRONOUS AND ASYNCHRONOUS

SOCIAL INTERACTIONS

Sociality may be examined through several perspectives. We first classify visitor interactions as synchronous or asynchronous and highlight the social afordances ofered in each case, while focusing on the latter category.

Asynchronous interactions are typically accomplished through “content sharing”. Realizing the value of visitor participation, as insightfully elaborated in the definition of the “Participatory Museum”, several works have aspired to empower active visitor involvement by supporting the creation and public access to usergenerated content (e.g. comments or tags, personal stories, emotions, opinions etc), or even the authoring of personal digital collections and cultural exhibitions. These systems cultivate the concept of “museums as social places” and aim to promote social awareness, showcasing the presence of prior museum visitors and visualizing their contributions. This objective is addressed both in the scope of onsite and remote visits. Some systems deployed for museum environments include Imprints [ 3 ], ArtLinks [ 8 ], MobiTags [ 7 ], and the Room of Opinion at the Hunt Museum [ 11 ]. In some cases, the user-generated content is also leveraged for improving public access and engagement with on-line cultural collections. See for instance the steve.museum project [ 5 ] where online visitors are prompted to annotate the displayed artworks with free-form tags. The collected data is used to create folksonomies that indicate the social perception of the corresponding artworks and enable a socially derived retrieval mechanism for accessing online collections. In all the above approaches, visitor interactions are indirect; they are accomplished by providing shared access to objects and applications, which occur at diferent times and without prescribing direct communication between the parties involved.

We believe that synchronous interaction is a key element that notably fosters the social context of cultural experiences. Previous work on the topic has experimented with systems that support museum co-visits between onsite and remote visitors suggesting that social presence is closely related to engagement and enjoyment and thus, an improved remote experience [ 14, 18 ]. When it comes to entirely online strategies, besides ofering free access to digital resources, some institutions have opted for broadcasting at specific times. Leaving aside the proprietary issues that may have motivated this approach, there is also a social side efect of the broadcasting scheme. But how will the users be aware of their common activity? In the broadcasting scheme, there is no shared environment of interaction between the spectators, yet we believe that its social dimension may be notably increased by digitally mediating the co-presence of other visitors: from indications of who or how many people are watching the same cultural content at the same time, to personalized notifications about acquaintances (obtained from social networking profiles) who are currently watching or now joining the same activity. The use of awareness supporting technologies that are traditionally applied in collaborative systems (see [ 16 ] for a recent systematic literature review) may also benefit the design of cultural experiences to foster their social dimension, particularly when experienced remotely.

Delving into the matter of spectators’ representation, a novel scheme of “live performances in Virtual Reality worlds” was recently proposed [http://hamletvr.org]: To Be With Hamlet is a live theater performance that enables collocated groups of visitors to navigate in the virtual 3D story world using VIVE equipment, in which they may see their fellow audience members alongside the live performance of actors (captured with motion-tracking technologies). Although this example is targeted towards collocated experiences, we believe that it paves the way for a new form of cultural productions, showcasing how virtual 3D environments may serve as the stage of live performances where audience members are explicitly represented through avatars and can socially interact with their companions.

During the pandemic outbreak, several conferences took place in the virtual world like IEEE VR, http://ieeevr.org/2020, (ofered a desktop VR option through mozilla hubs, slack and a teleconferencing system, with presentations happening in VR according to the participant’s decision). Due to the increased need, ACM also published a guide for virtual conferences with multiple platforms and tools [ 1 ]. The first objective of using these platforms was to provide a shared multimedia presentation environment that would support question-answering interaction between the presenter and the audience. In the cultural experiences ofered by the Natural History Museum (London), the same goal was addressed through a diferent scheme, i.e. by live broadcasting scientists’ talks in which question-answering was accomplished using social media platforms, an approach that is nowadays often used alongside physical conferences as well, taking more and more ground. In this case, the focus Cultural Heritage and Social Experiences in the Times of COVID 19 of the experience design is to serve the communication between the presenter and the audience, rather than the social interaction between audience members (although such instances often occur as well). But was this objective reached? Did these platforms foster social interactions between the attendees? Despite the significant technological advances in related hardware and software components, we observe that the promotion of social interactions within virtual worlds continues to raise significant challenges, echoed also by prior research in the cultural heritage field.

4 CHALLENGES AND GUIDELINES FOR THE

FUTURE

COVID 19 found cultural institutions unprepared, without an established and efective virtual presence for something other than dissemination of their on-site activities. Those few, on the other hand, ofering digital products possibly lacked the approach to disseminate them under the right context and in the proper channels. One would argue that this critique is harsh. As discussed in [ 21 ], many institutions, going beyond a basic web presence with general information, ofer diferent types of digitized content, available after the organized investment on the digitization of the past decades. Many museum websites ofer access to hundreds of digitized artifacts through high-quality images and their relevant information. These oferings have, in theory, “opened up the museums to wider and more diverse user groups” and have led to an increase in the number of on-line visitors. A great percentage of these visitors, however, leave the website after looking at one or two pages, generally in less than 10 seconds [ 6 ].

As regards onsite visits, during the times of COVID 19 museums are likely to face significant challenges that will influence social interaction design in the following areas: a) visitor strategy and management policies, b) spatial configurations and exhibition design and, c) movement flow based on social distancing parameters. Considering the above, will we witness a return to individual user interfaces? if so, will we still continue to design for social interactions? or is this something that due to the necessity of staying safe, keeping the numbers down and the curve flat, will be completely abandoned?

Overall, cultural institutions need to re-invent the way they approach both their audience and technology, they need to transform their narratives from object-centric to people-centric. “Stories are about people, not things”, as discussed in [ 9 ] and stories have always been the most basic method we communicated experiences, oral traditions and knowledge, always in a social context where the storyteller and their audience are involved in a dynamic and interactive process and often even exchange roles. This simple and fundamental truth, if applied in a museum context, would have the potential to create connection and engagement. Technology and the vast possibilities for digital experiences, on the other hand, should be put at the service of this principle instead of being an objective by themselves. We need to design for sociality, either onsite or in virtual spaces, and we need to place afective connection, perspective-taking and empathy at the center of this design.

As regards collaborative interfaces, designing for intra-group social interactions between strangers has been a persistent research challenge, however in this climate, it may well become an irrelevant or obsolete endeavor. Fully immersive spaces of interpretation and co-located social interaction such as Caves, Panoramic Cylinders and iCinemas may also witness a severe decline because they are very costly to create while the chances of revenue due to social distancing measures are significantly reduced. Nevertheless, in the case of families and dyads, it is important to build on previous work that enhances sociability through conversation [ 13, 19 ].

In terms of providing a shared environment for online interactions, virtual museums need to first support direct voice or/and text-based communication between visitors. To avoid attention trapping on separate interface elements, “bubble-based” approaches are often employed for text-based interaction - as for instance in the virtual multi-user museum of Piet Mondrian by MUSEUM3D (https://museum3d.eu) or in the virtual exhibition implemented by [ 20 ].

Moreover, the representation of the virtual self and others includes several complementary aspects, such as avatar form, gaze, facial expressions, gestures and posture, which continue to pose important challenges in the virtual reality community. Comparing to collocated interactions that unfold in physical museum environments, social encounters in virtual worlds often need to cope with the absence of behavioral social cues (particularly when experienced with desktop-based interfaces), which however provide an important, indirect way of communication. We expect that in the virtual museum context these issues will be suficiently addressed to enforce the social dimension of cultural visits.

We observe that only a few systems have addressed the “hybrid visit scenario”, enabling local and remote visitors to share their experience in real-time and communicate with each other [ 4, 20 ]. In the after COVID 19 era, hybrid visits may potentially provide a very promising use-case, ofering important socialization opportunities for vulnerable groups, who are currently facing the danger of being left out.

Finally, it seems that the technology is there to support social experiences in cultural heritage, either onsite or online. The technology has been extensively tested and validated in many contexts, and for many years, from gaming applications to social media. The main challenge lies in the successful application of these approaches in defining concrete and relevant objectives and realizing them through an informed experience concept design. Cultural institutions are complex ecosystems and social experiences are important parts. The during and post-COVID museum experience should include carefully designed social interactions, respecting the social distancing requirements of diferent groups of visitors and users.

References

[1] ACM Presidential Task Force . 2020 . Virtual conferences: A guide to best practices. Report on What Conferences Can Do to Replace Face-to-Face Meetings . https://people.clarkson.edu/~jmatthew/acm/VirtualConferences_ GuideToBestPractices_CURRENT.pdf

[2] Sue Allen . 2003 . Looking for learning in visitor talk: A methodological exploration . In Learning conversations in museums. Routledge , 265 - 309 .

[3] Kirsten Boehner , Jennifer Thom-Santelli, Angela Zoss, Geri Gay, Justin S. Hall, and Tucker Barrett . 2005 . Imprints of Place: Creative Expressions of the Museum Experience . In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05) . Association for Computing Machinery , New York, NY, USA, 1220 - 1223 . https://doi.org/10.1145/1056808.1056881

[4] Barry Brown , Ian MacColl, Matthew Chalmers, Areti Galani, Clif Randell, and Anthony Steed . 2003 . Lessons from the Lighthouse: Collaboration in a Shared Mixed Reality System . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03) . Association for Computing Machinery , New York, NY, USA, 577 - 584 . https://doi.org/10.1145/642611.642711

[5] Susan Chun , Rich Cherry, Doug Hiwiller, Jennifer Trant, and Bruce Wyman . 2006 . Steve. museum: an ongoing experiment in social tagging, folksonomy, and museums . In Museums and the Web . 22 - 25 .

[6] CIBER Research . 2013 . Europeana 2012-2013: usage and performance update : Tech. rep., CIBER Research . http://ciber-research.eu/

[7] Dan Cosley , Jonathan Baxter, Soyoung Lee , Brian Alson, Saeko Nomura, Phil Adams, Chethan Sarabu, and Geri Gay . 2009 . A Tag in the Hand: Supporting Semantic, Social, and Spatial Navigation in Museums . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09) . Association for Computing Machinery , New York, NY, USA, 1953 - 1962 . https://doi.org/10.1145/ 1518701.1518999

[8] Dan Cosley , Joel Lewenstein, Andrew Herman, Jenna Holloway, Jonathan Baxter, Saeko Nomura, Kirsten Boehner, and Geri Gay . 2008 . ArtLinks: Fostering Social Awareness and Reflection in Museums . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08) . Association for Computing Machinery , New York, NY, USA, 403 -- 412 . https://doi.org/10.1145/1357054. 1357121

[9] Chris Crawford . 2012 . Chris Crawford on interactive storytelling . New Riders.

[10] John H. Falk and Lynn D. Dierking . 2000 . Learning from museums: Visitor experiences and the making of meaning.

[11] Kieran Ferris , Liam Bannon, Luigina Ciolfi, Paul Gallagher, Tony Hall, and Marilyn Lennon . 2004 . Shaping Experiences in the Hunt Museum: A Design Case Study . In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '04) . Association for Computing Machinery , New York, NY, USA, 205 - 214 . https://doi.org/10.1145/1013115.1013144

[12] Christian Heath and Dirk vom Lehn. 2008 . Configuring 'Interactivity'. Social Studies of Science 38 , 1 (Feb. 2008 ), 63 - 91 . https://doi.org/10.1177/0306312707084152

[13] Akrivi Katifori , Sara Perry, Maria Vayanou, Angeliki Antoniou, IoannisPanagiotis Ioannidis, Sierra McKinney , Angeliki Chrysanthi , and Yannis Ioannidis . 2020 . “Let Them Talk!”: Exploring Guided Group Interaction in Digital Storytelling Experiences. J. Comput. Cult. Herit . 13 , 3 , Article 21 ( Aug . 2020 ), 30 pages. https://doi.org/10.1145/3382773

[14] Galena Kostoska , Marcos Baez, Florian Daniel, and Fabio Casati . 2015 . Virtual, remote participation in museum visits by older adults: a feasibility study . In 8th International Workshop on Personalized Access to Cultural Heritage (PATCH 2015 ), ACM IUI 2015 . CEUR-WS .org, Aachen, DE, 1- 4 .

[15] Gaea Leinhardt and Kevin Crowley . 2002 . Objects of learning, objects of talk: Changing minds in museums. Perspectives on object-centered learning in museums ( 2002 ), 301 - 324 . http://www.museumlearning.org/leinhardtcrowley.pdf

[16] Gustavo Lopez and Luis A. Guerrero . 2017 . Awareness Supporting Technologies Used in Collaborative Systems: A Systematic Literature Review . In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17) . Association for Computing Machinery , New York, NY, USA, 808 -- 820 . https://doi.org/10.1145/2998181.2998281

[17] Network of European Museum Organisations . 2020 . NEMO report on the impact of COVID-19 on museums in Europe . https://www.ne-mo.org/fileadmin/Dateien/ public/NEMO_documents/NEMO_COVID19 _Report_12.05 . 2020 .pdf

[18] Galena Pisoni , Florian Daniel, Fabio Casati, Charles Callaway, and Oliviero Stock . 2019 . Interactive Remote Museum Visits for Older Adults: An Evaluation of Feelings of Presence, Social Closeness, Engagement, and Enjoyment in an Social Visit . In 2019 IEEE International Symposium on Multimedia (ISM) . IEEE. https: //doi.org/10.1109/ism46123. 2019 .00023

[19] Maria Vayanou , Akrivi Katifori, Angeliki Antoniou, and Angeliki Chrysanthi . 2016 . Collocated Interaction in Cultural Storytelling Experiences: How to Coordinate Visitor Actions . In proceedings of the workshop Collocated Interaction: New Challenges in'Same Time, Same Place'Research , CSCW Conference.

[20] Spyros Vosinakis and Ioannis Xenakis . 2011 . A Virtual World Installation in an Art Exhibition: Providing a Shared Interaction Space for Local and Remote Visitors . Rethinking Technology in Museums ( 2011 ).

[21] David Walsh , Mark M. Hall , Paul Clough, and Jonathan Foster . 2018 . Characterising Online Museum Users: A Study of the National Museums Liverpool Museum Website . International Journal on Digital Libraries 21 , 1 ( July 2018 ), 75 - 87 . https://doi.org/10.1007/s00799-018-0248-8