Towards Advanced Interfaces for Citizen Curation

Abstract

The SPICE project builds on the growing trend for museums, rather than providing an authoritative view, to present multiple voices related to their collection and exhibitions. In SPICE, an approach we term citizen curation is proposed as a way of supporting visitors to share their own interpretations of museum objects and reflect on the variety of interpretations contributed by others. In order to capture a wide range of voices, interfaces will be designed specifically to engage minority groups that tend to be under-represented in cultural activities. To achieve this goal, the interface will need to be intuitive, aesthetic and accessible for diferent audiences. The paper presents the challenges we face and initial proposals for engaging visitors in citizen curation.

CCS CONCEPTS

• Information systems → Asynchronous editors. 1

INTRODUCTION

Museums, rather than providing an authoritative view, increasingly attempt to present multiple voices related to their collection and exhibitions, including from the museum visitors themselves [ 16 ]. This creates a number of methodological and technical challenges in particular: how to support people in interpreting cultural objects for themselves; how to support both museums and visitors in exploring and reflecting on the range of accumulated contributions; and how to engage under-represented groups in the process, such as older people, whose voices are least likely to be heard. To address these challenges, we are developing tools and methods to support a process we term Citizen Curation. Methods will be co-designed that can be used by citizen groups to produce personal interpretations of cultural objects and analyze and compare them against the interpretations of others. Tools will be developed for modelling users and groups and recommending content in a way that assists citizen groups in building a representation of themselves and appreciating variety within groups and similarity across groups, to enhance social cohesion. A Linked Data (LD) infrastructure will support citizen curation using social media platforms in a way that gives heritage institutions control over rights protected digital assets and access to citizens’ responses to their collections. User experiences will be designed that enable inclusive participation in citizen curation activities across cultures and abilities. A series of citizen curation case studies with a diverse set of museums and citizen groups will demonstrate how the approach can promote inclusive participation and social cohesion in a variety of contexts. Engaging visitors is one of the major challenges of the project. This requires an integrative approach, encompassing the content to be delivered, user interface and interaction design. These challenges and ideas for addressing them are discussed in this position paper. 2

THE SPICE PROJECT

Tools and methods to support citizen curation are being developed as part of the H2020 funded SPICE project (https://w3id.org/spice). We define citizen curation as citizens applying curatorial methods to archival materials available in memory institutions in order to develop their own interpretations, share their own perspective and appreciate the perspectives of others. Our definition is informed by previous initiatives that have engaged citizens in the curatorial process. Mauer [ 9 ] and Hill et al. [ 7 ] describe citizen curation as a process in which citizens with little or no background in museum curation are taught and guided to create their own physical and virtual exhibitions. Moqtaderi [ 10 ] uses the term citizen curator to describe members of the public voting for an artwork to be included in an exhibition curated by the museum. The citizen curator initiative developed by Ride [ 13 ] involved citizens sharing contributions via Twitter which were later used in a video installation developed by the museum. Within SPICE, we aim to engage citizens in curatorially-inspired activities that personally engage the visitor and promote reflection, and are also accessible and open to all without additional training.

Our approach will provide scripted support for a range of citizen curation activities (see Figure 1 – the numbers in the description refer to items in the figure). We envision a distributed architecture leveraging LD, where each system has access to a number of objects, whether provided by one or more museum content management systems, other repositories, or contributed by citizens (1). Each system has its own autonomy in modelling and managing data, including specific ontologies (2). In addition, systems implement a set of scripts (3). Each script is made up of a combination of interpretation activities (e.g. collecting, artefact analysis) and reflection activities (e.g. search, visualizing similarities and diferences). These scripts difer across the systems to fit the specificity of use cases and may organize activities into diferent orders. For example, the citizen may see an overview of what contributions others have made (i.e. reflection) before and/or after making their own contribution (i.e. interpretation). Interpretation activities will be developed that draw on various curatorial methods related to the study, interpretation and communication of museum objects, such as collecting, artefact analysis, storytelling, and exhibition design (4). Activities will vary depending on the level of involvement required in order to appeal to both casual and engaged visitors. Activities inspired by collecting will enable visitors to produce their own collection (5). Casual activities could involve tracking the visitor in the (virtual or physical) museum space and constructing a collection automatically based on their dwell time in front of particular objects. More engaged activities could involve the visitor (such as a child or family group) ifnding objects in response to challenges in a form of treasure hunt.

Activities inspired by artefact analysis will enable visitors to ofer their own interpretation in response to particular objects (6). Casual activities could involve associating a caption or title with an artwork. More engaged activities could involve telling a story related to, for example, the characters, setting or theme of the artwork. Visitors could also be assisted in developing their own exhibitions (7). Casual activities could involve exhibitions being generated automatically from the path a visitor takes through the museum space. More engaged methods could lead the visitor through a curatorial design process in which they thematically organize and present a series of objects. Methods of reflection enable those interpretations to be used to foster mutual understanding and support research (8). Forms of analysis include the searching and browsing of content and the identification of similarities and diferences across contributions. Searching and browsing activities can be used to suggest to the user alternative interpretations of the same object in order to encourage them to recognize and take multiple perspectives (9). Visual analytical tools will support the user in understanding more broadly the variety of responses produced by a particular activity (item 10). Visualizations will emphasize the variety rather than the popularity or similarity of opinions. Visual analytical tools will also support the investigation of similarities across diferent groups enabling citizens to identify issues of common concern rather than just emphasize average diferences in response (11). Analytical tools will also enable the responses stored in the LD servers to be used as a research resource (12). The architecture would enable citizen tagging and interpretation activities with particular communities in order to build vocabularies that can be used to associate that group with cultural objects. The responses made to interpretative activities can be researched in a number of ways, analyzing the nature and depth of the cultural participation diferent activities facilitate and the diferent sociocultural perspectives that can be brought to bear on the same cultural objects. 3

THE CHALLENGES OF ENGAGING

VISITORS

One of the main challenges we face is visitor engagement: how to attract the visitors’ attention, get them to interact with the objects in the exhibition, the content about them, interpret the content, explore it and reflect on it. Achieving this goal requires an integrated approach, which includes engaging interfaces, engaging and attractive content, and engaging interaction. 3.1

Interface

The user interface is the point of contact between the user and the system, hence it has a key role in the process of engaging citizens with the curatorial activities proposed by the interpretationreflection loop embedded in the system-user interaction process. It has the role of attracting users’ attention, whether visually or auditorily (or in combination of the two), getting their attention and encouraging them to start interacting with the system. Hence an engaging and attractive interface is a key element in initiating interaction. At the same time, as the project includes target groups of diferent ages, abilities and requirements, the use of specific media types (sound, text, etc.) should be traded-of with the preferences and limitations of potential users (think for example of hearing or visually impaired museum visitors). 3.2

Content

Having an engaging and attractive interface may be a good starting point. However, there is a need for relevant, interesting and engaging content to be presented to the visitors. The content should be presented in an attractive way - evoking emotions, triggering curiosity - i.e. it should make the visitor curious to explore what is there for them. The content delivered should be then composed (at least in part) on the fly, considering the profile, competences, and interests of an individual or a group. The semantically specified LD in SPICE will enable the required flexibility. 3.3

Interaction

An engaging interface is a must, as much as intriguing and interesting content. Still the way the content is delivered to the visitors (e.g. the interaction) needs to be carefully designed so to keep the user engaged. Within the museum, interaction needs to considered not only in terms of the design of the interface but also the context of use: visiting individually or in a group, the time available to the visitor, the primacy of the cultural objects that have motivated the visit, and the physical layout of the museum including its galleries, family room and grounds. 4

INITIAL IDEAS FOR ENGAGING VISITORS

We aim at a co-design process for developing the solution. As Sanders and Stappers [ 15 ] note, a key benefit of co-design activities is not only that they can enable collaboratively organized creativity but also how latent needs can be jointly explored in a supportive environment. The participatory space includes museum professionals, developers, designers, humanities scholars as well as end-user communities. Five Case Studies each dealing with distinct user communities in five diferent languages will participate in application development, testing and demonstration of the citizen curation methods. Each of the Case Studies will cover a facet of the accessibility and inclusiveness dilemma: generational and geographical distance; disruption due to conflict and illness; lack of access to educational resources resulting in a lack of interest in learning; as well as political and religious conflict are among the topics that will be handled in these pilot applications. 4.1

The Interpretation-Reflection Loop

The SPICE platform will embed methods to stimulate and prompt citizens to produce their own interpretations of cultural objects and artefacts in museums and heritage institutions. Once meaningful discourses and narratives have been generated and contributed by the interpretation process (e.g. narratives of personal or collective identity, generated by engaging with cultural artefacts), we can trace patterns that can aggregate such interpretations to support mutual understanding and reflection among participants. For this purpose, we will explore a repertoire of activities in order to test which ones are more efective in stimulating a variety of audiences to produce interpretations and perspectives that in turn will enable them to participate in a rich and diverse cultural space for reflection. We envision that the interpretation methods will have the potential to assist citizens in articulating their own points of view (perspective-making), as well as understanding the views of others (perspective-taking), turning the museum into a safe space for unsafe ideas, in which the museum, rather than providing an authoritative view, accommodates multiple voices [ 1 ]. 4.2

Emotional Engagement

Often considered an essential element of people’s response to artworks, emotions can be the key to engage visitors in the interpretation of cultural objects. Previous work on the emotional response to art has provided insight on the emotions elicited by art, with some non-obvious implications for the SPICE project. Saif and Kiritchenko [ 14 ] have investigated the expression of emotions evoked by visual art by applying Sentiment Analysis Techniques to the tags added to artworks by crowdsourced annotators. Their results confirm the relevance of emotions in the collected tags. However, they focus on the homogeneity of the emotional response and on its relations with the visual content of artworks, while SPICE aims at diversity rather than accordance. In addition, recent work by Rao et al. [ 12 ] has investigated the role of emotional priming in the production of narrative content.

Emotive (https://emotiveproject.eu) European project [ 11 ] has further analyzed the role of emotions in users’ engagement as with Cultural Heritage (CH), testing it in collaborative interactive and immersive virtual reality environments, where a required cooperation led two players to discuss abstract concepts (such as rituals, artistic expression, sharing), developing comparisons with their own experience. Within this specific context, it was possible to identify the role of social interaction in triggering emotions and understanding concepts. A further study, conducted within the GIFT EU project (https://gifting.digital), has set up an experiment where diferent stimuli (audio evocative narrative and visual stimuli of a selection of artworks) were used during the museum visit, while the emotional feedback was double checked through a portable EEG and the indications of the visitors themselves. The result led to developing a prototype of an ‘emotion mapper’ (https://gifting.digital/emotionmapper).

Recent studies in neurosciences are also demonstrating that some stimuli could create a resonation between two people, even if the stimulus is provided to only one [ 4, 5 ], opening up a number of new potential uses of this discovery. Experiments in this direction could help SPICE to better identify those mechanisms in association with specific multimedia. The experiments conducted by the authors have demonstrated that, if users are exposed to emotional stimuli, they tend to create emotionally richer narrative content. Moreover, the role of interaction, of the exchange among participants, of the selection of specific stimuli, has also been found to have an impact on emotion and engagement. Although the work by Rao et al. [ 12 ] has revealed that producing emotional expressions (rather than being exposed to them) was the most efective priming type, the design of user engagement devices in SPICE should take into account the role of emotional priming, using afective elements of diferent media types to promote emotional engagement. 4.3

Attracting attention

Encouraging visitors to interact with a system, providing their interpretation and reflecting on what others have said is not an easy task, as users do not tend to engage themselves in lengthy interaction while visiting CH sites. However, there is a variety of persuasion or marketing techniques that may be subtly applied in order to attract their attention and tempt them to interact. One option may be to follow the COMBI Model suggested by Klein et al. [ 8 ] that was developed for behavior change and includes a gradual persuasion in five stages, involving precontemplation, contemplation, preparation, action, and maintenance. These stages are devised to lead the user to be aware, then to be motivated and finally to act. Another example may be Digital Nudging [ 17 ], which is the use of user-interface design elements to guide people’s behavior in digital choice environments. 4.4

Linked Data Social Media Layer

A distributed LD infrastructure will be developed to connect cultural objects, collections, and citizen contributions (e.g. interpretations). Similarly to typical social media, such as Facebook or Pinterest, it will permit users and organizations to: search and retrieve digital artworks from multiple sources, share digital objects, share curated presentations containing the artworks, and link to external content published on the Web. Drawing on the concepts of LD, Distributed Online Social Network, and Digital Asset Management, we will develop an innovative technology stack for supporting citizen curation. The resulting distributed system will be a semantically-aware network of systems, organizations, datasets, and services that will constitute a policy and privacy-aware environment for the construction of a new generation of tools for cultural engagement. Data will be integrated from multiple points, leaving partners and institutions free to use their own systems, vocabularies, schemas. SPICE will provide methods to automatically extract knowledge from text, to reconcile data expressed in diferent schemas, to share ontologies that facilitate data interoperability, interpretation sharing, flexible scripting, similarity/diference identification, and recommendation capabilities. 4.5

Gamification

In CH the exploitation of narrative based exploration as a method for the suggestion of museum artworks, represents one of the most dificult yet promising attempts for users’ engagement. Such a method has been already adopted in projects like Labyrinth [ 2, 3 ], where the use of semantic descriptions of the narrative content of artefacts was a driver for visual experiences in both web and 3D visual interfaces. In SPICE, the overall approach targeting narrative explorations relies on the use of a network of ontologies to model narrative concepts. Narrative ontologies (including and connecting notions concerning stories, roles, characters and events described by a given item) have been proposed with the aim of enabling the discovery of unknown and serendipitous connections between artworks, so as to expand and support the available connections through automated reasoning. Given these narrative descriptions, in fact, several relations can be detected: beside classical standard relations based on author or resource type, indeed, narrative content relations enable the discovery of artworks that display the same characters, depict the same action type (e.g. “killing” or “kissing”), refer to the same story and its related stories. In this line of investigation, then, the discovery of the read threads connecting artworks can represent the starting point for gamification experiences involving diferent groups of visitors and bring to the front shared archetypal stories connecting individuals and groups.

Storytelling

Storytelling is a well-known technique for engaging the public in interaction. In SPICE, we will review the current state of datacentric, ontology-based storytelling (see Winer [ 18 ] for a survey), reusing experiences such as MakeBelieve, DRAMMAR, and Griot, which have used formal semantics to generate conceptual structures customized to storytelling purposes. Thanks to a multi-layered design of ontologies and data, previously applied to represent legal norm entrenchment, sentiment semantics, metaphors [ 6 ], etc., the results of the SPICE interpretation-reflection loop, jointly with scripting and CH knowledge graphs, will feed ontology-based narrative schemas, adapting storytelling to the entrenched nature of multiple individual /group perspectives 5

SUMMARY

SPICE aims for a number of very challenging goals: a) to support visitors in interpreting cultural objects for themselves; b) to support both museums and visitors in exploring and reflecting on the range of accumulated contributions; and c) to develop the tools and methods that can support a wide range of voices being heard, including from minority groups. These challenging goals require a sophisticated infrastructure, as presented. However, eventually everything boils down to the point of interaction - the user interface. We claim that the project success heavily depends upon interaction design, the design of the user interface and the content itself. Hence we present our initial ideas about ways to address the above-mentioned challenges.

ACKNOWLEDGMENTS

This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 870811.

References

[1] Fiona Cameron . 2008 . Safe Places for Unsafe Ideas?: History and Science Museums, Hot Topics and Moral Predicaments . Social History in Museums 32 ( 2008 ), 5 - 16 .

[2] Rossana Damiano , Vincenzo Lombardo, and Antonio Lieto. 2015 . Visual Metaphors for Semantic Cultural Heritage . In Proceedings of the 2015 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN) (INTETAIN '15) . IEEE Computer Society, USA, 100 -- 109 .

[3] Rossana Damiano , Vincenzo Lombardo, Antonio Lieto, and Davide Borra . 2016 . Exploring Cultural Heritage Repositories with Creative Intelligence . The Labyrinth 3D System. Entertainment Computing 16 ( July 2016 ), 41 - 52 . https://doi.org/10.1016/j.entcom. 2016 . 05 .002

[4] Guillaume Dumas , Mario Chavez, Jacqueline Nadel, and Jacques Martinerie . 2012 . Anatomical Connectivity Influences both Intra- and Inter-Brain Synchronizations . PLoS ONE 7 , 5 (May 2012 ), e36414 . https://doi.org/10.1371/journal.pone.0036414

[5] Guillaume Dumas , Jacqueline Nadel, Robert Soussignan, Jacques Martinerie, and Line Garnero . 2010 . Inter-Brain Synchronization during Social Interaction . PLoS ONE 5 , 8 (Aug. 2010 ), e12166 . https://doi.org/10.1371/journal.pone.0012166

[6] Aldo Gangemi , Mehwish Alam, and Valentina Presutti . 2018 . Amnestic Forgery: An Ontology of Conceptual Metaphors . arXiv preprint arXiv: 1805 . 12115 ( 2018 ).

[7] Amanda Hill , Mark Kretzschmar , David Morton, and Sara Rafel . 2018 . ” Eenie Meenie Miney Mose”: Using Experimental Citizen Curating To Engage Visitors With Racial Ephemera . Florida Studies Review 62 ( 2018 ).

[8] Michel Klein , Nataliya Mogles, and Arlette van Wissen. 2014 . Intelligent Mobile Support for Therapy Adherence and Behavior Change . Journal of Biomedical Informatics 51, C ( Oct . 2014 ), 137 - 151 . https://doi.org/10.1016/j.jbi. 2014 . 05 .005

[9] Barry J Mauer . 2017 . The Citizen Curating Project Confronts the Pulse Nightclub Shooting. The St . John's University Humanities Review ( 2017 ).

[10] Heather Moqtaderi . 2019 . Citizen Curators: Crowdsourcing to Bridge the Academic/Public Divide. University Museums and Collections Journal 11 ( 2019 ), 204 - 210 . Issue 2.

[11] Sara Perry . 2019 . The Enchantment of the Archaeological Record . European Journal of Archaeology 22 , 3 ( June 2019 ), 354 - 371 . https://doi.org/10.1017/eaa. 2019 .24

[12] Nanjie Rao , Sharon Lynn Chu, Randi Weitzen Faris, and Daniel Ospina . 2019 . The Efects of Interactive Emotional Priming on Storytelling: An Exploratory Study . In Interactive Storytelling . Springer International Publishing, 395 - 404 . https://doi.org/10.1007/978-3- 030 -33894-7_ 42

[13] Peter Ride . 2013 . Creating# citizencurators: Putting Twitter Into museum Showcases . In Proceedings of the 19th International Symposium of Electronic Art , K. Cleland , L. Fisher, and R. Harley (Eds.). ISEA International.

[14] Mohammad Saif and Svetlana Kiritchenko . 2018 . WikiArt Emotions: An Annotated Dataset of Emotions Evoked by Art . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018 ). European Language Resources Association (ELRA), Miyazaki , Japan. https: //www.aclweb.org/anthology/L18-1197

[15] Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008 . Co-creation and the New Landscapes of Design . CoDesign 4 , 1 (March 2008 ), 5 - 18 . https://doi.org/10.1080/ 15710880701875068

[16] Nina Simon . 2010 . The Participatory Museum (1 ed .). Museum 2 .0, Santa Cruz , CA, USA.

[17] Markus Weinmann , Christoph Schneider, and Jan vom Brocke . 2016 . Digital Nudging . Business & Information Systems Engineering 58 , 6 (Oct. 2016 ), 433 - 436 . https://doi.org/10.1007/s12599-016-0453-1

[18] Dov Winer . 2014 . Review of Ontology Based Storytelling Devices . In Language, Culture, Computation. Computing of the Humanities, Law, and Narratives . Springer Berlin Heidelberg, 394 - 405 . https://doi.org/10.1007/978-3- 642 -45324-3_ 12