Towards Advanced Interfaces for Citizen Curation Luis Emilio Bruni Enrico Daga Rossana Damiano Department of Architecture, Design Knowledge Media Institute, The Open Computer Science and CIRMA, and Media Technology University University of Turin Alborg University, Denmark Milton Keynes, UK Turin, Italy leb@create.aau.dk enrico.daga@open.ac.uk rossana.damiano@unito.it Lily Diaz Tsvi Kuflik Antonio Lieto Department of Media, Media lab, Information Systems, The University Computer Science and CIRMA, Aalto University of Haifa University of Turin Aalto, Finland Haifa, Israel Turin, Italy lily.diaz@aaltol.fi tsvikak@is.haifa.ac.il antonio.lieto@unito.it Aldo Gangemi Paul Mulholland Silivio Peroni DHARC - Department of Classical Knowledge Media Institute, The Open DHARC - Department of Classical Philology and Italian Studies, University Philology and Italian Studies, University of Bologna and ISTC-CNR Milton Keynes, UK University of Bologna Rome, Italy paul.mulholland@open.ac.uk Rome, Italy aldo.gangemi@cnr.it silvio.peroni@unibo.it Sofia Pescarin Alan Wecker CNR ISPC Information Systems, The University Rome, Italy of Haifa sofia.pescarin@cnr.it Haifa, Israel ajwecker@gmail.com ABSTRACT and Alan Wecker. 2020. Towards Advanced Interfaces for Citizen Curation. The SPICE project builds on the growing trend for museums, rather In Proceedings of 𝐴𝑉 𝐼 2𝐶𝐻 2020: Workshop on Advanced Visual Interfaces and Interactions in Cultural Heritage (𝐴𝑉 𝐼 2𝐶𝐻 2020). ACM, New York, NY, than providing an authoritative view, to present multiple voices USA, 5 pages. related to their collection and exhibitions. In SPICE, an approach we term citizen curation is proposed as a way of supporting visitors to share their own interpretations of museum objects and reflect on 1 INTRODUCTION the variety of interpretations contributed by others. In order to cap- Museums, rather than providing an authoritative view, increasingly ture a wide range of voices, interfaces will be designed specifically attempt to present multiple voices related to their collection and to engage minority groups that tend to be under-represented in exhibitions, including from the museum visitors themselves [16]. cultural activities. To achieve this goal, the interface will need to be This creates a number of methodological and technical challenges intuitive, aesthetic and accessible for different audiences. The paper in particular: how to support people in interpreting cultural objects presents the challenges we face and initial proposals for engaging for themselves; how to support both museums and visitors in ex- visitors in citizen curation. ploring and reflecting on the range of accumulated contributions; and how to engage under-represented groups in the process, such CCS CONCEPTS as older people, whose voices are least likely to be heard. To address • Information systems → Asynchronous editors. these challenges, we are developing tools and methods to support a process we term Citizen Curation. Methods will be co-designed KEYWORDS that can be used by citizen groups to produce personal interpre- tations of cultural objects and analyze and compare them against Citizen curation, User engagement, Museum information system, the interpretations of others. Tools will be developed for modelling Interpretation, Reflection. users and groups and recommending content in a way that assists ACM Reference Format: citizen groups in building a representation of themselves and ap- Luis Emilio Bruni, Enrico Daga, Rossana Damiano, Lily Diaz, Tsvi Kuflik, An- preciating variety within groups and similarity across groups, to tonio Lieto, Aldo Gangemi, Paul Mulholland, Silivio Peroni, Sofia Pescarin, enhance social cohesion. A Linked Data (LD) infrastructure will support citizen curation using social media platforms in a way 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy that gives heritage institutions control over rights protected digital © Copyright 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). assets and access to citizens’ responses to their collections. User experiences will be designed that enable inclusive participation in 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Luis Emilio Bruni et al. citizen curation activities across cultures and abilities. A series of Activities inspired by artefact analysis will enable visitors to offer citizen curation case studies with a diverse set of museums and their own interpretation in response to particular objects (6). Casual citizen groups will demonstrate how the approach can promote activities could involve associating a caption or title with an art- inclusive participation and social cohesion in a variety of contexts. work. More engaged activities could involve telling a story related Engaging visitors is one of the major challenges of the project. This to, for example, the characters, setting or theme of the artwork. requires an integrative approach, encompassing the content to be Visitors could also be assisted in developing their own exhibitions delivered, user interface and interaction design. These challenges (7). Casual activities could involve exhibitions being generated au- and ideas for addressing them are discussed in this position paper. tomatically from the path a visitor takes through the museum space. More engaged methods could lead the visitor through a curatorial design process in which they thematically organize and present a 2 THE SPICE PROJECT series of objects. Methods of reflection enable those interpretations Tools and methods to support citizen curation are being developed to be used to foster mutual understanding and support research (8). as part of the H2020 funded SPICE project (https://w3id.org/spice). Forms of analysis include the searching and browsing of content We define citizen curation as citizens applying curatorial methods and the identification of similarities and differences across contri- to archival materials available in memory institutions in order to butions. Searching and browsing activities can be used to suggest develop their own interpretations, share their own perspective and to the user alternative interpretations of the same object in order appreciate the perspectives of others. Our definition is informed to encourage them to recognize and take multiple perspectives (9). by previous initiatives that have engaged citizens in the curatorial Visual analytical tools will support the user in understanding more process. Mauer [9] and Hill et al. [7] describe citizen curation as a broadly the variety of responses produced by a particular activity process in which citizens with little or no background in museum (item 10). Visualizations will emphasize the variety rather than the curation are taught and guided to create their own physical and popularity or similarity of opinions. Visual analytical tools will virtual exhibitions. Moqtaderi [10] uses the term citizen curator also support the investigation of similarities across different groups to describe members of the public voting for an artwork to be in- enabling citizens to identify issues of common concern rather than cluded in an exhibition curated by the museum. The citizen curator just emphasize average differences in response (11). Analytical tools initiative developed by Ride [13] involved citizens sharing contri- will also enable the responses stored in the LD servers to be used butions via Twitter which were later used in a video installation as a research resource (12). The architecture would enable citizen developed by the museum. Within SPICE, we aim to engage citi- tagging and interpretation activities with particular communities zens in curatorially-inspired activities that personally engage the in order to build vocabularies that can be used to associate that visitor and promote reflection, and are also accessible and open to group with cultural objects. The responses made to interpretative all without additional training. activities can be researched in a number of ways, analyzing the Our approach will provide scripted support for a range of citizen nature and depth of the cultural participation different activities curation activities (see Figure 1 – the numbers in the description facilitate and the different sociocultural perspectives that can be refer to items in the figure). We envision a distributed architecture brought to bear on the same cultural objects. leveraging LD, where each system has access to a number of objects, whether provided by one or more museum content management systems, other repositories, or contributed by citizens (1). Each sys- 3 THE CHALLENGES OF ENGAGING tem has its own autonomy in modelling and managing data, includ- VISITORS ing specific ontologies (2). In addition, systems implement a set of One of the main challenges we face is visitor engagement: how to scripts (3). Each script is made up of a combination of interpretation attract the visitors’ attention, get them to interact with the objects activities (e.g. collecting, artefact analysis) and reflection activities in the exhibition, the content about them, interpret the content, (e.g. search, visualizing similarities and differences). These scripts explore it and reflect on it. Achieving this goal requires an inte- differ across the systems to fit the specificity of use cases and may grated approach, which includes engaging interfaces, engaging and organize activities into different orders. For example, the citizen attractive content, and engaging interaction. may see an overview of what contributions others have made (i.e. reflection) before and/or after making their own contribution (i.e. interpretation). Interpretation activities will be developed that draw 3.1 Interface on various curatorial methods related to the study, interpretation The user interface is the point of contact between the user and and communication of museum objects, such as collecting, artefact the system, hence it has a key role in the process of engaging citi- analysis, storytelling, and exhibition design (4). Activities will vary zens with the curatorial activities proposed by the interpretation- depending on the level of involvement required in order to appeal reflection loop embedded in the system-user interaction process. to both casual and engaged visitors. Activities inspired by collecting It has the role of attracting users’ attention, whether visually or will enable visitors to produce their own collection (5). Casual ac- auditorily (or in combination of the two), getting their attention tivities could involve tracking the visitor in the (virtual or physical) and encouraging them to start interacting with the system. Hence museum space and constructing a collection automatically based an engaging and attractive interface is a key element in initiating in- on their dwell time in front of particular objects. More engaged teraction. At the same time, as the project includes target groups of activities could involve the visitor (such as a child or family group) different ages, abilities and requirements, the use of specific media finding objects in response to challenges in a form of treasure hunt. types (sound, text, etc.) should be traded-off with the preferences Towards Advanced Interfaces for Citizen Curation 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Figure 1: SPICE overall conceptual model and limitations of potential users (think for example of hearing or environment. The participatory space includes museum profession- visually impaired museum visitors). als, developers, designers, humanities scholars as well as end-user communities. Five Case Studies each dealing with distinct user com- 3.2 Content munities in five different languages will participate in application development, testing and demonstration of the citizen curation Having an engaging and attractive interface may be a good starting methods. Each of the Case Studies will cover a facet of the accessi- point. However, there is a need for relevant, interesting and en- bility and inclusiveness dilemma: generational and geographical gaging content to be presented to the visitors. The content should distance; disruption due to conflict and illness; lack of access to be presented in an attractive way - evoking emotions, triggering educational resources resulting in a lack of interest in learning; as curiosity - i.e. it should make the visitor curious to explore what is well as political and religious conflict are among the topics that will there for them. The content delivered should be then composed (at be handled in these pilot applications. least in part) on the fly, considering the profile, competences, and interests of an individual or a group. The semantically specified LD in SPICE will enable the required flexibility. 4.1 The Interpretation-Reflection Loop The SPICE platform will embed methods to stimulate and prompt 3.3 Interaction citizens to produce their own interpretations of cultural objects and artefacts in museums and heritage institutions. Once meaningful An engaging interface is a must, as much as intriguing and inter- discourses and narratives have been generated and contributed by esting content. Still the way the content is delivered to the visitors the interpretation process (e.g. narratives of personal or collective (e.g. the interaction) needs to be carefully designed so to keep the identity, generated by engaging with cultural artefacts), we can user engaged. Within the museum, interaction needs to considered trace patterns that can aggregate such interpretations to support not only in terms of the design of the interface but also the context mutual understanding and reflection among participants. For this of use: visiting individually or in a group, the time available to the purpose, we will explore a repertoire of activities in order to test visitor, the primacy of the cultural objects that have motivated the which ones are more effective in stimulating a variety of audiences visit, and the physical layout of the museum including its galleries, to produce interpretations and perspectives that in turn will en- family room and grounds. able them to participate in a rich and diverse cultural space for reflection. We envision that the interpretation methods will have 4 INITIAL IDEAS FOR ENGAGING VISITORS the potential to assist citizens in articulating their own points of We aim at a co-design process for developing the solution. As view (perspective-making), as well as understanding the views of Sanders and Stappers [15] note, a key benefit of co-design activities others (perspective-taking), turning the museum into a safe space is not only that they can enable collaboratively organized creativity for unsafe ideas, in which the museum, rather than providing an but also how latent needs can be jointly explored in a supportive authoritative view, accommodates multiple voices [1]. 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy Luis Emilio Bruni et al. 4.2 Emotional Engagement et al. [8] that was developed for behavior change and includes Often considered an essential element of people’s response to art- a gradual persuasion in five stages, involving precontemplation, works, emotions can be the key to engage visitors in the interpreta- contemplation, preparation, action, and maintenance. These stages tion of cultural objects. Previous work on the emotional response are devised to lead the user to be aware, then to be motivated to art has provided insight on the emotions elicited by art, with and finally to act. Another example may be Digital Nudging [17], some non-obvious implications for the SPICE project. Saif and Kir- which is the use of user-interface design elements to guide people’s itchenko [14] have investigated the expression of emotions evoked behavior in digital choice environments. by visual art by applying Sentiment Analysis Techniques to the tags added to artworks by crowdsourced annotators. Their results confirm the relevance of emotions in the collected tags. However, 4.4 Linked Data Social Media Layer they focus on the homogeneity of the emotional response and on A distributed LD infrastructure will be developed to connect cultural its relations with the visual content of artworks, while SPICE aims objects, collections, and citizen contributions (e.g. interpretations). at diversity rather than accordance. In addition, recent work by Similarly to typical social media, such as Facebook or Pinterest, it Rao et al. [12] has investigated the role of emotional priming in the will permit users and organizations to: search and retrieve digital production of narrative content. artworks from multiple sources, share digital objects, share curated Emotive (https://emotiveproject.eu) European project [11] has presentations containing the artworks, and link to external content further analyzed the role of emotions in users’ engagement as with published on the Web. Drawing on the concepts of LD, Distributed Cultural Heritage (CH), testing it in collaborative interactive and im- Online Social Network, and Digital Asset Management, we will mersive virtual reality environments, where a required cooperation develop an innovative technology stack for supporting citizen cura- led two players to discuss abstract concepts (such as rituals, artistic tion. The resulting distributed system will be a semantically-aware expression, sharing), developing comparisons with their own expe- network of systems, organizations, datasets, and services that will rience. Within this specific context, it was possible to identify the constitute a policy and privacy-aware environment for the construc- role of social interaction in triggering emotions and understanding tion of a new generation of tools for cultural engagement. Data will concepts. A further study, conducted within the GIFT EU project be integrated from multiple points, leaving partners and institutions (https://gifting.digital), has set up an experiment where different free to use their own systems, vocabularies, schemas. SPICE will stimuli (audio evocative narrative and visual stimuli of a selection provide methods to automatically extract knowledge from text, to of artworks) were used during the museum visit, while the emo- reconcile data expressed in different schemas, to share ontologies tional feedback was double checked through a portable EEG and the that facilitate data interoperability, interpretation sharing, flexible indications of the visitors themselves. The result led to developing scripting, similarity/difference identification, and recommendation a prototype of an ‘emotion mapper’ (https://gifting.digital/emotion- capabilities. mapper). Recent studies in neurosciences are also demonstrating that some stimuli could create a resonation between two people, even if the 4.5 Gamification stimulus is provided to only one [4, 5], opening up a number of new potential uses of this discovery. Experiments in this direction could In CH the exploitation of narrative based exploration as a method help SPICE to better identify those mechanisms in association with for the suggestion of museum artworks, represents one of the most specific multimedia. The experiments conducted by the authors difficult yet promising attempts for users’ engagement. Such a have demonstrated that, if users are exposed to emotional stimuli, method has been already adopted in projects like Labyrinth [2, 3], they tend to create emotionally richer narrative content. Moreover, where the use of semantic descriptions of the narrative content of the role of interaction, of the exchange among participants, of artefacts was a driver for visual experiences in both web and 3D the selection of specific stimuli, has also been found to have an visual interfaces. In SPICE, the overall approach targeting narrative impact on emotion and engagement. Although the work by Rao explorations relies on the use of a network of ontologies to model et al. [12] has revealed that producing emotional expressions (rather narrative concepts. Narrative ontologies (including and connecting than being exposed to them) was the most effective priming type, notions concerning stories, roles, characters and events described the design of user engagement devices in SPICE should take into by a given item) have been proposed with the aim of enabling account the role of emotional priming, using affective elements of the discovery of unknown and serendipitous connections between different media types to promote emotional engagement. artworks, so as to expand and support the available connections through automated reasoning. Given these narrative descriptions, in fact, several relations can be detected: beside classical standard 4.3 Attracting attention relations based on author or resource type, indeed, narrative con- Encouraging visitors to interact with a system, providing their tent relations enable the discovery of artworks that display the interpretation and reflecting on what others have said is not an same characters, depict the same action type (e.g. “killing” or “kiss- easy task, as users do not tend to engage themselves in lengthy ing”), refer to the same story and its related stories. In this line of interaction while visiting CH sites. However, there is a variety of investigation, then, the discovery of the read threads connecting persuasion or marketing techniques that may be subtly applied in artworks can represent the starting point for gamification experi- order to attract their attention and tempt them to interact. One ences involving different groups of visitors and bring to the front option may be to follow the COMBI Model suggested by Klein shared archetypal stories connecting individuals and groups. Towards Advanced Interfaces for Citizen Curation 𝐴𝑉 𝐼 2𝐶𝐻 2020, September 29, Island of Ischia, Italy 4.6 Storytelling [11] Sara Perry. 2019. The Enchantment of the Archaeological Record. European Journal of Archaeology 22, 3 (June 2019), 354–371. https://doi.org/10.1017/eaa. Storytelling is a well-known technique for engaging the public 2019.24 in interaction. In SPICE, we will review the current state of data- [12] Nanjie Rao, Sharon Lynn Chu, Randi Weitzen Faris, and Daniel Ospina. 2019. The Effects of Interactive Emotional Priming on Storytelling: An Exploratory centric, ontology-based storytelling (see Winer [18] for a survey), Study. In Interactive Storytelling. Springer International Publishing, 395–404. reusing experiences such as MakeBelieve, DRAMMAR, and Griot, https://doi.org/10.1007/978-3-030-33894-7_42 which have used formal semantics to generate conceptual struc- [13] Peter Ride. 2013. Creating# citizencurators: Putting Twitter Into museum Show- cases. In Proceedings of the 19th International Symposium of Electronic Art, K. Cle- tures customized to storytelling purposes. Thanks to a multi-layered land, L. Fisher, and R. Harley (Eds.). ISEA International. design of ontologies and data, previously applied to represent le- [14] Mohammad Saif and Svetlana Kiritchenko. 2018. WikiArt Emotions: An An- gal norm entrenchment, sentiment semantics, metaphors [6], etc., notated Dataset of Emotions Evoked by Art. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). the results of the SPICE interpretation-reflection loop, jointly with European Language Resources Association (ELRA), Miyazaki, Japan. https: scripting and CH knowledge graphs, will feed ontology-based nar- //www.aclweb.org/anthology/L18-1197 [15] Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008. Co-creation and the New rative schemas, adapting storytelling to the entrenched nature of Landscapes of Design. CoDesign 4, 1 (March 2008), 5–18. https://doi.org/10.1080/ multiple individual /group perspectives 15710880701875068 [16] Nina Simon. 2010. The Participatory Museum (1 ed.). Museum 2.0, Santa Cruz, CA, USA. 5 SUMMARY [17] Markus Weinmann, Christoph Schneider, and Jan vom Brocke. 2016. Digital SPICE aims for a number of very challenging goals: a) to support Nudging. Business & Information Systems Engineering 58, 6 (Oct. 2016), 433–436. https://doi.org/10.1007/s12599-016-0453-1 visitors in interpreting cultural objects for themselves; b) to sup- [18] Dov Winer. 2014. Review of Ontology Based Storytelling Devices. In Language, port both museums and visitors in exploring and reflecting on the Culture, Computation. Computing of the Humanities, Law, and Narratives. Springer Berlin Heidelberg, 394–405. https://doi.org/10.1007/978-3-642-45324-3_12 range of accumulated contributions; and c) to develop the tools and methods that can support a wide range of voices being heard, including from minority groups. These challenging goals require a sophisticated infrastructure, as presented. However, eventually everything boils down to the point of interaction - the user in- terface. We claim that the project success heavily depends upon interaction design, the design of the user interface and the content itself. Hence we present our initial ideas about ways to address the above-mentioned challenges. ACKNOWLEDGMENTS This project has received funding from the European Union’s Hori- zon 2020 research and innovation program under grant agreement No 870811. REFERENCES [1] Fiona Cameron. 2008. Safe Places for Unsafe Ideas?: History and Science Muse- ums, Hot Topics and Moral Predicaments. Social History in Museums 32 (2008), 5–16. [2] Rossana Damiano, Vincenzo Lombardo, and Antonio Lieto. 2015. Visual Metaphors for Semantic Cultural Heritage. In Proceedings of the 2015 7th In- ternational Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN) (INTETAIN ’15). IEEE Computer Society, USA, 100––109. [3] Rossana Damiano, Vincenzo Lombardo, Antonio Lieto, and Davide Borra. 2016. Exploring Cultural Heritage Repositories with Creative Intelligence. The Labyrinth 3D System. Entertainment Computing 16 (July 2016), 41–52. https://doi.org/10.1016/j.entcom.2016.05.002 [4] Guillaume Dumas, Mario Chavez, Jacqueline Nadel, and Jacques Martinerie. 2012. Anatomical Connectivity Influences both Intra- and Inter-Brain Synchronizations. PLoS ONE 7, 5 (May 2012), e36414. https://doi.org/10.1371/journal.pone.0036414 [5] Guillaume Dumas, Jacqueline Nadel, Robert Soussignan, Jacques Martinerie, and Line Garnero. 2010. Inter-Brain Synchronization during Social Interaction. PLoS ONE 5, 8 (Aug. 2010), e12166. https://doi.org/10.1371/journal.pone.0012166 [6] Aldo Gangemi, Mehwish Alam, and Valentina Presutti. 2018. Amnestic Forgery: An Ontology of Conceptual Metaphors. arXiv preprint arXiv:1805.12115 (2018). [7] Amanda Hill, Mark Kretzschmar, David Morton, and Sara Raffel. 2018. ”Eenie Meenie Miney Mose”: Using Experimental Citizen Curating To Engage Visitors With Racial Ephemera. Florida Studies Review 62 (2018). [8] Michel Klein, Nataliya Mogles, and Arlette van Wissen. 2014. Intelligent Mobile Support for Therapy Adherence and Behavior Change. Journal of Biomedical Informatics 51, C (Oct. 2014), 137–151. https://doi.org/10.1016/j.jbi.2014.05.005 [9] Barry J Mauer. 2017. The Citizen Curating Project Confronts the Pulse Nightclub Shooting. The St. John’s University Humanities Review (2017). [10] Heather Moqtaderi. 2019. Citizen Curators: Crowdsourcing to Bridge the Aca- demic/Public Divide. University Museums and Collections Journal 11 (2019), 204–210. Issue 2.