The paper presents the H2020 CrossCult project, providing a short overview, a summary of the platform developed by the project, a description of the consortium, lessons learnt in three main dimensions (humanities, technology and business), the open challenges and the main tools developed by the project.
CrossCult is an H2020 European research (2016-2019), developing and experimenting with IT technologies that aim to triggering reflection as a result of the user experience. The ultimate goal is to spur a change in the way European citizens explore, reflect and interpret their common History by asking participants to (re-) interpret what they may have learnt, in the light of cross-border connections among historic sources, cultural venues and other citizens’ viewpoints.
The project has two expected outcomes. The first is to lower cultural barriers to enable wider access and participation in cultural heritage experiences and to create cross-border perspectives, by connecting existing digital historical resources, physical objects or places and by creating new contributions through the participation of the public. The second outcome is to provide long-lasting experiences of social learning and entertainment that will help towards an improved understanding and reinterpretation of European (hi)story.
Technologically speaking, CrossCult offers a flexible services platform together with an ecosystem of mobile applications, dedicated to cultural heritage sites and their users, and designed especially to trigger reflection on cultural/historical topics, objects and places. This has been tested in is four pilots having different characteristics and is being extended to new use-cases progressively with stakeholders approaching the project through its Living Lab (https://www.crosscult.eu/en/living-lab/definitionimplementation/).
From an impact perspective, CrossCult aims to provide an affordable ecosystem that will permit museums, cultural institutions and cities of all sizes, to develop a sustainable strategy in interpretation and interaction with visitors and their collection or with cultural heritage and history of their city. An open source philosophy drives the core of the business model, supported with a living lab concept for stakeholders and potential customers.
The research group behind the CrossCult project forms an heterogeneous consortium of 11 partners from 7 European countries (France, Greece, Italy, Luxembourg, Malta, Spain, UK), consisting of computer and humanity scientists, with expertise in personalisation, semantic web and knowledge modelling, crowd computing, gaming, archeology, social sciences, locative media, educational sciences, psychology, geography, museology, etc., completed by a company with deep knowledge of business in the areas of cultural heritage, museums, galleries and cities. 2
Nothing in History occurs in a vacuum, everything has to be understood in a wider context. The European history, highly interconnected by nature, is a good example. Far from a collection of unconnected events, European history is a huge mesh of interconnected narratives that are interpreted and built from different evidence and material, which has long been understood, taught and discussed in academic settings. Due to the intensive and complex nature of historical research it is commonly conducted through a close deep reading of selected historical resources but recently there is a need to provide tools that facilitate distant reading across wider geographical scales.
CrossCult sets out to develop mobile experiences that foster interpretations of cross-border interconnections via reflection upon pieces of cultural heritage, other citizens' viewpoints and physical venues and places. It seeks to increase retention, stimulate reflection and help citizens appreciate their common past and present, presented in a more holistic manner. Within CrossCult, new user-friendly tools have been developed to help researchers, experience designers, museum experts and external stakeholders create rich interconnected information presentations. The generated information is stored in flexible, semantic, data stores and presented via customised mobile apps.
Crosscult is using 4 flagship pilots across Europe (London, Lugo, Chaves, Epidaurus, Montegrotto Terme, Tripolis, Luxembourg City, Valetta) to present participants with different types of experiences in order to look at past and present societies with a critical mind, evaluate major events and characters on the grounds of economic, political, cultural or environmental realities.
Pilot 1- Large multi-thematic venue The broad collection of the National Gallery, London (UK), is used to illustrate the connections among people, places and events across European history.
Pilot 2 - Many small venues Spanish, Portuguese, Italian and Greek small venues, respectively the Roman healing spas of Lugo and Chaves, the Archaeological site of Montegrotto Terme and the ancient sanctuary of Epidaurus are connected in the pilot. It highlights the inherently cross-border nature of History by engaging people of multiple nationalities in the discovery of connections between their respective bodies of cultural heritage.
Pilot 3 - One venue, non-typical transversal connections In the Archaeological Museum of Tripolis in Greece, visitors go beyond the typical level of history presentation (e.g. type of a statue, or its construction date), into deeper levels of reflection, over social aspects of life in antiquity, power structures, etc.
Pilot 4 - Multiple cities, “Past & Present” interplay Outdoors in Luxembourg and Malta, more precisely in Luxembourg City and Valetta, this pilot challenges the visitors’ current perceptions on migration as a contemporary emotive topic and engages people in exploring the past to understand the present.
The interactive pilot experiences and the narratives they present are designed to trigger reflection around four major principles: 1.Raise consciousness about different historical topics. 2.Tackle the study of History from a multi-faceted perspective. 3.Approach History not only through the written texts from successive eras, but also through all the traces left by those societies (archaeological remains, iconography, epigraphy, numismatics, architecture, art, etc.). 4.Reckon that there are no absolute truths in History interpretations, but various possible interpretations of the archaeological remains and contrasting viewpoints. 3
The project has created an open platform for web and mobile applications backed by robust standards lead storage and data processing systems to deliver new experiences in line with the desire of implementing citizen-centred approaches to stimulate knowledge integration, reflection and dissemination of Cultural Heritage. The platform builds upon two main pillars: • The Semantic Web technologies, which, combined with the numerous and growing Linked Data resources, offer a means for semantically describing and interrelating pieces of evidence (facts), historical events and the associated cultural heritage resources. • The availability of multi-platform software libraries to render interactive, databased visualizations of such descriptions and associations.
The CrossCult platform is a complex ensemble of software aimed to provide services to different types of stakeholders, including museum curators and experts, data scientists, cultural app developers and system administrators (through different webbased front-ends) as well as current and future museum visitors (through Android or iOS apps). The operation of the web-based and mobile frontends is supported by a backend that provides infrastructure and instrumentation for hosting the core components (see Figure 1 below).
There is also a Front End for administrators such as curators, data scientists, museum experts, etc. namely The Services Portal, that gives access to services to configure the behavior at the Apps, and also common features to personalize and monitor every single project made with CrossCult.
At the core of the platform, the CrossCult Knowledge Base is a repository for storage, management and retrieval of semantic information. It implements a single and generic upper-level structure that acts as a semantic layer of common concepts and relationships, building on standard Semantic Web technologies to facilitate interoperability with Linked Data resources. These include CIDOC-CRM (the Conceptual Reference Model of the International Council of Museums and the International Committee for Documentation) (www.cidoc-crm.org), which is the international standard (ISO 21127:2006) for modeling cultural heritage information,and generalpurpose resources such as SKOS (Simple Knowledge Organization System) (www.w3.org/2004/02/skos) and the FOAF (Friend Of a Friend) ontology (http://xmlns.com/foaf/spec/).
On top of the CrossCult Knowledge Base, a number of software modules provide high-level, application-oriented services covering six major functional areas, namely “Association discovery”, “User profiling”, “Recommendation”, “Context awareness”, “Social interaction” and “Visualization”. Each functional area is covered by one or more technological modules, which offer distinct services within the area (e.g. chatting and micro-blogging) or address different facets of a single issue in a complementary fashion (e.g. carousel-based profiling vs interaction-based profiling, item recommendation vs path recommendation, etc. CrossCult is an interdisciplinary project that brings together Computer Science, social scientists and Humanities scholars.
Luxembourg Institute of Science and Technology: LIST acts as coordinator of the CrossCult project, participating mainly with the ADAPT (Knowledge-based and context-aware adaptive systems) research group, which focuses on IT systems adaptation to their user and context of use, in virtual-physical environments. The team brings expertise and know-how in recommendation and personalization, semantic web technologies, indoor localisation technologies, and personalised crowd systems.
University of Peloponnese: UoP participates with the Human-Computer Interaction and Virtual Reality lab and the Knowledge and Uncertainty Research Laboratory (ΓΑΒ LAB), two academic laboratories that work in cultural informatics and bring additional expertise in HCI, human factors, augmented and virtual reality, profiling tools and profiling games, semantics, smart spaces, social media analytics and the handling of uncertainty.
University of Malta: UoM participates with the Institute of Digital Games, an institute dedicated to game design and development with a strong focus on Game AI research. The Institute of Digital Games has participated in several EU projects on game-based learning (ILEARN-RW), learning analytics (ENVISAGE), games for change (C2Learn, eCrisis) and others. UoM researchers participating in CrossCult have expertise in artificial intelligence, computational creativity, affective computing, player experience and game design.
University College London: UCL participates with the Department of Information Studies and the UCL Centre for Digital Humanities (UCLDH). It brings expertise and know-how in knowledge representation, Semantic Web technologies, and Cultural Heritage applications, as well as broad experience of research at the intersection of digital technologies and the humanities.
University of Luxembourg: University of Luxembourg brings experience in the field of digital geography, locative media and humanities research with particular expertise in the user design, development and evaluation of geohumanities applications as well as a broad experience of fostering interdisciplinary research that builds bridges with social science and computer science.
University of West Attica: The Department of Archives, Library and Information Studies brings its expertise in handling, managing and reusing information and content produced in different formats (i.e. image, video, text) and in different means (online, printed); in creating, mapping and handling metadata by using specific and international metadata standards (i.e. Dublin core, VRA, METS, MODS) and ontologies (i.e. CIDOC, RDF) in the cultural heritage sector among others.
National Gallery: The National Gallery, London, houses the national collection of paintings in the Western European tradition from the 13th to the 19th centuries, providing a perfect venue for one of the project's four pilots. The Gallery's aim is to care for the collection, to enhance it for future generations, and to study it, while encouraging access to the pictures for the education and enjoyment of the widest possible public now and in the future. Staff working on the Crosscult project bring expertise in the collection and its visitors, along with relevant technical expertise and experience with the storage, semantic description and presentation of cultural heritage information.
University of Vigo: UVIGO participates with a mixed Humanities-IT team. The Department of History, Art and Geography brings two archaeologists with expertise in Classical Antiquity and water, which are key topics of CrossCult pilots 2 and 3. The Department of Telematics Engineering brings a team of computer scientists with experience in semantics and personalization technologies.
GVAM: Is a private Spanish company specialized in guiding visitors (1,5 million per year) with mobile devices and interactive Apps.
University of Padova: The University of Padova partecipes in the CROSSCULT project through the Department of Cultural Heritage, involved with multiple researches dedicated to analysis and valorisation of archaeological sites and the cultural heritage. In the project, UNIPD introduces the ancient Roman spa of Montegrotto Terme which is used as a venue for Pilot 2. The UNIPD staff also offer their expertise in specifying the pilot requirements and evaluating the pilot results from a social sciences and humanities point-of-view.
Centre national de la recherche scientifique: The team participating to the CROSSCULT project is a computer science team. CNRS brings its expertise in: (i) knowledge discovery for extracting useful and reusable patterns from data, (ii) knowledge engineering for representing the extracted patterns as knowledge units to be reused in problem solving and reasoning. CNRS develops several platforms related to data mining and recommendation and its members work on various application domains including biology, chemistry, medicine, e-learning, e commerce, online music services and museology. 5
In Humanities: • The concept of reflection means many different things and there is a lack of consensus in the literature about its meaning and definition. Therefore for the purpose of this research we consider the reflection to be a process that triggers ever increasing deep thought. Thus, we have developed a multifaceted toolset for analyzing reflection involving user logs, user generated content, interviews and questionnaires. The result a more effective understanding of the mobile experiences and how it triggers thought processes. • We have identified a need to consider methodologies that involve the expression of user opinions and discussions both with the apps and via social media, regarding cultural discussions. People express their feelings and opinions before, during and after the cultural experience on social media and this could be a valuable source of data. • Emojis were found to be quick ways to record responses and emotions of visitors. • Personalisation is only cost-effective when we do not adapt content but only make different suggestions based on individuals profiles. For the purposes of cultural heritage, investing in creating alternative versions of content for different visitors could be avoided due to the increased resources required and instead we suggest the creation of different paths and thematic tours that can cover the diverse needs of different visitors. • Storytelling has been found to be an effective means to foster reflection and increase retention in technology-enabled cultural heritage experiences. The most effective storytelling strategy is heavily-dependant on the scenarios targeted by each application, given the nature of the contents to explore, the type of users expected, whether they will be visiting a specific venue or site and for how long, etc.
Three basic ingredients for storytelling have been identified and tested in the project, from national as well as cross-border perspectives: (i) stories driven by common features, e.g. buildings that share architectural elements, historical events with overlapping chronologies, characters sharing some traits or affected by the same condition, paintings created with pigments obtained by the same technique, etc. (ii) stories driven by universal themes, e.g. to bring several art pieces or locations together because of their relation to any aspects of rituals, folklore, war, sexuality, religion, women’s role in society, migration, … and (iii) phenomena of situational curiosity and serendipity, bringing about connections of heritage items and their interpretations to special dates that may have some general meaning or appeal to some specific users, as well as connections to trending topics.
In Technology: • Museums around Europe seem to be finally agreeing on how to organize the information about their collections. CrossCult has adopted the CIDOC-CRM standard to create the CrossCult Knowledge Base (CCKB) as a pan-European repository of semantic knowledge linked to cultural heritage resources, containing not only the information kept in traditional catalogues (title, author, year, technique, …) but also topics (from universal and ad-hoc vocabularies), stories to be told in relation to each item, experts’ and citizens’ opinions and viewpoints, and links to external Linked Data resources. • On the foundations provided by the CCKB, the project has merely started to scratch the surface of what could become a whole area of research in Digital Humanities in the future: Association Discovery. The goal is to systematize the processing of formal and informal sources of knowledge in order to identify connections among multiple heritage items, that could become the seed for interesting and engaging storytelling, along the above-mentioned guidelines of common features, universal themes, situational curiosity and serendipity. Pathfinding in semantic networks and word/document embeddings created by neural networks are two of the approaches explored in the project for association discovery, with initial but very promising results. • Mobile technologies are advanced enough to create cultural heritage experiences based on rich interactive visualizations of stories, multimedia contents, geographical and historical data, associations among heritage items, etc. • Both experts and citizens are called to contribute to the growth of the CCKB to fulfill its impact. To that aim, both must be provided with convenient tools: on the one hand, paying proper attention to the long-standing workflows of the Humanities area; on the other, empowering technologies of crowdsourcing and crowd computing in a way that really motivates people to contribute.
In Business:
Time scope of EU projects is not enough to enroll external stakeholders to test and assess the development process. It is necessary either to extent the project period or to invite stakeholders earlier in the development process.
A concrete to generic business policy is more efficient in terms of awareness, resources, technological assessment. A sector´s approach, products or demonstrators can be used to design a multiple and versatile marketing tactics and plans. The process may start with an assets business value matrix that may point out the most viable business models. Defining an initial economical value, with or without much information is necessary to make the wheel start and set out a starting point that will lead to the constitution of a legal entity that takes care of the future sustain and marketing of CrossCult.
Today, CrossCult provides a flexible software platform as well as an ontology and a vocabulary dedicated to cultural heritage that can be exploited in various sites and context. By design, it was built to enrich the user experience focusing on reflection triggering, through knowledge discovery, gaming and personalisation applied on non classical contexts where sites, objects and content are interlinked.
From a scientific perspective, the four pilots have shown that there were indicators of reflection and learning, in the different contexts and with the different applications they implement. However, reflection is a concept that remains difficult to define and which requires focused studies. In addition, measuring visitor reflection remains an open challenge, since longitudinal studies might be necessary.In this light, observing what happens after a visit or a session with the Apps, for example: on social networks and on other sites is helpful. Additionally, reflection in cultural heritage, is also a notion deeply linked to the emotion triggered by the artefacts and places, which needs further investigation. The challenges open by our research are both from a social sciences/humanities and computer science perspective, but the big question is probably to find which is the best combination of technologies to use for each context, that will stimulate reflection to a maximum.
From a business and technical perspective, the main challenge is to make the CrossCult Platform adopted by a community of actors in the CH. This passes through the application to new use-cases and the extension of the platform to integrate new services and technologies, without forgetting a suitable business model allowing this. 7
CrossCult follows an open data and software policy. After the completion of the project a large number of data, tools and methods will be open for use. The final licencing scheme for the CrossCult Platform and its components will be defined by the end of the project (February 2019); until then, interested parties can become member of our Living Lab and test components under specific condition of use for collaboration during the project, by contacting the consortium (contact@crosscult.eu).
IT service platform dedicated to Cultural Heritage: The CrossCult platform offers a unique, flexible and extensible platform providing IT services for cultural heritage sites. Precisely, it offers services for personalizing the user experience, discovering links hidden behind cultural artefacts, historical facts or venues, crowd-sourcing, social network connections, tracks and interaction logs analysis, mobile application customization, etc. Among the numerous services offered (see project website or deliverables), we can highlight the following ones.
experiences in a museum or serendipitous discovery in the city. The system allows contributions both from experts but also from the wider public whilst all of the apps are used to trigger reflection on history and draw upon people’s own experiences prior to their visit. In particular, pilot 1 uses the broad collection of the National Gallery (London, UK) to illustrate the connections among people, places and events across European history. Pilot 2 focuses on 4 venues: the Roman healing spas of Lugo (Spain), Chaves (Portugal) and Montegrotto Terme (Italy) and the ancient site and theatre of Epidaurus (Greece). It highlights the inherently cross-border nature of History by engaging people of multiple cultural backgrounds in the discovery of connections between their respective bodies of cultural heritage. Pilot 3 takes place at the Archaeological Museum of Tripolis (Greece) and will allow visitors to go beyond the typical level of history presentation (e.g. type of a statue, construction date, etc), into deeper levels of reflection over social aspects of life in antiquity, like power structures, educational and religious practices, etc. Finally, pilot 4 is outdoors in Luxembourg city (Luxembourg) and Valletta (Malta) and challenges the visitors’ current perceptions on migration as a contemporary emotive topic, engaging people in exploring the past to consider the present situation.
Recommender systems: Different kinds of Recommender Systems (RS) are proposed, allowing personalised recommendations of respectively persons, items, paths and knowledge associations. • The Person RS allows creating sporadic social networks, dealing with the creation or update of groups of participants in app experiences according to the interests and traits they may have in common. • The Item RS recommends items (exhibits; POIs; etc.) individual users might like based on their profile. • The Path RS recommends to users a path/route in the museum following a sequence of items selected from a set of items. Sequences and paths are dynamically built according to some constraints like the duration of a visit and movements of the user. • The Association RS matches the outputs from the consolidated association discovery processes with the information stored in a user’s profile, in order to filter which of the many potential associations among cultural heritage items, reflective narratives, locations, people and dates could be most relevant according to the user’s interests.
Profilers: CrossCult has developed different profilers like image-based ones and game-based. The carousel (image-based profiler) allows to quickly gather interests from users, avoiding classical cold-start effects. For game-based profiling, dedicated games have been developed that create user profiles based on the users’ personality traits, while the user is playing.
Association Discovery: This is an intra and inter-venue associations discovery tool that aims to automate the discovery of associations among multiple venues, collections, items, etc. on the ground of common features or relation to relevant dates. A range of AI mechanisms are applied such as semantic reasoning and deep learning.
The CrossCult Knowledge Base: The Crosscult Knowledge Base (CCKB) is a comprehensive structure of semantic definitions and formalisms, developed for facilitating interoperable connections between the cultural heritage datasets contributing to Crosscult. It is written in OWL2 (https://www.w3.org/TR/owl2-overview/), the standard ontology language for the Semantic Web, and enables augmentation, semantic linking, semantic-based reasoning and retrieval across disparate data resources.
The Living Lab: The Living Lab is not a tool per se, in the sense that it is not a library that can be ported into a new project independently from CrossCult. It is a setting, in which external entities, be they projects, businesses, institutions, research groups or independent researchers, can interact with the CrossCult consortium, experiment with the available software, familiarize themselves with the technologies and try out new solutions. From this of view, we see the Living Lab as a tool for experimentation and development of new tools and services for the cultural sector. 8
This work has been supported by CrossCult that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no 693150.