The moral values present in classical texts are only implicitly codified and are a form of intangible cultural heritage. In this paper, we focus on capturing the perceptions of these references of values, with the aim to study how they evolve over space and time. To this end, we present our approach that consists of a meta-model and two intertwined design processes for the creation of a knowledge graph capable of capturing an integrated representation of both values and perceptions. In particular, we illustrate how the input collected from diferent activities with the general public can feed the graph to represent multiple and possibly divergent perceptions of values in classical texts. Our meta-model allows for the integration of data from both established scientific techniques such as expert annotations on the one hand, and on the other from standard tests, social media activity, visual representations and games. The proposed approach gives practical means to make explicit both historical and current perceptions of values in classical works of art. Our approach and the resulting knowledge graph enable the comparative analysis of values and their perceptions over space and time.
The moral values present in classical texts are an important part of our intangible cultural
heritage, and in particular when studying a common European identity. Yet, being only implicitly
codified in artifacts, in diferent regions and diferent time periods (what we refer to as “over
space and time”) the perceptions of these values diverge [
As a way to moral values, we take as a reference the definition of Schwartz’s basic human
values, defined as “trans-situational goals, varying in importance, that serve as guiding principles
in the life of a person or group.” His refined theory of values [
The knowledge graph proposed here features four major aspects of user interactions with values present in a historical artifact: i) the participant’s individual characteristics such as demographics or expressed beliefs (who), ii) the description of the artifact and experience that define the interaction ( how), iii) the collection of the expressed perceived values (what), and iv) the spatio-temporal context of the interaction (where and when).
Next in this paper we review previous research, both in terms of representation of relevant knowledge, and collaborative knowledge graph creation. In Section 3.1 we present our conceptual model and knowledge graph engineering process. This is followed by examples of concrete knowledge externalisation activities and details about our technical implementation. We conclude with
The two relevant fields of research are works about formalisms and models for representation of
knowledge in the humanities, as well as collaborative methodologies for knowledge graph
engineering. An important role in the crossroads of the two is the already mentioned ValueNet [
When looking for ways to represent historical interpretations, another natural candidate could
be the CIDOC Conceptual Reference Model (CRM). It features elements to allow for the
representation, integration, mediation, and interchange of documents and scientific activities [
Beyond the aforementioned limitations, for the formal representation of values, we note that
CIDOC-CRM supports the appellation relationship that can be employed for a simplistic to
representation between concepts (as used in VAST) and their corresponding terms. The authors
of the PhiloSurfical model [
When multiple contributors are involved in the specification of an knowledge graph, a clearly
deifned process becomes a necessity. The NeON methodology framework to collaborative ontology
engineering provides variations of a process to be used in diferent circumstances, depending
on planned project duration or availability of pre-existing knowledge resources [
As an alternative, the UPON Lite framework consists of 6 steps identified by their objectives: terminology, glossary, then taxonomy, predication and parthood performed in parallel and, ifnally, ontology [ 17]. These steps can be considered as a detailed expansion of the design and implementation phases in NeON.
The idea of involving non-expert users in ontology engineering has been discussed in the literature by proposing to employ crowdsourcing-based solutions. Daga et al [18] consider how citizen are contributing to cultural heritage archives. However, none of the considered examples are addressing the need to organise diverse knowledge in a single knowledge graph. Highly structured software for ontology construction and crowdsourcing from social networks has been used as a validation mechanism [19]. Games with a purpose (GWAP) have been also proposed as a common solution to employ crowdsourcing contributions for the creation of knowledge graphs. These have been categorised into four types: specification of term relatedness, verification of relationship correctness, specification of relation type, and verification of domain relevance [ 20]. An example of specification of term relatedness is Free Association [ 21], in which given a word, the user is asked to find others that are related. A recent example for relationship verification is Indomilando [22] where players are asked to choose between diferent predefined relationships in the form of multiple-choice questions) as a way to verify that an undisclosed one of them is correct. A typical model for specification of relationship type is a game called SpotTheLink [ 23] where players are given two concepts and asked to identify a relationship that could connect them.
We describe our participatory knowledge graph design providing two overarching perspectives, one static describing the meta-model and one dynamic describing the process of knowledge externalisation.
Starting from the tangible artifacts and the users interacting and interpreting them, we develop a representation of the activities and the elicited values. This is reflected in the four principle domains of Figure 1 and discussed below.
Artifacts. The knowledge graph that we present here features a unified representation of tangible resources, from historical documents and pieces of art to questionnaires and activity recordings. A common feature of the representation of all these is their association to a specific spatio-temporal context. Whereas to the knowledge graph artifacts are simply stored objects, some of them might also have a role in the knowledge graph creation as structural representations of knowledge, including annotations, questions and answers or digital games provoking particular behaviours, as elaborated in Section 4. Correspondingly, to be able to facilitate a focused reflection or discussion of a particular part of an artifact, the knowledge graph features also a representation of a segment of an artifact, be it text, video or a more structured digital tool.
Group metadata - composition (number) - nationality composition -- aggeendceormcpoomspitoiosnition -- cmuolttuhrearltpornegfeureenccoemsposition - education composition - school data (optional) (0,N) (1,1) User (0,N)
(1,N) Group (0,N) (0,N) (0,N) (0,N) (1,1) (0,N) Activity (0,1) (0,1) (0,N) Event
(0,N) Context Terms VAST keywords Audio Video Image Tools Document (0,1) Statement
(0,N) (1,1) (0,N) ((01,,N1)) C(0o,nNc)ept (0,N)
Relation (0,N) Profile (0,N)
(0,N) Schwartz value (0,N) (1,1) (0,N)
Artifact (1,1) (0,N) Blobs (0,N)
Questionnaire Interview (1,N)
MindMap (1,1) (0,N) (1,1) Segments
(1,1) Annotation (1,1)
Expert User metadata - name - age - gender - education - nationality -- cmuolttuhrearltpornegfeureences Non Expert Supportmaterials (0,N)
Step (0,N) (0,N) (0,N)
Stimoulus Activity metadata - date - event - description - nature (online, in person) - tongue ACTIVITY REPRESENTATION Users. To capture their perceptions of values over space and time, our knowledge representation features two types of users as represented in Figure 1, who could contribute diferently: (i) Expert users include humanities researchers that study and interpret the historical intentions behind an artifact, social science researchers that study current perceptions of it and value communicators that reenact its contemporary significance. This separation of responsibility needs not be strict, but the contribution of each user has a clearly defined spatio-temporal context, which for expert users is their domain of expertise, as further explained in the Section 3.2. (ii) Non-expert users are the general audience participants that engage in VAST activities and share their perceptions and views. Based on an experience (activity) related to a historical artifact. This could happen in a museum or theatre visit, in a school activity or online, which defines their corresponding spatio-temporal context.
Activities. Users engage with artifacts in the context of particular activities that also need
to be represented in the knowledge graph. This is not only due to the need to store the time
(technically – date) and space (geolocation) of when and where these take place, but also their
structure. In order to capture the potential influences that might shape the user experience,
its steps (i.e. method) and stimuli (i.e. artifacts used as supplementary materials) are being
represented. This allows the knowledge graph to relate responses to stimuli from diferent steps
within a single activity, even when the related data is collected in anonymised form. Finally, the
context of the activity is represented through the recordings of activity sessions (i.e. events),
involved organisations, and individual and/or group demographics. Providing an example for a
complete activity is out of the scope of this paper. Instead, here we take a data-centric approach,
focusing on representation and collection of data. Accordingly, in Section 4 we provide examples
of diferent types of used artifacts and the corresponding digitisation of concepts.
Concepts. To represent values, the VAST knowledge graph features two main entities:
concepts and terms. Concepts, a core part of which are the Schwartz values, are abstractions that
could feature also other commonly established psychological constructs beyond those studied
by Schwartz. By adopting standard psychological instruments, such as the European Social
Survey [
Our approach to knowledge graph engineering is based on an iterative-incremental design
approach inspired by the NeON methodology [
Vocabulary definition. Starting from EU values, experts from the three pilots define an extended set of terms that they consider relevant to their respective domain, both spatio-temporal and in terms of activities. A single vocabulary of terms and concepts is defined containing the whole set provided by all pilots, so that a term can be used where found appropriate regardless of the activity in which it emerged.
Schema definition. As explained in the previous subsection and shown in Figure 1, a general schema has been defined beforehand. Beyond that, the decisions of how to represent individual artifacts and activities, as well as how to connect these terms are taken by the core team of domain experts representing each of the three pilots.
Schema population. The content generation is reduced to micro-tasks doable within single isolated sessions, in which users are assumed not to have prior awareness and need to be introduced to the entire context. Thus, these sessions (i.e. activities) are much closer to field studies in museums, theatre, schools or particular online settings. They represent a highly structured way of schema population akin to citizen curation [18] and games with a purpose [20] designed by domain experts. Particular examples of activities including such micro-tasks with non-experts have been presented elsewhere [24, 25, 26] and an illustration of the corresponding integration of the data in the knowledge graph follows.
Our approach allows for the integration of a wide range of user contributions into the knowledge graph as a way to formalise contributor knowledge. To illustrate this, here we discuss five distinct examples that illustrate the full range of structured and unstructured contributions. These are text annotation, questionnaire responses, dedicated social network discussions about values, mind map creation and digital game interactions.
Text Annotation. An established content analysis technique is used to annotate values in classical texts with dedicated annotation tool and methodology [27]. In particular, we work on English translations of the considered textual artifacts and users are asked to identify segments in artifacts and tag them with (one or more) appropriate terms from the vocabulary. In contrast to other activities described below, text annotation is a lengthy process that requires longer periods of engagement, but is also accessible to experts. Users are first asked to annotate explicit references to the considered values in the artifact. Then, they are asked to annotate implicit or indirect references to values, according to their subjective interpretation.
Questionnaires. Three models for mapping answers of common question types to the
knowledge graph are supported, namely multiple choice questions, scale questions and open-answer
questions. Multiple choice questions allow for formulation of questions aimed specifically at a
particular set of alternative relationships, be it between terms, concepts, profiles , artifacts, etc.
An example of such a question regarding the relationship between a segment of an artifact and
a term is provided in Figure 2. These questions can typically be formulated to enquire about
relationships proposed by experts and allow probing for their perceived validity across various
user groups. Scale questions allow to examine the strength of relationships, for example using
Likert-scale questions of standardised questionnaires from psychology [
Social Network Activity. Using the specific functionalities of social networks, such as polls, images and replies, questionnaire items can be represented online to reach wider audience, as shown in Figure 2. Posting such questions with specific hashtags or in dedicated groups provides a community context remotely similar to the one of a bespoke activity, albeit with limited demographics data.
Mind maps. Another widespread technique to collect opinions, that is both an intuitive representation of complex knowledge and naturally adapts itself to a knowledge graph, is network-based visualisations such as mind maps, and the similar concept maps and topic maps. Such data is being collected in a range of activities within the VAST project (see Figure 3), both on paper to be curated and digitised by educators, and in a dedicated digital tool that allows data to be directly imported in the knowledge base. Typically in such activities, students are asked to start from a concept or term present in the knowledge graph and – in a group or individually – expand the map with their own views. The nodes that are being introduced in the process are either mapped to terms with coinciding labels, or introduced as new terms [24]. A particularity of this type of activities is that more often than not, relationships are unlabeled. In such cases, we employ a general relatedness (shows-features-of) relationship.
Digital Game Experiences. Yet another very broad category of activities are bespoke digital games that provide highly structured experiences and are designed to generate specific representations of player perceptions based on their choices within the game. Two examples that have been developed within the VAST project and shown in Figure 4 are games where players are asked to role-play the Little Red Riding Hood and The trial of Antigone [25, 26] and value preferences are being derived from the player choices within the games.
The VAST platform is built on top of a GraphDB instance which ensures a RDF-compliant storage and a SparQL querying endpoint. This graph database is used as a data warehouse, synchronised via API endpoints and synchronisation scripts with the various components. These components include a text annotation tool, a survey management tool, and a set of digitisation tools for diferent activities. Beyond the specific bespoke integration with the digital mind maps and games, also a generic digitisation tool which allows for the import of generic, including non-digital activities. The resulting integration, including content produced from the above-mentioned digitisation types, is accessible via the VAST platform3.
In this paper we described an approach to the construction of a knowledge graph of moral values – a particular type of abstract concepts. The necessity to represent both past and present interpretations of the values conveyed in historical artifacts has led us to consider a broad range of contributors. We employ an iterative-incremental process. We also demonstrate a range of mechanisms that engage users to contribute to our knowledge graph.
The produced participatory knowledge graph is made available via the VAST platform. Besides supporting both qualitative and quantitative research (e.g. providing baselines for values mining and natural language processing), this collection of structured datasets can be used for citizeninformed curation, and design of future museum activities.
An important aspect of knowledge graph creation is interoperability with other relevant semantic resources, such as CIDOC-CRM and ValueNet. While not discussed here, this has been envisioned by trying to reuse semantics as much as possible, e.g. relationship types from CIDOC-CRM and shared concept entities with ValueNet. Yet, the actual integration remains to be detailed.
Furthermore, answer matching in the described approaches could be expanded by identifying concepts and terms with their corresponding appellations from established thesauri. Beyond this, answers to open questions could be further analysed for further semantic interpretation, e.g. identifying responses to generic questions who, how [28], where [29], etc.
Although it is beyond the scope of this paper, an important aspect of the knowledge graph design process are the activities that involve contributors. Whether it is questionnaires or mind map compilation done within conventional museum workshops or self-paced activities in social networks or games, enriching text content with multimedia is expected to contribute to engagement and retention. One way to make this more accessible, that has recently emerged, is through the semi-automatic generation of contextual illustrations. While text-to-image generator models are not commonly grounded in pre-existing text, preliminary tests have shown very positive results [30].
⋆⋆⋆⋆⋆⋆⋆⋆⋆⋆⋆⋆ This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101004949. This document reflects only the author’s view and the European Commission is not responsible for any use that may be made of the information it contains. 978-3-642-24794-1_2. [17] A. De Nicola, M. Missikof, A lightweight methodology for rapid ontology engineering,
Communication of the ACM 59 (2016). doi:10.1145/2818359. [18] E. Daga, L. Asprino, R. Damiano, et al., Integrating citizen experiences in cultural heritage archives: Requirements, state of the art, and challenges, J. on Computing and Cultural Heritage 15 (2022). doi:10.1145/3477599. [19] V. Iyer, L. M. Sanagavarapu, Y. Raghu Reddy, A framework for syntactic and semantic quality evaluation of ontologies, in: Secure Knowledge Management In The AI Era, Springer, Cham, 2022, pp. 73–93. doi:10.1007/978-3-030-97532-6_5. [20] G. Wohlgenannt, M. Sabou, F. Hanika, Crowd-based ontology engineering with the uComp
Protégé plugin, Semantic Web 7 (2016). doi:10.3233/SW-150181. [21] D. Vickrey, A. Bronzan, W. Choi, A. Kumar, et al., Online word games for semantic data collection, in: Empirical Methods in NLP, ACL, Honolulu, Hawaii, 2008, pp. 533–542. URL: https://aclanthology.org/D08-1056.pdf. [22] I. Celino, G. Re Calegari, A. Fiano, Refining Linked Data with Games with a Purpose, Data
Intelligence 2 (2020). doi:10.1162/dint_a_00056. [23] K. Siorpaes, M. Hepp, Games with a purpose for the semantic web, IEEE Intelligent
Systems 23 (2008). doi:10.1109/MIS.2008.45. [24] E. Fani, A. Ferrara, C. Gagliardi, et al., Participatory citizen curation in the VAST project framework, in: ExICE - Extended Intelligence for Cultural Engagement, Bologna, Italy, 2023. doi:10.5281/zenodo.7973656. [25] E. Aristodemou, S. Montanelli, M. Ruskov, Fairy tales as a swipe card game: Engaging digital audiences in value-centred discourses using storytelling, in: ExICE - Extended Intelligence for Cultural Engagement, Bologna, Italy, 2023. doi:10.5281/zenodo.7973654. [26] L. Arnellou, M. Dagioglou, D. Katsamori, et al., Digitising intangible cultural heritage: The use case of Antigone, in: ExICE - Extended Intelligence for Cultural Engagement, Bologna, Italy, 2023. doi:10.5281/zenodo.7973658. [27] G. Petasis, M. Ruskov, A. Gradou, et al., The VAST collaborative multimodal annotation platform: Annotating values, in: Information Systems and Technologies WorldCIST, Springer, Pisa, Italy, 2023. Publication pending. [28] M. Ruskov, Who and How: Using sentence-level NLP to evaluate idea completeness, in:
AI in Education, 2023, pp. 284–289. doi:10.1007/978-3-031-36336-8_44. [29] S. Gupta, K. Nishu, Mapping local news coverage: Precise location extraction in textual news content using fine-tuned BERT based language model, in: 4th W’shop on NLP and Computational Social Science, ACL, Online, 2020, pp. 155–162. doi:10.18653/v1/2020. nlpcss-1.17. [30] M. Ruskov, Grimm in Wonderland: Prompt engineering with Midjourney to illustrate fairytales, in: 19th Conf. on Information and Research Science Connecting to Digital and Library Science, 3365, 2023, pp. 180–191. URL: https://ceur-ws.org/Vol-3365/paper6.pdf.