Digital Libraries containing metadata of diverse cultural heritage objects are meant to be accessible not only to domain experts but also to the general population. This calls for information services that can provide ease and efficiency to search, retrieval and exploration. Knowledge graphs (KGs) are essential for representation, organization, integration, and analysis of hierarchical and heterogeneous information. However, most KGs suffer from incompleteness and inaccuracies. This work intends to address various challenges arising from construction and refinement of a KG populated with historical objects, by defining domain-and application-appropriate ontologies and leveraging approaches in information extraction (IE) for improving metadata quality.
The German Digital Library 1 (DDB) collects, aggregates, transforms, and publishes metadata representing tens of millions of digitized cultural heritage objects (eg. books, paintings, archival documents, photographs, audio recordings). These objects span several millennia and belong to the holdings of various memory institutions all across Germany. Due to its historical significance, this collection is meant to be accessed and explored by users from diverse backgrounds.
However, the sheer volume, granularity, and heterogeneity of this collection hampers the ease in search, retrieval, and exploration. These hurdles call for the construction of a knowledge graph (KG) to represent and to organize the objects and their contextual descriptions, while enabling data integration and analytics.
As the national aggregator to the Europeana [1], DDB's metadata collection is represented using an extension of the Europeana Data Model (EDM) 2 . EDM favors simplicity and offers flexibility in the choice of metadata element sets, as well as the range of possible values for properties describing the objects. These design considerations lead to modeling challenges described by Tan et al. [2]. In addition, the metadata collection suffers from incompleteness and inaccuracies as described in Tan et al. [3]. This prevents the underlying retrieval engine from properly indexing the objects.
To address these challenges necessitates a combination of solutions in knowledge representation, knowledge refinement, and information extraction. Therefore, this thesis proposes i) an ontology that enables interoperability across different types of CHOs while maintaining domain-specific semantics as discussed in Section 5.1; ii) a KG refinement approach leveraging NLP teachniques to improve metadata quality of historical objects; and iii) an Entity Linking approach for entities in historical objects.
This work will benefit not only the general population, but also the domain experts such as librarians, curators, and archivists. Proposed solutions will empower users from diverse backgrounds to seamlessly and efficiently search, retrieve, and explore Germany's rich and voluminous collection.
Recent developments in AI can be leveraged to address the technical challenges facing the DDB. This work is relevant to the researchers working at the intersection of Semantic Web (SW), Digital Humanities (DH), and Natural Language Processing (NLP).
There have been several notable data models or ontologies proposed for cultural heritage representation. Liu et al. [4] provided a review of CIDOC-CRM, Sampo Model, and EDM specific to the museum use case only. Cultural heritage data models are delineated along two modeling paradigms: object-centric and event-centric. CIDOC-CRM follows the former, while EDM follows a mixture of both paradigms. Object-centric modeling defines attributes directly by describing the object, while event-centric modeling defines these attributes through a series of events associated with the object. Object-centric modeling favors conciseness, while event-centric modeling emphasizes completeness.
A pioneer in the application of of SW technologies, the Sampo series of semantic portals showcase the national heritage of Finland. These systems make use of the modular FinnONTO ontology infrastructure [5]. However, FinnONTO is not a full-featured ontology, but a taxonomy of CHOs encoded as Simple Knowledge Organization System (SKOS) concepts. Following the modular modeling approach is Italy's ArCO3 [6], where each module is intended to describe a CHO 4 in the context of cataloging activities and events.
The core design principles of EDM, and by extension DDB-EDM, lead to definitions of general classes that require the bare minimum of metadata properties and controlled vocabularies. Thus, all CHOs, regardless of their sources, media types and object types, are instances of the class edm:ProvidedCHO, while their digitized representations on the Web are instances of the class edm:WebResource. This flexibility however results in imprecise representations and loss of semantics inherent in the original objects [7]. In particular, it is not possible to model the concepts and level of abstractions widely-accepted in the bibliographical domain.
The International Federation of Library Associations and Institutions (IFLA) developed the Functional Requirements for Bibliographic Records (FRBR) [8], where a book can be represented as several entities and the relationships that exist among these entities. A copy of a book (frbr:Item) is a specimen or exemplification of a specific publication (frbr:Manifestation), which is an embodiment of an expression frbr:Expression that realizes the ideas of a creative work (frbr:Work).
Most Europeana users are less likely to search for specific items (11.3%) and are more inclined to search by category (47.1%) and by subject (24.6%) [9]. This supports the need to align bibliographic objects from the Item level to their respective higher-level abstractions (Work, Expression, Manifestation). Consequently, the process of alignment sets a prerequisite for objects to possess identifiable properties and attributes, such as title, agents, dates, and subject heading. However, due to the age of the objects, a high level of uncertainty with respect to proper author or date attributions is apparent.
The challenges of filling missing information and identifying erroneous information in a knowledge graph fall under the umbrella of Knowledge Graph Refinement. In particular, Knowledge Graph Completion (KGC) deals with the former challenge, while Error Detection deals with the latter.
By definition, internal methods for KGC use the content of the current KG either to determine class membership or to predict relations between entities. These methods require the current KG to at least possess reasonable quality in order for large scale evaluation to be feasible [10]. On the other hand, external methods leverage other sources of knowledge for refinement, such as other knowledge graphs or text corpora.
With the rapid development in the area of Natural Language Processing (NLP), text corpora have become an excellent source of external knowledge. The subfield of Information Extraction (IE), an intermediate step to knowledge graph construction, can be defined as the process of gleaning structured information from unstructured text [11]. A concrete example of this task would be to extract distinct properties and attributes identifying a literary work from the title.
An IE pipeline starts with Named Entity Recognition (NER), or the detection and classification of named entities mentioned in the text. Types of entities can be coarse-grained such as PERSON, WORK_OF_ART, DATE, et cetera or fine-grained such as AUTHOR, PUBLISHER, ARTWORK, PUBLISHER, LITERARY_WORK, PUBLICATION DATE, et cetera.
Specific entity types (fine-grained) are often found in domain-specific texts, or even timespecific texts where concept drift is quite common. In the field of digial humanities, there are a number of studies on NER with historical text [12], however, coverage goes back to 17 th century B.C. at best (DROC [13]) and none belong to the domain of bibliography.
Once the entities have been detected and classified, they are linked to specific entries in reference knowledge bases or KGs. Entity Linking is particularly challenging due to the surface form variations. In particular, names in historical texts can be multilingual, refer to aliases or contain initials, include honorifics and designations. The names of geop-olitical entities are also known to change through time. Pontes et al. [14] proposed an end-to-end multilingual NER and EL (NERL) approach to address some of these challenges using some of the datasets mentioned in Ehrmann et al. [12].
This section formulates the research questions (RQs) to address the challenges and limitations of existing approaches described in Section 3. RQ1: How can existing ontologies be adapted and extended to suit the domain and application profile of digital libraries, such as the DDB? Cultural heritage practitioners have been developing ontologies for specifc domains and applications. As of this writing, only EDM is used to represent metadata from several cultural institutions. In order to prevent data model silos [15] and to promote reusability, it is beneficial to consider existing ontologies that are applicable and appropriate for the use case of the DDB. Preliminary results are discussed in Section 5.1.
Non-contemporary titles in the DDB (<dc:title>) encode details that can be used to fillout missing properties, such as the title itself, author, publisher, editor, subject headings, and dates. Hence, this calls for extractive NLP approaches. Section 5.2 presents some preliminary results.
In order to address the succeeding RQs, an evaluation dataset for IE is required. Section 5.2 briefly describes what has been done so far.
The goal here is to address open challenges in the area of historical NER, such as how to properly handle the dynamics of an evolving language, where spelling and naming conventions change through time, and noise resulting from OCR engine. Dataset construction, design of experiments, and model development will be accomplished.
The goal here is to accurately disambiguate named entities and link them to entries in external KGs, while addressing the challenges associated with historical texts. Moreover, entities that do not exist in the reference KG can used as further contribution to increase the coverage of authority files.
The entirety of this work is envisioned to guide the construction and refinement of a knowledge graph representing DDB's cultural heritage objects. In addition, some open questions have yet to be addressed concerning NERL in historical texts.
The section describes preliminary work conducted to address the open questions presented in Section 4.
Extensive quantitative and qualitative analysis of the entire DDB metadata collection have been conducted in order to ascertain the applicability of existing CH ontologies. Initially, objects were logically classified according to their originating institution, whether from libraries, archives, museums, media libraries, or historical preservation. In addition, the media type of an object was also taken into account. Taking up a large proportion of the entire collection, the alignment of textual bibliographic resources to an extension of FRBR5 have been presented [2] and implemented as a SPARQL Endpoint [16]. Domain-specific ontologies have been adapted to have more precise semantic representation objects (eg. components of bibliographic objects, hierarchy of archival objects, level of representations of an image, etc.) Existing audio ontologies intended for other domains have been extended to represent intangible audio heritage [17]. The DDB-O Namespace6 is available online. A formal and complete specification is under review and yet to be published.
FRBR, as the upper ontology, requires that each object is looked up against a list of creative works, such as the German Authority File or Gemeinsame Normdatei (GND 7 ). This ensures that the relationship between different objects resulting from the same creative work is represented in the KG [18].
The alignment of bibliographic items to their corresponding literary works proved to be a challenging task due to incomplete object descriptions [18]. Taking advantage of the greater textual content encoded in the titles, several NLP tasks were reformulated in order to extract contextual details present in the title. Several state-of-the-art, off-the-shelf NER and extractive QA models, as well as LLMs were used in the experiments.
As described in [3], the objects in the evaluation dataset were selected according to language, hierarchy type, existence of agent and date properties, format, and title length (>30 tokens).
A more forgiving evaluation measure (Precision@n) described in Section 6.2 was defined to take into account the various naming conventions found in the text. An NER model (FLERT) [19] that can detect literary works and dates was initially used to test the hypothesis, and to refine the evaluation dataset for the succeeding tasks. The results shown in Table 1 illustrate that these models can be leveraged but only to a lesser extent. The results were poor since the models were not adapted to the age and domain of the texts. In addition, the results are not indicative of the actual model performance due to evaluation dataset inaccuracies [3].
The research questions enumerated in Section 4 require different evaluation procedures, dataset, and metrics. These are described in the succeeding subsections.
There several ways in evaluating ontologies. One of which is using is using competency questions (CQs). A collection of CQs published in GitHub8 are included in the partial ontological definitions and alignment activities. In addition, SPARQL query processing time for CQs that can be answered with DDB-EDM will be compared with queries using the proposed ontology.
Name matching for historical documents is non-trivial due to various naming conventions and spelling variations. In a QA task, the most forgiving measure is Accuracy@1, which returns 1 if there is a single token overlap between the ground truth and the answer. Precision@n measure is a combination of 2 matching criteria: an exact match of the DDB object ID and an approximate match for names using the Levenshtein edit distance [3]. The evaluation measures for RQ3 will not be any different from those associated with EL. A large proportion of the agents in the DDB are already linked to GND Persons. And there already exist links between GND and Wikidata entities. This means that it is trivial to combine naming variations and multilingual names for the evaluation dataset. Evaluating geopolitical entities will require prior knowledge of the age of the object in question.
As discussed in Section 5.2, the lack of a gold standard evaluation dataset brings a level of uncertainty to the experimental results. This will be addressed with the creation of a manually annotated dataset with fine-grained entities. Consequently, this dataset will be used to address RQ2.2. In addition, the work conducted to address RQ1 need to be finalized. Finally, entities that already exist in GND will be linked, while non-existing ones can be used to further increase the coverage of GND and Wikidata.
I would like to thank my supervisors Prof. Dr. Harald Sack and Dr. Shufan Jiang for their invaluable mentoring and support.