Archival repositories contain vast amounts of historical data within unstructured textual documents, posing significant challenges for extracting coherent insights. This paper presents ongoing work towards an optimised workflow for constructing a knowledge graph from millions of archival records related to the “Wiedergutmachung” process in Germany. These records, documenting compensation and restitution eforts following World War II, ofer insights into the aftermath of the National Socialist regime. The proposed workflow involves converting document images to machine-readable formats, ontology design, information extraction, and entity linking. Leveraging both traditional methods and transformer-based technologies, the workflow addresses unique challenges inherent in historical documents.
Since the 1990s, researchers from various domains have been increasingly engaged with
providing access to information held within archival records, through online channels [
This paper proposes an optimised workflow for constructing a knowledge graph from millions of archival records from the “Wiedergutmachung” 1 process in Germany, for utilisation in the semantic portal “Themenportal Wiedergutmachung” 2. The term “Wiedergutmachung compensation records” in this paper, refers to collections of documents, records, and materials related to the process of compensation and restitution eforts that followed World War II and the fall of the National Socialist regime in Germany. Wiedergutmachung compensation records which contain an estimated amount of 100 km of archival documents, originate from the State Ofices for Compensation ( Ämter für Wiedergutmachung in German) installed by the German government in every German Land after the war. These collections contain a wide range of documents, including index cards, application forms, documents on legal proceedings, correspondence, testimonies, and other materials that pertain to individuals, families, and communities seeking compensation for forced labour, imprisonment, injuries, and other damages caused by National Socialist Injustice.
Wiedergutmachung compensation records serve as a historical account of the processes that took place to acknowledge and address the immense human sufering caused by the Nazi regime. They play a crucial role in documenting the complex journey of survivors and their families in seeking justice, recognition, and support for the harm they endured. They also provide insights into the legal, bureaucratic, and social challenges faced by those seeking compensation in the aftermath of such a devastating period in history. The Wiedergutmachung knowledge graph (Wiedergutmachung KG) contributes to our understanding of the impact of the totalitarian government in Germany on individuals and society and the ongoing eforts to address historical injustices. Construction of the Wiedergutmachung KG, illuminates the historical information hidden within unstructured Wiedergutmachung records, which were formerly tucked away in archival repositories, only accessible to archivists and a limited number of individuals entitled to see them for family or scientific research purposes. For instance, Wiedergutmachung KG can display connections between “claimants” and “compensation decisions”, elucidating trends and disparities within the Wiedergutmachung process.
The proposed digitalisation workflow starts with conversion of document images (i.e., scanned documents) to machine-readable formats through Optical Character Recognition (OCR). Subsequently, the workflow entails ontology design, information extraction, and linking entities with external data sources, to construct the Wiedergutmachung KG. Due to the historical nature of the documents, each of these stages present unique challenges that can be tackled using the more traditional methods, while the advancements in transformer-based technologies can also be used to address them with a more direct approach. The speed of technological advancements in the field of AI calls for a dynamic pipeline with a modular design that allows for substitution or 1“Wiedergutmachung” is a German word that translates to “making good again” or “making amends”. In the context of National Socialism in Germany and its aftermath, it specifically refers to the eforts made to compensate survivors and victims for the losses they sufered during the rule of the Nazi regime. 2https://www.archivportal-d.de/themenportale/wiedergutmachung combination of rule-based methods for accuracy with nascent AI technologies for optimisation. Keeping this consideration in mind, the same digitalisation workflow can be extended to further use cases and Digital Humanities research can benefit from the lessons learnt during the design of such a pipeline.
The remainder of this paper delves into details of the diferent components of the proposed digitalisation pipeline for Wiedergutmachung compensation records and discusses the intricacies of working with historical archival records. Section 2 outlines similar eforts for construction of domain-specific KGs in the context of Cultural Heritage. In Section 3 the research questions are introduced and methodologies for addressing them are discussed. Section 4 concludes the paper and sketches the next steps for future work.
As archives undergo mass digitisation and the volume of digital records grows, there arises a
rich but underutilised resource for researchers in the Digital Humanities. Integration of data
from historical archival records into the Semantic Web and transformation of archival data
according to Linked Open Data (LOD) principles has been receiving significant attention by
scholars [
The Sampo series of semantic portals is a pioneering efort in applying Semantic Web
technologies to showcase Finland’s national heritage [
The Jewish Contemporary Documentation Center has created an online LOD database 3
focused on Italian Holocaust victims and persecution events. Additionally, they’ve developed
an associated application for utilising this valuable data [
In the Netherlands, “Oorlog voor de Rechter” (“War in Court”) 4 aims to unlock historical knowledge by making The Central Archives of the Special Jurisdiction (CABR) accessible online, leveraging advanced technologies and a user-centric design. CABR is the largest war archive in the Netherlands, containing files of over 400,000 people suspected of collaboration with the National Socialist regime in Germany.
The European Holocaust Research Infrastructure (EHRI) Portal 5 serves as a valuable resource
for researchers and historians interested in Holocaust-related archival material. It provides
access to electronic finding aids, inventory information on institutions holding
Holocaustrelated records, and vocabularies related to archival descriptions [
In Germany, to the best of our knowledge, this work marks the first efort to develop an LOD-based semantic portal from archival records pertaining to World War II and National Socialist Injustices.
Transformation of data hidden within Wiedergutmachung compensation records into LOD for increased accessibility, interoperability, explorability, and semantic enrichment is the main goal of this work. Therefore, the overarching research question in this work is: What is the most eficient pipeline for transformation of historical archival records into a Knowledge Graph-based information system, for integration into the Semantic Web and publication as Linked Open data? In order to accurately address the overarching research question, it seems necessary to break it down into the diferent components of such a pipeline, for an informed design decision. The research questions derived from a modular design for this pipeline are as follows: RQ1: What advanced techniques and methodologies can be developed to improve the accuracy of text recognition for digitised archival records with challenging characteristics such as faded ink, handwritten annotations, and non-standard fonts? RQ2: What are the most efective Information Extraction (IE) techniques for accurately and eficiently identifying and retrieving structured data, such as names, dates, and locations, from digitised archival records? RQ3: How can existing ontologies be adapted and extended to develop an ontology for representation of archival historical records, that accurately reflects the hierarchical structure of archival records, semantic annotation of the records, and information about the agents involved in the creation and archiving of the records RQ4: How can we establish reliable links between historical entities (e.g., people, places, and events) extracted from digitised archival records and relevant external databases, authority ifles, or reference materials? The methodologies and initial experiments for addressing RQ1, RQ2, and RQ3 are explained below. Solutions for RQ4 are yet to be explored as a part of future work.
The development of Wiedergutmachung KG hinges on the creation of an ontology capable of modelling relationships among archival documents, court proceedings, individuals, and organisations involved in the compensation application, decision-making, and document archiving. Ensuring the validity and reliability of this model requires incorporation of domain experts’ requirements and knowledge. Archivists from the State Archives of Baden-Württemberg 6 collaborated with the author to formulate a list of competency questions 7, serving as the foundation for the ontology’s conceptual modeling. These questions, catering to researchers/historians and relatives/dependents of persecuted persons, provide insights into the domain’s scope, structure, and concepts.
According to the competency questions and based on the best practices in the field of Ontology
Design, the CourtDocs Ontology [
Archival Hierarchy and Provenance. The Records in Context ontology (RiC-O8) [
Court Procedures. The PROV Ontology (PROV-O9) [
Biographical Information of Persons Involved. CIDOC-CRM is employed as the
ontological foundation for representing the biographical information of individuals involved in the
compensation process 10. The use of a harmonising data model facilitates connection with other
materials and external sources. Moreover, the event-centric approach of CIDOC-CRM, in which
an individual’s existence is perceived as a series of interconnected events spanning across time
and space [
Creation of transcripts from scanned documents using OCR systems, greatly accelerates and
streamlines the retrieval of information. However, OCR system eficacy is contingent upon
factors such as text and font styles. OCR systems are typically specialised for either machine-printed
or handwritten text due to their distinct visual characteristics [
Wiedergutmachung compensation records constitute of diferent document types, such as application forms and index cards, and diferent layouts for each document type that vary based on time and across diferent institutes. In a traditional information extraction pipeline, this necessitates implementation of an automatic document identification system that classifies the documents based on their type or layout, and feeds them into the respective information extraction script customised for each document type. Our experiments on rule-based information extraction using Apache Uima Ruta [20] with a subset of 75 documents with three diferent layouts show an accuracy of 75%, for exact matches only. However, with the advance of Large Language Models (LLMs), there is an opportunity to streamline the information extraction process, instead of laboriously crafting multiple scripts for each document and layout type. In the proposed LLM-based approach, a unified prompt, coupled with the appropriate context, can facilitate the extraction of information from all the documents spanning across document types and layouts. The quality of information extraction from archival records with LLMs is still to be evaluated for a more accurate analysis and comparison with the rule-based methods.
The presented research provides a significant contribution to Digital Humanities research, particularly on topics related to World War II and National Socialist Injustice. Furthermore, the ifndings from implementations of diferent techniques on archival records can be extended to similar eforts on transformation of archival records into LOD.
In the next stages of the research, the focus will shift towards implementation of transformerbased technologies in the LODification pipeline and comparing the performance of these methods against the more traditional methods for each of the constituent pipeline components. Moreover, RQ4 from Section 3 will be addressed to interconnect the KG with external sources and authority files. This is a crucial step to facilitate content-based and federated semantic search and to enrich the KG. In case of the Wiedergutmachung KG, apart from Wikidata 12, there are several other knowledge- and databases, representing data on German figures and the victims of National Socialism that can be interlinked with the KG. It is also necessary to map all the extracted information to specific unique entities (e.g., persons) by means of disambiguation and entity resolution techniques.
This work is funded by the German Federal Ministry of Finance (Bundesministerium der Finanzen) 11https://www.transkribus.org/de 12https://www.wikidata.org/ and supervised by Prof. Dr. Harald Sack. cation in historical archival documents, in: Archiving Conference, volume 19, Society for Imaging Science and Technology, 2022, pp. 15–20. [17] M. Vafaie, J. Waitelonis, H. Sack, Improvements in Handwritten and Printed Text Separation in Historical Archival Documents, in: Archiving Conference, volume 20, Society for Imaging Science and Technology, 2023, pp. 36–41. [18] M. Li, T. Lv, et al., Trocr: Transformer-based optical character recognition with pretrained models, in: Proc. of the AAAI Conf. on Artificial Intelligence, volume 37, 2023, pp. 13094–13102. [19] P. B. Ströbel, T. Hodel, W. Boente, M. Volk, The Adaptability of a Transformer-Based OCR Model for Historical Documents, in: Intl. Conf. on Document Analysis and Recognition, Springer, 2023, pp. 34–48. [20] P. Kluegl, M. Toepfer, P.-D. Beck, G. Fette, F. Puppe, Uima ruta: Rapid development of rule-based information extraction applications, Natural Language Engineering 22 (2016) 1–40.