RiC-O Converter is an open-source command-line tool to convert EAD finding aids and EAC-CPF authority records to RDF files conforming to Records in Contexts ontology, in a robust manner. It was developed for the Archives nationales of France (ANF) but is aimed to be reused by other archival institutions, and to this aim is fully documented in English. It is based on XSLT stylesheets that take into account the variability of EAD content. It enabled the ANF to convert 15000 EAC-CPF files and 29000 EAD files into an homogeneous knowledge graph. Such a graph opens new perspectives for navigating and linking from/to archival metadata.
1 ICA Records in Contexts Ontology (RiC-O) [
Florence Clavaud and Tobias Wildi. 2 RiC-O Converter source code: https://github.com/ArchivesNationalesFR/rico-converter (last accessed 2021/07/03).
Many archival institutions and projects (like portals such as Archives Portal Europe and
FranceArchives) around the world use XML/EAD and XML/EAC-CPF files to
describe their collections and the agents related to them. Based on ISAD(G) standard,
XML/EAD (EAD) [
RiC-O Converter project is based on the statement that transforming EAD and EAC files to RDF, thus creating knowledge graphs about archives and their contextual entities, results in an homogeneous and interoperable data structure, that is compliant with the FAIR principles,3 and opens new perspectives related to querying, browsing, reusing, publishing and linking from / to archival metadata.
The ANF4 have been interested in entity-relationships models and graph
technologies since 2013, one of the reasons being that they already have authored a significant,
and growing, number of authority records that were linked to each other and to the
descriptions of the archives themselves. This in essence constitutes a very dense
oriented graph, whose relations are not really displayed, and cannot be queried and
processed in the ANF current information system [
The ANF first built a qualitative proof of concept (PIAAF)5, to show that converting existing archival metadata to RDF datasets conforming to RiC-O was possible, to interconnect datasets from different institutions, and to visualize and explore them in a new way. But the PIAAF prototype did not include a large quantity of metadata nor took into account the variety of their structure and content. Therefore, the ANF needed to move from this qualitative proof of concept to a large-scale project. Indeed, the ANF hold a significant amount of metadata to be converted, which implied to develop a reliable, efficient, configurable, tool. The tool was designed to process finding aids and authority records only, even if the ANF also hold several controlled vocabularies. Building RDF/RiC-O vocabularies is a different process and is still a work in progress in the institution, since the internal data structure of these controlled vocabularies, that currently conform to a very poor, locally defined, model, should change soon.6
The conversion tool was required to be industrial (meaning: performant and capable of processing tens of thousands of input files in a reasonable amount of time); tested (to guarantee the coverage of all possible situations encountered in source files); verbose (to produce log files to follow its execution), easy to install (so it can run on a typical desktop machine), configurable and adaptable (to suit the ANF needs as well as different needs of potential reusers), well documented. 3
The proposed solution relies on XSLT stylesheets, encapsulated in a Java script. The Java wrapping of the XSLT ensures a convenient command-line interface, the proper sequencing of the conversion steps, and portability to all operating systems.
The stylesheets convert EAD and EAC in RDF/XML containing instances of RiCO classes and properties. They live in a separate directory from the java command itself, ensuring that modifications can easily be made in conversion logic without the need to recompile the tool.
The development methodology relied on unit tests to cover all possible situations that could be found in input files. Each of the 90 unit tests is specified in an input EAD or EAC file with a corresponding expected RDF/XML file.7 When run, the output of the converter is compared to the expected file, and, if differences are found, the test fails. The tests can be run directly from the command line, so that any user can verify the tests, and add its own, if the stylesheets are modified. The tests ensure no regressions are introduced when the software evolves.
Running the tool is as simple as running a bat or sh script. The script asks for the action to be executed (EAC or EAD file conversion) and the option properties file to use.
The EAC to RIC-O conversion process is summarized in the following diagram: 4 1. Input EAC files are converted into RiC-O RDF/XML. Each input file yields a corresponding RDF/XML file. An option allows to stop processing here to examine the raw output of the conversion. 2. The content of the raw RDF/XML files is reorganized to split the output in folders corresponding to agents, places, and relations. Relations are grouped into large files, each corresponding to a high-level relation class in RiC-O: Agent Hierarchical Relations, Agent Origination Relations, Agent Temporal Relations, Agent To Agent Relations, Family Relations, Membership Relations, Work Relations. 3. The relations are deduplicated to remove those that appear more than once. As the original relation is expressed in the source files for both related entities, the same relation expressed in RiC-O was generated twice in step 1.
The EAD to RiC-O conversion process is summarized in the following diagram: non-public files or components of files can be excluded from the conversion process. 2. The filtered EAD files are then converted into RiC-O RDF/XML. Each input file yields a corresponding RDF/XML file. 3. If requested, the output files can be splitted into smaller files, with the top Record It should be noted that the conversion step to RiC-O RDF/XML takes the assumption the corresponding Record Resource URI is derived.
The EAD conversion takes into account some variability of what can be found in EAD files, where the same element is allowed to contain different content. For extent includ vocabulary, result in different outputs, as shown in the corresponding unit test.8
The performance of the tool is very good: 15200 EAC files are processed in approximately 15 minutes, yielding 0.7 million triples. 29000 EAD files are processed in approximately 30 minutes, yielding approximately 155 million triples. 4
The software is fully documented in English to reach the international community. The project team produced mappings between EAC-CPF and EAD to RiC-O, which will be very useful to the archives community, especially institutions considering converting their metadata to RDF. The mappings are available in the documentation section9 of the tool. The conversion process and how to customize the conversion are documented too. The source code is open and freely accessible on GitHub. The software is licensed under the terms of the CeCILL-B license.10
The converter was developed to address first the needs of the ANF but we kept in mind its potential uses for any other archival institution; therefore the code is easily configurable. Typically, the root URI for the URIs to be generated is an option that can be easily changed. However archival institutions would probably need to adapt the software to their own systems. For example, we did not take into account some elements in the conversion process because these elements are not used in the ANF EAD finding .11 5
The ANF have converted nearly 29000 finding aids and 15200 authority records using RiC-O Converter. We can convert them again when needed, for example when major updates occur in our metadata.
Quality issues appeared during development time (lack of precision or worse, bad use of EAD format). More generally speaking, improving the quality of archival metadata is a key issue for the ANF, and quality management and data governance also has to be enhanced. In a way, processing the RDF datasets generated can help assess this problem and solve it. Examples include: aligning the data on the agents to other RDF datasets e.g. those of the French national Library (BnF)12 or of Wikidata13 in order to enrich them, or (not investigated yet but an important need) linking (merging) distinct descriptions, authored through time in distinct finding aids, of the same archival resources.
Also related to quality would be the use of SHACL14 rules to assess the conformance of the generated graph structure against some rules; these rules could be derived directly from the RiC-O ontology15 (typically cardinality, domain and range check), or they can be hand-written to validate business oriented patterns in the graph. This is something to be done in the future.
In terms of challenges, RDF datasets resulting from the conversion are not published or searchable yet because of the lack of infrastructure in the ANF information system. The ANF do not have any triplestore available online. However, the ANF published dumps of the RDF datasets.16 Besides, the data are already used in research projects such as ALEGORIA,17 a project that aims at facilitating the promotion of iconographic institutional collections describing the French territory in various periods going from the interwar period to our days. A triplestore accessible through a SPARQL endpoint will soon be released. It will be connected to a web application demonstrating 3D immersive navigation through geolocalised photographs.
Moreover, the ANF should release a quite large-scale prototype by the beginning of 2022, including an easy-to-use, visual, SPARQL query interface.18
More generally speaking, RiC-O Converter needs to evolve for several reasons. RiC-O Converter is based on RiC-O 0.1 (dated December 2019); but Records in Contexts has evolved since that date: RiC-O 0.2 was released in February 2021, and introduces new components (like the Extent class) as well as updates (particularly as concerns the names of several object properties). The corresponding changes will be made in RiC-O Converter before the end of 2021.
RiC-O Converter does not convert XML/SEDA19 files, used in French digital archives management systems to describe these digital archives. These files include technical and preservation metadata, whose definition is inspired by PREMIS data dictionary20 which is more widely known in archives. Mapping SEDA to RiC-O and its transformation is a task to do in a future version of RiC-O Converter.
The transition from existing formats to novel graph-based and web-oriented conceptual models represents a challenge that can hinder the adoption of such new models. We especially think of FRBR and LRM in the library world, or CIDOC CRM for museums. By providing RiC-O Converter, a robust, adaptable and off-the-shelf tool to transition from EAD and EAC to RiC-O, the archival community aims at soothing and encouraging this transition, in order to make archival data part of the Web of data.