One of the great promises of Linked Data and the Semantic Web standards is to provide a shared data infrastructure into which more and more data can be imported and aligned, forming a sustainable, ever growing knowledge graph or linked data cloud, Web of Data. This paper studies and evaluates this idea in the context of the WarSampo Linked Data cloud, providing an infrastructure for data related to the Second World War in Finland. As a case study, a new database of prisoners of war with related contents is transformed into linked data and integrated into WarSampo. Lessons learned are discussed in relation to using traditional data publishing approaches.
Data about the Second World War (WW2) is heterogeneous and distributed in
di erent organizations and countries. WarSampo [
In the Linked Data architecture [
This paper presents preliminary results of integrating data about the Finnish prisoners of war into the WarSampo infrastructure. This data publishing method is compared to two competing traditional, approaches that could also have been used for the data publication. We evaluate the following three data publishing approaches: 1. Data les. Publishing the data as separate data les, e.g., as spreadsheets. 2. Database & web interface. Inserting the data into a (relational) database, and creating a web user interface for querying and displaying the data. 3. Linked Data. Publishing the data as Linked Data, using the WarSampo infrastructure.
These approaches are compared to each other to nd the bene ts and
drawbacks of each. The dataset is being published using the WarSampo approach,
but exists also as spreadsheets, which e ectively allows for comparison of these
approaches. The Database and web interface option is evaluated based on our
experience with these technologies, and also on examination of the online
publication of the Finnish WW2 casualties of war5 database by the National Archives
of Finland, a dataset that has also previously been integrated into WarSampo [
A new application perspective was created into the WarSampo portal for studying the prisoners of war dataset individually, while also integrating the prisoners into the WarSampo person perspective, and extending person pages to show the new prisoner data. The new perspective facilitates prosopographical study of the prisoners using the whole dataset, or a subset of the data based on user interest.
The core of the new data is a register of Finnish prisoners of war, containing data about some 4 460 prisoners in the Soviet Union as a spreadsheet. Additional information about related prisoner camps and hospitals is contained in separate spreadsheets. The data includes also documents about the prisoners of war to provide additional information.
There are many studies about the Finnish WW2 prisoners of war [
The prisoner data has originally been published in a book [
The data in the prisoner register has been gathered from several Finnish and
Russian archives [
The alternative approaches to publishing the prisoner data are discussed in the next two sections, followed by the description and evaluation of publishing the dataset as part of the WarSampo data service (SPARQL endpoint) in the Linked Data Finland service6 and the semantic portal7 for end users. 2
The simplest way of publishing the dataset would be to make it available as spreadsheets in, e.g., CSV format. This enables easy open access to the dataset, but does not facilitate the creation of interactive applications using the data. The data would also not bene t from information in other datasets, as no links to other datasets would exist. The bene ts (+) and challenges (-) of this approach are: + Low e ort. Not much work is required to publish the data online. - Static data. The dataset can only be accessed as static les, and not through a programmable interface. - No linking. The data forms a new silo that can not communicate with other datasets. - No interactive applications. The data is in a format that is not feasible to use for creating interactive web applications. 3
In order to manage the data in a structured way, and to develop interactive applications using it, the dataset could be stored in a relational database. This would also enable the use of pre-existing tools and frameworks for said tasks. An application programming interface (API) could be developed for programmatic online access to the data, and a web-based application for end-users to browse the data.
However, this approach would still lack a standard way of linking the data to other data sources, and access to the data would be limited by the custom API. The pros and cons are thus: + Existing tools. Tools and frameworks for creating and managing relational databases and applications built using them are ubiquitous. - No linking. The data still forms a new silo that can not communicate with other datasets. - Non-standard data publication. The dataset can be made available through a custom API, but not freely queryable in a standard way.
Publishing data as Linked Data makes it possible to link individual pieces of information to each other via di erent relations. Related datasets can be linked to each other if they share same concepts or entities, like persons, military units or places. WarSampo is based on Linked Data, and is published openly via a SPARQL endpoint.
In contrast to the data le or relational database approaches, the linked data
approach requires tighter co-operation with the domain experts and data
publishers, especially in the creation phase in the life cycle of historical information
[
Choose the values that will be automatically linked to the WarSampo domain ontologies, and separate them into distinct columns.
Introduce identi ers for entities that appear on multiple spreadsheets. Develop a common practice for attaching a source for each value in a spreadsheet cell.
Decide how to express partially or completely missing information.
After initial versions of the spreadsheets were nished, a pipeline for converting the prisoner data into RDF format and linking them to the WarSampo domain ontologies was created. Source codes for data conversion and linking are available online8.
Due to lack of user friendly tools for editing the data in RDF format, it was decided that the domain experts kept on working with the spreadsheets that they were accustomed to use. Because of this the pipeline had to be adjusted every time the structure of a spreadsheet was modi ed. However, running the pipeline regularly fostered iterative development as the domain experts were able to preview how the spreadsheet data is visualized in the WarSampo web portal, and to suggest corrections. 4.1
The prisoners of war as Linked Data, created with the data transformation pipeline, is published in the WarSampo SPARQL endpoint9, in a separate named graph (http://ldf.fi/warsa/prisoners).
We use a simple primary data model for the data in RDF, resembling the
original format, in which the data was presented as spreadsheets. The data model
is similar to that of the WarSampo casualties [
have been harmonized using the dumb-down principle of Dublin Core10, i.e., by using shared super-properties and super-classes where applicable.
WarSampo uses the CIDOC Conceptual Reference Model (CRM)11 as a harmonizing data model. Individual prisoner records are modeled as instances of the CRM document class (E31 Document). Each record corresponds to a simple RDF transformation of a single database row, where the properties used correspond to the columns of the table. Using these properties, the data can be shown to the end user in an intuitive way similar to the orginal table.
In addition, this data is then used to create CIDOC CRM descriptions of the
actual persons and events, when appropriate. WarSampo person instances [
The Linked Data publication stores source information when present in the
original data. There are many ways of presenting this kind of provenance
information in RDF [
Some parts of the data had to be left out of the online data publication due to privacy legislation. This is done automatically based on the dates when prisoners have died, since the legislation does not concern people who have passed away more than 50 years ago. Basic information, like name and date of birth, can be published of all prisoners of war. 4.2
One of the most challenging aspects of the data transformation pipeline is
matching the persons from di erent datasets. The data models and content can
be highly varying, and di erent parts of person data can be missing in some
datasets. We were able to link 1 417 prisoners (32% of all prisoners) to persons
that already existed in the WarSampo actor ontology [
The prisoners' rank and military unit information were linked to the corresponding WarSampo datasets, of which 99% and 90% were linked successfully, respectively. Successful linking allows for enriching all of these datasets with each other's information content. 4.3
The new prisoner perspective of WarSampo uses SPARQL Faceter [
The WarSampo person perspective shows a biographical view of a single person by bringing together all information that exists about the person | including the information in the prisoner dataset. Each person's page shows the basic information about a person, while also showing the prisoner record, and additional documents, like interrogation sheets, pictures, and videos that exist about that individual.
A major bene t of the Linked Data approach is that the SPARQL endpoint allows to nd answers to many complex research questions by querying the data directly. Asking complex queries, however, requires a person to be familiar with the SPARQL syntax. We can, for example, easily nd out what are the battles of the Winter War, where the most soldiers have become prisoners, by exploiting the unit links of the prisoner records, and the existing data links in WarSampo. A single SPARQL query14 can reveal this information. The result of this query shows us what military unit was involved in a battle, how many prisoners of war were taken during a battle, how many prisoners of war were taken from the unit totally, and how many soldiers perished during the battle.
The most prisoners were taken during the battle of Leipasuo. This also accounts for 86% of the prisoners of war of the involved military unit (JR9 ). This information also enriches the prisoners of war data, since prisoners taken from JR9 were not previously annotated with any place of capture | now due to linking we know where they were at the time. The ve Winter War battles found in WarSampo in which the most prisoners were taken are the following: 4.4
Bene ts and challenges of the Linked Data approach are: + Availability. The data is published on an open standardized API for others to use. Resolvable identi ers allow anyone to look up entities using normal web technologies. 13 https://github.com/SemanticComputing/prisoners-demo 14 The query is available at http://yasgui.org/short/Hy2bTFOCb + Identi ers. Identi ers are used for all information; this enables the linking of the pieces of data. Also future datasets can refer to the same identi ers. + Interlinking. The data can be enriched from multiple related datasets. + Web user-interfaces. It is easy to implement user-friendly web perspectives that dynamically use the data via a SPARQL API. + Validation. The data transformation process validates the structure of the data, and can be used to nd structural errors easily. - More work needed. Creating a new RDF data model, transforming the data into RDF, interlinking to other datasets, and extending the web portal requires more work. - Coordination. Data linking requires more coordination between data producers and data publishers. - Slower querying. Querying data in the RDF data model usually results in higher response times than data in simpler formats. 5
This paper overviewed a case study of publishing Finnish prisoners of war content as linked open data, using the WarSampo infrastructure. We discussed the pros and cons of this approach in relation to two competing traditional approaches.
A signi cant di erence between the traditional data publishing approaches (data le, relational database) and the Linked Data approach is that the traditional approaches allow for more straightforward work ows. Data cleaning and validation is not as critical, and the data gathered by domain experts can be converted into a desired format using simple scripts that do not necessarily modify the original data in any way. The Linked Data approach, on the other hand, always requires careful inspection of the data provided by the domain experts.
This paper illustrated that linking the data to the domain ontologies is one key distinction between the traditional and Linked data approaches. The linking process includes demanding tasks, such as choosing which part of the data is suitable for linking, cleaning and harmonizing the data, and nally developing automatic or manual work ows for data linking.
Our experiment suggests that the data linking process should take place as early as possible in the lifecycle of publishing historical information, preferably in the data creation phase, in order to maximize precision and recall. With this project it was possible to facilitate the linking process with the domain experts during the data creation phase, but often this is not the case. Applying Linked Data publishing principles to a dataset that has been previously published as a data le or database requires either error-prone interpretation of the original data, or co-operation with the domain experts that created the original data. Our work was funded by the Association for Cherishing the Memory of the Dead of the War15, Open Science and Research Initiative16 of the Finnish Ministry of Education and Culture, the Finnish Cultural Foundation, and the Academy of Finland. 15 http://www.sotavainajat.net/in_english 16 http://openscience.fi/