With the growing availability of Linked Open Data (LOD) and the consequential generation of derived and aggregated data, the need for trustworthy, reproducible and accessible provenance information has increased. Yet, no consistent mechanism has been established to manage provenance data of LOD on a global dataset-level. Decentralized networks and peer-to-peer mechanisms have made their revival in the last years with blockchain and similar distributed ledger technologies. We propose a novel approach to track and store provenance information for LOD on a dataset-level by sharing an immutable, common state between data providers. The basic architecture will not disrupt existing methodologies and standards for publishing LOD, but will be transparently integrated into existing ecosystems as an additional layer to foster broad acceptance. We will investigate the application of emerging blockchain technologies and established Linked Data specifications for building this decentralized anchor of truth. We are actively involved in the design and implementation of LOD and Open Data platforms and will evaluate our approach in real-world scenarios regarding feasibility, governance, scalability and usability.
The Linked Open Data (LOD) movement is a global phenomenon, driven by the
fact that additional value is generated by interlinking structured data. LOD is
Linked Data, which can be distributed by everyone anytime without any
restrictions. An active community has evolved around the publication and generation
of LOD. A popular publisher is WikiData, which freely offers comprehensive
data, completely serialized in the Resource Description Framework (RDF). [
Our work focuses on the latter approach, which is typically applied by Open
Data portals, aggregating public data, published by public administrations or
research organizations. Well-known examples are OpenAIRE [
The widespread distribution and availability of Open Data leads to the creation
and publication of derivative or edited datasets. Data from different sources and
origins is copied, aggregated, converted and/or enriched. Furthermore, claims
and conclusions are inferred from (combinations of) these datasets. The
traceability and repeatability of such data and its processing is critical for maintaining
trust and accountability. A central foundation for that is the presence of
expressive and valid provenance information for each dataset in an LOD processing
chain. No unified mechanism is established to record and track the
provenance of LOD on a dataset-level. The very nature of LOD causes
barriers in establishing appropriate measures. The intrinsic reason for this claim is:
LOD ecosystems constitute highly distributed and decentralized systems, where
data is acquired and aggregated across distinct organizational and technical
levels. An illustrating example for this is the harvesting process of LOD published
by public services. Typically, harvesting is conducted in a bottom-up form, where
municipal data providers publish data independently. This is followed by an
aggregation towards the next higher organizational level and so forth, e.g., towards
data portals of cities, federal states, etc. [
This methodology and the design of LOD leads to major challenges with respect to tracking the provenance of a single dataset: Firstly, there is no established approach for uniquely and persistently identifying a distinct dataset. Although the Linked Data principles require the use of Uniform Resource Identifiers (URIs) as unique identifiers, they can be easily reassigned and the DNS itself is transient.
Especially in the domain of research data, the application of Digital Object
Identifiers (DOIs) as a centralized workaround is established. Secondly, provenance
information is often not set, fragmentary or not correctly forwarded in an
acquisition and processing chain. Expressive and rich specifications for encoding
provenance are available, foremost the W3C PROV [
These limitations could be solved by establishing and agreeing on a central management system for provenance and identifier information. However, the very character of LOD is decentralization and sovereignty, involving multiple stakeholders and heterogeneous infrastructures. This environment makes the successful implementation of a centralized system infeasible. This leads to the essential hypotheses of this dissertation: An additional, immutable and decentralized network can help to overcome current drawbacks in LOD provenance tracking and incentivize its broad application. The recent developments in blockchain systems and related peer-to-peer (P2P) technologies will
be an important foundation for implementing such a network.
This approach can accomplish both: The management of provenance information through a homogeneous system and the protection of the independence of the data providers. This will support the fact that from an organizational point of view, LOD forms a highly decentralized system, which requires a single point of truth in order to ensure integrity and trust. 2
Due to the ongoing worldwide digitization, data has become a most valid asset
and the basis of many value-added processes and business models. Although our
work focuses on LOD, this is true for both, public domain data and proprietary
data. The relevance of trustworthy information about the provenance and
lineage of data will continue to increase. Simmhan et al. write "With a growing
number of datasets available in the public domain beyond the confines of a single
organization, it has become increasingly important to determine the veracity and
quality of these datasets." [
A lot of research was conducted in the relevant fields of our work. Our approach
crosses established research of provenance for LOD with blockchain and
distributed ledger technologies, which has been already examined to some degree.
In general, data provenance has been widely studied with respect to its use,
subject, representation, storing and dissemination and a variety of software
solutions have been developed for managing provenance. These approaches mainly
focus on local data, typically generated by a particular scientific domain, e.g.,
Physics, Earth Sciences, etc. [
Early work on provenance for Linked Data focused on modelling RDF
vocabularies and ontologies, which can be used to describe the provenance of published
RDF data and query it respectively. [
First research exists on the connection of distributed ledgers/blockchain
technologies and Linked Data/Semantic Web, spanning multiple aspects. English et
al. endorse the notion of improving the persistent identification of RDF resources
with blockchain. [
Blockchain and related technologies are vivid topics of research, where most
work focuses on privacy and security aspects. [
Based on the problem statement, the related work and the recent impact of distributed ledger technologies, new approaches for addressing provenance of LOD will emerge. Our work will focus on an additional, decentralized layer, accompanying existing solutions for publishing LOD. Therefore, we formulate the following research questions, where RQ1 represents the overall question.
RQ1: Can we manage the provision, management and traceability of provenance information for LOD datasets by applying an additional, decentralized layer?
RQ2: How can we persistently identify and represent provenance information of LOD in a globally unique way?
RQ3: What consensus and governance mechanisms can be applied to ensure the integrity of such a system?
RQ4: Which paradigms and tools are suitable to implement the proposed approach, considering expectations in flexibility, scalability and usability? 5
The following hypotheses relate to the aforesaid research questions. H1 depicts the overall hypothesis of the proposed thesis.
H1: A decentralized network, which holds a globally shared state for all data providers will improve the tracking and storing of provenance information of LOD in comparison to locally published provenance data.
H2: An immutable and transparent global database will improve the persistent and unique identification and management of provenance information over established transient approaches and enables a long-term preservation.
H3: An authority-based governance model and voting-based consensus mechanism will ensure a consistent state of the network and prevent misuse.
H4: Blockchain and related technologies can serve as a technical foundation for the proposed decentralized network. 6
In this section, we present first results and experiences from previous and ongoing work in the LOD and distributed ledger domains.
Our work on the EDP [
We have conducted several practical case studies with blockchain and distributed
ledger technologies to classify their opportunities and challenges. Based on the
public Ethereum blockchain we have implemented a decentralized digital
identity management system. It allows human users to acquire a persistent
identifier and link public properties, like date of birth, to it. The work is based
on Ethereum smart contracts and the Decentralized Identifiers (DIDs)
specification. [
cooperation beyond organizational borders. We acquired fundamental knowledge about governance, scalability and representing custom data in such a decentralized environment. 7
The overall approach is divided into four steps. Steps 1 and 2 refer to RQ2/H2, step 3 to RQ3/H3 and step 4 to RQ4/H4. The overall outcome relates to RQ1/H1.
LOD and the Semantic Web follow a decentralized methodology, still some aspects require a central authority to be most effective and accurate. An indicator for that are various approaches to harmonize LOD by some kind of central stewardship, e.g., Linked Open Vocabularies (LOV) or GeoNames.org.5 Our approach is the establishment of a decentralized network, which holds a globally shared state for all data providers and acts as an anchor of truth about provenance information. It is a single point of access for this global information, which simplifies management and traceability. Each participant of the ecosystem will act as distributed database node. The central premise is not to disrupt existing methodologies and standards, but to transparently integrate into existing ecosystems. Fig. 1 illustrates an exemplary high-level process for LOD aggregation, processing and use, including the decentralized provenance network. An LOD provider holds metadata and data and publishes a reference of them to the network. (here illustrated as hash values and a persistent identifier) An LOD aggregator copies only the metadata and extends the original reference accordingly. (indicated as sameAs) An LOD processor creates new data based on the original data and adds a new reference to the network, including a derivedFrom indication. Finally, a data publisher creates a visualization of the data, linking it to the reference in the network, allowing a clear tracking of the provenance of the data.
1. Formal definition of a dataset: LOD constitutes a set of triples (aka
statements), forming a multigraph. Our work does not base upon this smallest entity
of LOD. Typically, a distinct subset of triples forms a self-contained information
unit, restricted by pre-defined boundaries. Concepts like named graphs reflect
this approach. [
We will evaluate these assumptions against real-world LOD ecosystems and
publication schemes. The outcome will lead to accurate guidelines about who will
be allowed to add what data when in the shared store, formed by the network.
Especially, the on-boarding process in this decentralized environment needs to
be investigated. These assumptions will be evaluated with practical artifacts,
either based on existing technologies or individually implemented.
4. Implementation and evaluation of the provenance network: In the final step,
we will implement the network and apply it in a production environment. With
blockchain and similar distributed ledger technologies, decentralized networks
and peer-to-peer mechanisms have made their revival in the last years. A variety
of tools offer improved possibilities for sharing a common state and reaching
consensus in a decentralized environment. Multiple implementations exist for
building customized decentralized networks with desired characteristics: from
public, permissionless to private, permissioned networks, including custom
security and consensus protocols. These recent developments can operate as a
technical foundation for the proposed decentralized network. Yet, your work will
not be limited to blockchain and distributed ledger technologies, but will also
consider traditional peer-to-peer mechanisms and implementations.
The work here will be mainly conducted on two levels. (1) Providing the means
for actually creating the network. This includes a deployable node and a proper
on-boarding process to become a participant in the network. The setup of a
node is envisioned to be as straight-forward as possible. Container technologies,
like Docker, might be suitable approaches here. [
We plan to evaluate our hypotheses with the following four approaches. 1. Working prototype: Based on a proof-of-concept system, we will test and evaluate the fundamental functionality of our approach. Test data will be generated in real-world volumes. Synthetic, but representative stakeholders and actors will use the network. We will use the results and findings for improving our approach in an iterative manner. 2. Application in a production environment: We are actively involved in the implementation of LOD portals, like the EDP. Hence, we will apply our solution in a production environment and monitor its qualities and possible adoption. A cooperation with external stakeholders, like original data publishers and data users are requested. 3. Practical usefulness: We will measure and qualify multiple characteristics of the synthetic and the production system. This includes overall performance, throughput, maximum load and scalability. Since no system for comparison exits, we will evaluate the findings on established expectations for central solutions, especially for provenance tracking. 4. User studies: The rate of adoption of such a system, is highly dependent on user acceptance. We will conduct user studies within two different user groups: (1) Data providers will be asked to join the network by integrating it into their systems. (2) Data consumers will use the provided information to express provenance statements about given datasets. It is planned to conduct the user studies twice, with a working prototype and a production version. 9
To the best of our knowledge, the proposed research questions and the proposed approach is a novelty. There does not exist an established solution for a tamperproof and globally accessible ledger for provenance information about LOD. The recent developments and successful real-world applications of blockchain and similar networks have demonstrated the success and acceptance of a globally shared state-machine. However, we think that blockchain still has a long way to go and are aware of its current limitations. A complete migration from established centralized systems and architectures, especially in LOD, is improbable. An additional, decentralized layer, respecting established mechanisms and standards will have a much better chance for adoption. We are actively involved in many production LOD, Open Data and Open Science projects. Among others, this includes the development of the EDP and the design and installation of a research data platform for the Weizenbaum Institute for the Networked Society. This allows us to work closely with many relevant stakeholders and consider their needs and requirements, e.g., the data providers, users or system administrators. 10
This work has been funded by the Federal Ministry of Education and Research of Germany (BMBF) under grant no. 16DII111 ("Deutsches Internet-Institut") and is supervised by Prof. Manfred Hauswirth.