-Linked data has finally arrived. But with the availability and actual usage of linked data, data from different sources gets quickly mixed and merged. While there is a lot of fundamental work about the provenance of metadata and the commonly recognized demand for expressing provenance information, there still is no standard or at least best-practice recommendation. In this paper, we summarize our own requirements based on experiences at the Mannheim University Library for metadata provenance, examine the feasibility to implement these requirements with currently available (de-facto) standards, and propose a way to bridge the missing gaps. By this paper, we hope to obtain additional feedback, which we will feed back into ongoing discussions within the recently founded DCMI task-group on metadata provenance.
At the Mannheim University Library (MUL), we recently announced a Linked Data Service1 (LDS). Our complete catalog with about 1.4 million is made available as RDF, with proper dereferenceable URIs and a humanreadable presentation of the data as HTML pages. The title records are linked to classification systems, subject headings and to other title records. The Cologne University Library made its catalog data available under a creative commons CC-0 license, so we converted it to RDF and made it available along our own catalog.
The HTML view2 provides browsable pages for all resources described in the RDF data. It fetches additional statements when users click on the URIs, provided that they are available by URI dereferencing. The resulting statements are presented to the user within the LDS layout and cannot be easily distinguished from the data that is made available by the Mannheim University Library itself. There is only a note about the “data space”, basically indicating the domain where the dereferenced URI resides.
A good thing is that the service is totally sourceagnostic and fetches and presents everything that is available. With two clicks, the user gets subject data from the library of congress (LoC), just because we use the German subject headings and the German National
Library (Deutsche Nationalbibliothek, DNB) provides skos:match statements to LoC subject headings (LCSH).
A bad thing is that the service is totally sourceagnostic (apart from the data-space notion). For example, the DNB states on its website that the data is provided only as a prototype, should only be used after a consultation and not for commercial applications. The LCSH data is public domain and freely available. But also within our triple store, there are different datasets. The MUL catalog is currently provided without a specific license, as questions about the proper licensing still are discussed. The data from the Cologne University Library has been processed by us and the processed data is provided by a the creative commons CC-0 license, too.
Our predicament: We do want the LDS to be sourceagnostic. But at the same time we want to know about the license of the data that is displayed to the user, and we want to present him with this information. Moreover, besides license and source information, we also have other information that we would like to make available to the user or other applications in a reusable way. But the current state of the art is that this information is either not made available within the RDF datasets yet – the case for DNB, LoC and our own data – or not in a consistent way. For example, the data from the OCLC service dewey.info3 contains licensing statements as part of the RDF statements about a given resource (Ex. 1). <http://dewey.info/class/641/2009/08/about.en> a <http://www.w3.org/2004/02/skos/core#Concept>; xhv:license
<http://creativecommons.org/licenses/by-nc-nd/3.0/>; cc:attributionName
"OCLC Online Computer Library Center, Inc."; cc:attributionURL <http://www.oclc.org/dewey/>; ... skos:prefLabel "Food & drink"; skos:broader
<http://dewey.info/class/64/2009/08/about.en>; cc:morePermissions <http://www.oclc.org/dewey/about/licensing/>.
Example 1: Provenance in dewey.info dataset
As another example, the New York Times expresses provenance outside the actual data record, more precisely by means of statements about the data record (Ex. 2). <http://data.nytimes.com/46234942819259373803.rdf> foaf:primaryTopic
<http://data.nytimes.com/46234942819259373803> dcterms:rightsHolder
"The New York Times Company" ... cc:license
<http://creativecommons.org/licenses/by/3.0/us/> cc:attributionURL
<http://data.nytimes.com/46234942819259373803> cc:attributionName
"The New York Times Company" <http://data.nytimes.com/46234942819259373803> ... a <http://www.w3.org/2004/02/skos/core#Concept> ... skos:prefLabel "Faircloth, Lauch" ...
Example 2: Provenance in New York Times dataset
Our goal is to make this kind of information available to the user in a consistent way. We respect all the different licenses and do not want to make users believe that all this data is provided by ourselves, without any licensing information.
Besides provenance information, we also need to
provide other information that further qualifies single
statements of the datasets. For example, in a past project we
automatically created classifications and subject headings
for bibliographic resources. We provide this data also via
the LDS which is very convenient and greatly facilitates
the reuse of the data. But automatically created results
often lack the desired quality, moreover the processes
usually provide further information, like a weight, rank
or other measures of confidence [
Data provided as RDF is not necessarily metadata in a strict sense; in general it is data about resources. But in many cases – and especially in the context of this paper – the resources are data themselves, like books, articles, websites or databases. In the library domain, the term “metadata” is thus established for all the data about the resources a librarian is concerned with – including, but not restricted to bibliographic resources, persons and subjects. This is the reason, why one cannot distinguish easily between data and metadata in the context of RDF. We therefore regard them as synonyms.
Metadata is itself data and there are a lot of use-cases where one wants to make further statements about metadata, just as well as metadata provides statements about data: who created the metadata, how was the metadata created, ... – in general additional statements to further qualify and describe the metadata. Thus we will refer to this kind of additional information unambiguously as “metametadata”.
To achieve interoperability for accessing metametadata, choosing a representation of the metametadata is only the first, merely technical step. In our opinion, the following principles and requirements have to be met to achieve this type of interoperability: 1) Arbitrary metametadata statements about a set of statements. 2) Arbitrary metametadata statements about single statements. 3) Metametadata on different levels for each statement or sets of statements. 4) Applications to retrieve, maintain and republish the metametadata without data loss or corruption. 5) Data processing applications to store the metametadata about the original RDF data.
Requirements 1 - 3 address the technical requirements that have to be met by the metadata format(s) in use. They are met by RDF, but in RDF there are two distinct approaches that can be used to represent metametadata:
Reification: RDF provides a means for the formulation of statements about statements, called reification. In the RDF model, this means that a complete statement consisting of subject, predicate and object becomes the subject of a new statement that adds the desired information.4
Named Graphs: Another technique that can be used
to provide statements about statements are the “Named
Graphs”, introduced by Carroll et al. [
To meet requirements 4 and 5, further conventions among interoperable applications are needed that have to be negotiated on a higher level and are (currently) beyond the scope of RDF. By virtue of the following usecases, we demonstrate that the technical requirements are already met and that we only need some conventions to represent such information in an consistent way – at least as long as the official RDF standard does not address the metametadata issue.
The following use cases7 are meant to be illustrating examples, especially to emphasize the need for the representation of arbitrary information – not only provenance – about data on various levels, from whole datasets over records to single statements or arbitrary groups of statements.
In this section, we develop a scenario where such metametadata can be used to prevent information loss while merging subject annotations from different sources. We show that this is the key to make transparent use of different annotation sources without compromises regarding the quality of your metadata. In line with our argumentation in this paper, we propose the storage of metametadata to mitigate any information loss and allow the usage of this information to achieve a better retrieval experience for the users. With various queries, we show that we can access and use the additional pieces of information to regain a specific set of annotations that fulfills our specific needs.
This scenario focuses on the merging of manually
assigned subject headings with automatically assigned
5See http://www.w3.org/2004/03/trix/ for a summary. There are
already further extensions or generalizations of Named Graphs, like
Networked Graphs [
6You can see the grouping of statements in a single RDF-XML file
as the notion of an implicit graph and use the URI of the RDF-XML
file to specify further statements about this graph, just like Ex. 2
7First published at the DC 2009 conference [
There is one document (ex:docbase/doc1) with assigned subject headings from two different sources. For each subject assignment, we see that a source is specified via a URI. Additionally, a rank for every assignment is provided, as automatic indexers usually provide such a rank. For example, a document retrieval system can make direct use of it for the ranking of retrieval results. For manual assignments, where usual no rank is given, this could be used to distinguish between high quality subject assignments from a library and, for example, assignments from a user community via tagging.
The statements #13 and #14 are used to further qualify the source, more precisely, to indicate, if the assignments were performed manually (ex:types/manual) or automatically (ex:types/auto).
Usually, the statements from Example 3 are available from different sources (as indicated) and might also belong to different shells in the shell model. The integration requires to merge them in a single store. An interesting side-effect of the use of RDF and reification is that the merged data is still accessible from every application that is able to use RDF data, even if it is not possible to make reasonable use of our metametadata. This is demonstrated by the first query in Example 4, which retrieves all subject headings that are assigned to a document. As in RDF all statements are considered identical that have the same subject, predicate and object, every subject heading is returned that is assigned by at least one source. In most cases, these completely merged statements are not wanted. As promised, we show with the second query in Example 4 that we are able to regain all annotations that were assigned by a specific source (here ex:sources/pfeffer).
SELECT ?document ?value WHERE {
?t rdf:subject ?document . ?t rdf:predicate dc:subject . ?t rdf:object ?value . } document subject ex:docbase/doc1 ex:thes/sub40 ex:docbase/doc1 ex:thes/sub30 ex:docbase/doc1 ex:thes/sub20 SELECT ?document ?value WHERE {
?t rdf:subject ?document . ?t rdf:predicate dc:subject . ?t rdf:object ?value . ?t ex:source <ex:sources/pfeffer> . } document ex:docbase/doc1 ex:docbase/doc1 subject ex:thes/sub40 ex:thes/sub30 source ex:sources/pfeffer ex:sources/pfeffer
In the following we show two extended queries that make use of the metametadata provided in our data store. Usually, one does not simply want to separate annotation sets that have been merged, but instead wants to make further use of these merged annotations. For example, we can provide data for different retrieval needs.
The first query in Example 5 restricts the subject headings to manually assigned ones, but they still can originate from different sources. This would be useful if we are interested in a high retrieval precision and assume that the results of the automatic indexers decrease the precision too much.
The second query, on the other hand, takes automatic assignments into account, but makes use of the rank that is provided with every subject heading. This way, we can decide to which degree the retrieval result should be extended by lower ranked subject headings, be they assigned by untrained people (tagging) or some automatic indexer.
Early initiatives to define a vocabulary and
usageguidelines for the provenance of metadata was the
ACore [
FILTER ( ?type = <ex:types/manual> ) } } document subject type ex:docbase/doc1 ex:thes/sub40 http://example.org/types/manual ex:docbase/doc1 ex:thes/sub30 http://example.org/types/manual SELECT DISTINCT ?document ?subject WHERE { ?t rdf:subject ?document . ?t rdf:predicate dc:subject . ?t rdf:object ?subject . ?t ex:source ?source . ?source ex:type ?type . ?t ex:rank ?rank .
FILTER ( ?type = <ex:types/manual> || ?rank > 0.7 ) document ex:docbase/doc1 ex:docbase/doc1 ex:docbase/doc1 subject ex:thes/sub40 ex:thes/sub30 ex:thes/sub30 Example 5: Ranked assignments and additional source information enough to enable applications the automatic integration of these information without proper knowledge, how the information is actually represented from a data model perspective.
An implementation with a clear semantic of metadata
provenance statements is included in the protocol for
metadata harvesting by the The authors (Rephrase with
cite) in [
Hillmann et al. [
Currently, the W3C Provenance Incubator Group
(Prov-XG, http://www.w3.org/2005/Incubator/prov/)
addresses the general issue of provenance on the web.
The requirements abstracted from various use-cases are
summarized and further explained in by Zhao et al.
[
Lopes et al. [
A general model for the representation of provenance
information as well as a review of provenance-related
vocabularies is provided by The authors (Rephrase with
cite) in [
With the Open Provenance Model (OPM, http://
openprovenance.org/) exists a specification for a
provenance model that meets the following requirements [
Finally, a comprehensive survey about publications
on provenance on the web was created by The authors
(Rephrase with cite) in [
The most powerful means to dealing with
metametadata in OWL is the use of higher-order logics, which
is supported, e.g., by OWL Full. However, as this type
of metamodeling comes at the expense of decidability
[
This paper is meant as a discussion paper. We have proposed five principles for the proper representation of metametadata which, in our opinion, have to be met by all source-agnostic, yet provenance-aware, linked data applications.
We have demonstrated that the technical requirements can already been met, and that the remaining problem is concerned with the establishments of conventions which define best-practice recommendations. In particular, these conventions should clarify how the metametadata is actually represented – so that an application can become aware of this metametadata, retrieve, maintain and republish it in a proper way. Currently, there is no accepted best-practice that follows our principles. We are involved in the Metadata Provenance Taskgroup of the Dublin Core Metadata Initiative9 which aims to develop such best-practice recommendations in an as-open-as-possible way. This is why we are seeking for feedback, ideas and contributions to the ongoing discussions and the outcomes of this task group – because we want metadata provenance. Now!
Acknowledgements. Johanna Vo¨lker is financed by a Margarete-von-Wrangell scholarship of the European Social Fund (ESF) and the Ministry of Science, Research and the Arts Baden-Wu¨rttemberg.