Provenance and other metadata are essential for determining ownership and trust. Nevertheless, no systematic approaches were introduced so far in the Linked Data publishing work ow to capture them. De ning such metadata remained independent of the rdf data generation and publishing. In most cases, metadata is manually de ned by the data publishers (person-agents), rather than produced by the involved applications (software-agents). Moreover, the generated rdf data and the published one are considered to be one and the same, which is not always the case, leading to pure, condense and often seductive information. This paper introduces an approach that relies on declarative descriptions of (i) mapping rules, specifying how the rdf data is generated, and of (ii) raw data access interfaces to automatically and incrementally generate provenance and metadata information. This way, it is assured that the metadata information is accurate, consistent and complete.
Nowadays, data owners publish their data at an increasing
rate. More and more of them publish also its
corresponding rdf representation and interlink it with other data.
However, even though provenance and other metadata
become increasingly important, most rdf datasets published
in the Linked Data cloud provide no or seldom narrow
metadata. To be more precise, only 37% of the published rdf
dataset provide provenance information or any other
metadata [
As a consequence, the lack of available metadata information neither allow being aware of the origin of the rdf data, nor reproducing the rdf data generation outside the context of the application that originally generated it. This occurs because most of the tools that generate rdf data derived from heterogeneous data, put the focus on independently providing the corresponding rdf representation, dissociating the resulting rdf data from its original source. In the same context, provenance and metadata information regarding the actual mapping rules which specify how the rdf data is generated from raw data, are not captured at all. Nevertheless, such information might equally in uence the assessment of the generated rdf data trustworthiness.
Similarly, data publishing infrastructures, such as triple stores, do not automatically publish any provenance or other metadata regarding the rdf data they host. Instead they would have been expected to enrich the metadata produced while the rdf data was generated with metadata associated with the publishing activity. Moreover, the rdf data generation and its publication are considered as interrelated activities that occur together. Although, this is not always the case. Therefore, the generated rdf data and the one subsequently published are not always one and the same. For instance, rdf data might be generated in subsets and published all together, or generated as a single dataset but published in di erent rdf graphs. Consequently, their provenance and rest metadata information is not identical.
In a nutshell, capturing provenance and metadata
information on every step of the Linked Data publishing
workow is not addressed in a systematic and incremental way so
far. In this paper, we introduce an approach that considers
declarative and machine-interpretable data descriptions and
mapping rules to automatically assert provenance as well
as other metadata information. Our proposed solution is
indicatively applied on mappings described using the rml
language [
The remainder of the paper is structured as follows: In Section 2, we outline the current state of the art. In Section 3, we discuss the essential steps of the Linked Data publishing cycle where provenance and metadata can be generated and in Section 4, we discuss the di erent levels of metadata details identi ed. In Section 5, we describe how machineinterpretable mapping rules are considered to automate the metadata generation and in Section 6 we showcase how we implemented it in the rml tool chain.
In this section, we investigate existing systems, involved in the Linked Data publishing work ow. Tools generating mappings and rdf data or publish rdf data are approached with respect to their support for automated metadata generation (Section 2.1). In addition, we outline the w3c recommended vocabularies for metadata description (Section 2.2), as well as the most well-known and broadly used approach for representing provenance and other metadata (Section 2.3). 2.1
In the Linked Data publishing work ow there are di erent activities taking place. Among them, the de nition of the rules to generate rdf data from raw data, its actual generation, its publishing and its interlinking are few of the most essential steps. However, the majority of the tools developed to address these tasks do not generate automatically any provenance or metadata information as the corresponding tasks are accomplished, let alone enriching metadata dened in prior steps of the Linked Data publishing work ow.
Hartig and Zhao [
None of the well-know systems that generate rdf representations from any type of (semi-)structured data provide any provenance or metadata information in conjunction with the generated rdf data, to the best of our knowledge. For instance, none of the prevalent tools for generating rdf data, such as DB2triples3, Karma4, or xsparql5, to indicatively mention a few of the prevalent tools. The main obstacle, at least with respect to provenance, is that it is hard to specify where the data originally resides. That occurs because most of these tools, consider a le as data input. However, where the data of this le is derived from is not known and, therefore, the corresponding provenance annotations can not be accurately de ned in an automated fashion.
The d2r server6 and the csv2rdf4lod7 are the only tool
that generates provenance and metadata information in
conjunction with the rdf data. However, the d2r server refers
only to data in relational databases, it supports a custom
provenance vocabulary, not the w3c-recommended
provo [
With the advent of mapping languages, such as the d2rq8,
sml9, or the w3c recommended r2rml [
The situation remains the same also in the case of
interlinking tools, such as the prevalent Silk [
In the same context, tools were developed to support data
owners to semantically annotate their data. However, those
tools still generate both the mapping rules and the
corresponding rdf data after the rules execution, without
providing any provenance or metadata information. To be more
precise, none of the tools that automatically generate
mappings of relational databases to its rdf representation, such
as BootOx [
Similarly, among the rdf data publishing infrastructures,
only Triple Pattern Fragments10 (tpf) [
Provenance and Metadata Vocabularies w3c recommended vocabularies were already de ned to specify rdf data provenance and metadata information: 2.2.1
The prov ontology (prov-o) [
According to the prov ontology, a prov:Entity is a physical, digital, conceptual, or other kind of thing. A prov:Activity occurs over a period of time and acts upon or with entities; 10http://linkeddatafragments.org/ 11http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/ 12http://4store.org/ 13http://lodlaundromat.org/ 14http://lodlaundromat.org/ontology/ it may include consuming, processing, transforming, modifying, relocating, using, or generating entities. A prov:Agent bears some form of responsibility for an activity taking place, for the existence of an entity, or for another agent's activity. 2.2.2
The Vocabulary of Interlinked Datasets (void) [
According to the void vocabulary, a void:Dataset is a set of rdf triples maintained or aggregated by a single provider. A void:Dataset is a meaningful collection of triples, that deal with a certain topic, originate from a certain source or process, and contains su cient number of triples that there is bene t in providing a concise summary. The concrete triples contained in a void:Dataset is established through access information, such as the address of a sparql endpoint. Last, a void:Linkset is a collection of rdf links whose subject and object are described in di erent datasets. 2.2.3
The Data Catalog Vocabulary (dcat) [
According to the dcat vocabulary, a dcat:Catalog represents a dataset catalog, a dcat:Dataset represents a dataset in the catalog, whereas a dcat:Distribution represents an accessible form of a dataset, e.g., a downloadable le, an rss feed or a Web service that provides the data. dcat considers as a dataset a collection of data, published or curated by a single agent, and available for access or download in one or more formats. This data is considered for generating an rdf dataset. Thus, the generated rdf dataset forms a dcat:Distribution of a certain dcat:Dataset. 2.3
Approaches for tracing PROV & metadata We outline methods for capturing provenance and other metadata information. We identify two approaches that capture provenance and other metadata information inline with the rest rdf data {Explicit Graphs (Section 2.3.3) and Singleton Properties (Section 2.3.2){ and two that trace them independently of the rdf data {rdf Rei cation (Section 2.3.1) and Implicit Graphs(Section 2.3.4). In the following subsections, we discuss in more details alternative approaches for de ning the provenance of the following rdf triple: 1 ex:item10245 ex:weight "2.4"^^xsd:decimal . 2.3.1
The rdf framework considers a vocabulary for describing rdf statements and providing additional information. rdf rei cation is intended for expressing properties such as dates of composition and source information, applied to speci c instances of triples. The conventional use involves describing an rdf triple using four statements. A description of a statement is called a rei cation of the statement. The rdf rei cation vocabulary consists of the type rdf:Statement, and the properties rdf:subject, rdf:predicate and rdf:object. rdf rei cation is the w3c recommended approach for representing provenance and metadata information.
The major disadvantage of rdf rei cation is the number of triples required to represent a rei ed statement. For each generated triple, at least four additional statements is required to be generated. So, for an rdf dataset of N triples, the metadata graph will be equal to four times the number of the rdf dataset triples in the best case where only the rdf rei cation statements are generated and no additional. 2.3.2
Singleton properties [
The Explicit Graphs approach relies on named graphs.
Named Graphs is a set of rdf triples named by a uri and
can be represented using TriG [
Implicit graphs are uris assigned implicitly to a dataset,
graph, triple or term. An Implicit Graph is aware of what it
represents but the represented entity is not directly linked
to its implicit graph. Implicit graphs might be used to
identify a dataset or a graph, but also triples. In the later case,
as Triple Pattern Fragments (tpf) introduced [
Provenance and other metadata information can be captured at di erent steps of the publishing work ow. Keeping track of metadata derived from the di erent steps of the rdf data generation and publishing work ow, results in more complete information regarding how an rdf dataset was generated and formed in the end. Moreover, provenance and metadata information generated at di erent steps of the publishing work ow o er complementary information.
We identify the following primary steps: mapping de nitions generation (Section 3.1), data source retrieval (Section 3.2), rdf data generation (Section 3.3), rdf data publication (Section 3.4). We consider each work ow step as an activity (prov:Activity) whose properties is needed to be traced. In Table 1, we summarize those activities and the information that needs to be de ned each time. The provenance and how the di erent steps are associated with each other are shown at Figure 1.
Same Dataset
Di erent
Dataset
Map. Gen. Pub. Gen. Pub. Link.
prov:Entity prov:wasGeneratedBy prov:wasDerivedFrom prov:wasAttributedTo
prov:Agent prov:actedOnBehalfOf void:Dataset { General
dcterms:creator dcterms:contributor dcterms:publisher dcterms:source dcterms:created dcterms:modi ed dcterms:issued dcterms:license
void:feature void:Dataset { Access void:Dataset { Structural void:Dataset { Statistics void:Linkset #G
#G #G #G #G #G #G #G #G #G #G #G #G #G A lled circle ( ) indicates that the property should be (re-)assessed in each of the marked steps. that property can be assessed in any of the marked steps.
A half- lled circle (G#) indicates that
Provenance and metadata information is required to be captured when the mapping rules are de ned (Fig. 1, Edit Map Doc). In this case, it is important to track when the mapping rules were edited or modi ed and by whom. An rdf dataset might have been generated using multiple mapping rules whose de nition occurred at di erent moments and by di erent agents.
Consequently, the generation of certain mapping rules (Fig. 1, Generate Map Doc) is an activity (prov:Activity) which is informed by all prior editing activities (Fig. 1, Edit Map Doc). For instance, a mapping rule might have been generated by a mapping generator or edited by a human-agent using a mapping editor. However, such a maping rule might have been modi ed or used in conjunction with other mapping rules which were generated, in their own turn, by another human-agent at a di erent time.
The agent who de ned the mapping rules (Fig. 1, Mapping Editor ) might di er from the one who generated (Fig. 1, Data Generator ) or published the data (Fig. 1, Data Publisher ), or even the owner of the data (Fig. 1, Data Owner ). Being aware of who de ned the mapping rules is of crucial importance to assess the trustworthiness of the nal rdf data, even though it is neglected so far. For instance, rdf data generated using mapping rules from an automated generator might be considered less trustworthy compared to rdf data whose mapping rules were de ned by a data specialist. 3.2
An rdf dataset might be derived from one or more heterogeneous data sources (Fig. 1, Data Source Acquisition). Each data source, in its own turn, might be derived from an input.
For instance, a table might be derived from a database or some json data might be derived from a Web api. Such a data source might be turned into an rdf graph partially or in its entirety.
entire stored data is retrieved but a selection is only used to generate the rdf data. For instance, only the data that ful ls an sql query could be retrieved to generate the rdf dataset, instead of the entire table or database.
For this activity, it is important to keep track of metadata regarding the data sources and their retrieval, as this indicates the original data sources of the generated rdf data. However, the originally stored data might have changed over time. For instance, in the case of an api, some data is retrieved at a certain time, but di erent data might be retrieved at a subsequent time.
Therefore, it is crucial to know when the data is accessed to assess its timeliness with the original data. For instance, comparing the last modi ed date of the original data and the generation date of the rdf data, indicates whether the available rdf representation is aligned with the current version of the original data or not. 3.3
As soon as the mapping rules and the data source are available, the rdf data is generated (Fig. 1, Generate RDF Data).
For this activity, it is important to keep track of (i) how the rdf data generation was triggered, i.e. datadriven or mapping-driven, from raw data (rdf generation) or from
rdf data (rdf interlinking); (ii) when the rdf dataset was generated, and (iii) how, i.e. in a single dataset or in subgraphs, subsets etc.
3.4
The published rdf data is not always identical to the generated one (Fig. 1, genRDF Vs. pubRDF ). For instance, it might be the result of merging multiple rdf datasets which are generated from di erent data sources at the same or different moments. Moreover, the published rdf dataset might be published in a di erent way compared to how the rdf data was generated. For instance, it could be split in di erent graphs to facilitate its consumption. This might lead to Data Owner actedOnBehalfOf Data Generator wasAssociatedWith gen RDF Data used
Publish RDF Data actedOnBehalfOf
Data Publisher Start Time startedAtTime Mapping Editor Edit Map Doc actedOnBehalfOf wasAssociatedWith wasInformedBy wasGeneratedBy
Map Doc Generate Map Doc generatedAtTime endedAtTime startedAtTime wasAttributedTo
Generate RDF data used used wasStartedBy wasStartedBy
End Time endedAtTime generated End Time Start Time generatedAtTime
wasDerivedFrom wasDerivedFrom hadPrimarySource wasDerivedFrom non-RDF
Data
generated Stored Data used
pub
RDF Data generatedAtTime generatedAtTime endedAtTime startedAtTime Data Source Acquisition endedAtTime startedAtTime
End Time Start Time End Time Start Time di erent metadata for the generation and publication activities, and these metadata sets might have di erent purposes.
For instance, void access information metadata is more meaningful and possible to be generated during the rdf data publication, whereas provenance information in respect to the original data can only be de ned during the rdf data generation activity. To the contrary, void structural or statistical metadata might be generated both during rdf data generation and publication. However, the generated rdf data is not always identical to the one published. If the generated rdf data di er from the one published, then such metadata should be de ned for both cases (see Table 1).
There are di erent details levels for capturing provenance and metadata information. However, in most cases so far, the provenance and metadata information is delivered on dataset level. This mainly occurs because the metadata information are only de ned after the rdf data is generated and/or published. However, di erent applications and data consumption cases require di erent levels of provenance and metadata information. Overall, the goal is to achieve the best trade-o between details level and number of additional triples generated for balancing information overhead and acceptable information loss in an automated metadata generation occasion. For instance, considering rdf rei cation for capturing all provenance and metadata information for each triple, means that metadata referring to the entire rdf dataset is captured repeatedly for each individual triple. To the contrary, considering an implicit graph on dataset level results in information loss in respect to the origin of each triple, if multiple data sources are used to generate the rdf dataset, because it is not explicitly de ned where each triple is derived from.
Automating the provenance and metadata information generation, allows exploiting hybridic approaches which can contribute in optimizing the metadata information balance. In this section, we outline the di erent details levels for capturing metadata that we identi ed: Dataset level (Section 4.1), named graph level (Section 4.2), partition level (Section 4.3), triple level (Section 4.4) and term level (Section 4.5). For each level, we describe what type of metadata is captured and we discuss the advantages and disadvantages when used in combination with di erent representation approach. 4.1
Dataset level provenance and metadata provide high-level information for the complete rdf dataset. This level of detail is meaningful for all metadata information that refer to the whole dataset, i.e. a void:Dataset and are the same for each triple. Therefore, among the alternative representation approaches, considering an explicit or implicit graph for the dataset to represent provenance and metadata annotations is su cient on dataset level and it requires the least number of additional triples. The alternative approaches in principle assign the same metadata information to each triple. Thus, the exact same information is replicated for each triple, causing unnecessary overhead.
Provenance information on dataset level is su cient if all triples are derived from the same original data source and are generated at the same time, as a result of a single activity. The same occurs if the overall origin source is su cient to assess the rdf dataset trustworthiness. On the contrary, if being aware of the exact data source is required, for instance to align the semantically annotated representation with the original data values, more detailed provenance information is desired, because the high level provenance information is not as complete and accurate to accomplish the desired task. 4.2
An rdf dataset might consist of one or more named graphs. Named graph based subsets of an rdf dataset provide conceptual partitions of rdf triples semanticfully distinguished in graphs. Named graph level provenance and metadata information refer to all rdf annotations which are related to a certain named graph and contain information for each one of the named graphs. Each named graph is a void:Dataset and consists a subset of the whole rdf dataset.
In the case of named graphs, it is not possible to represent metadata and provenance information using explicit graphs, because the rdf statements are already quads and the named graph has di erent semantics than providing metadata information. As in the case of dataset level, implicit graphs for each named graph and for the complete dataset generate the minimum number of additional rdf triples. Moreover, the named graph level metadata information are su cient if all triples of a certain named graph are derived from the same data source. Otherwise, there is information loss which can be addressed at a narrower detail level. 4.3
A dataset might be partitioned based on di erent aspects. The most frequent partitions are related to (i) the underlying data source or the triple's (ii) subject, (iii) predicate, or (iv) object. Besides the aforementioned partitions, any other custom partition can be equally considered. A source-based partitioned rdf dataset is an rdf dataset whose subsets are formed with respect to their derivation source. To be more precise, all rdf terms and triples are derived from the same original data source. Source-based partitioned rdf datasets derived from a single data source are not considered because they coincide with the actual rdf dataset. A subject-based partitioned rdf dataset is the part of an rdf graph whose triples share the same subject. Consequently, subject-level metadata provides information for all triples which share the same subject. It similarly applies in the case of predicatebased or object-based partitions.
Partitioned datasets might be treated in the same way as named graphs, but it is also possible to use explicit graphs to de ne the subsets metadata. An implicit graph for each subset of the rdf dataset which resembles a partition achieves generating the minimum number of additional triples for the metadata information. In the particular case of predicatebased partition, representing the provenance and metadata information using singleton properties would cause generating almost the same number of additional triples as in the case of de ning an explicit or implicit graph per partition. 4.4
If metadata is captured on triple level, it becomes possible to keep track of the data source each triple was derived from. However, that causes the generation of rdf annotations for metadata whose number of triples is larger than the actual dataset. In the simplest case, the number of additional triples for the metadata information depends on the number of data sources. The more data sources, the more metadata information to be de ned. Triple level metadata become meaningful also in the case of big data or streamed data where the time one triple was generated might signi cantly di er compared to the rest triples of the rdf dataset.
In the case of triple level metadata, singleton properties become meaningful when statements about all triples sharing the same property share the same metadata information. For instance, if all triples whose rdf terms are associated using a certain predicate, share the same metadata, e.g., they are all derived from the same data source. 4.5
Even rdf terms that are part of a certain rdf triple can derive from di erent data sources. For instance, an rdf term is generated considering some data value derived from a source A. This rdf term might constitute the subject of an rdf triple whose object though is an rdf term derived from a source B. In this case, even more detailed metadata information is required to keep track of the provenance information. Among the alternative approaches for representing metadata, the rdf rei cation becomes meaningful at this level of detail. To be more precise, the rdf rei cation is meaningful in the cases that the rdf terms that consist an rdf triple and form a statement derive from di erent data sources and/or are generated at a di erent time. 5.
METADATA GENERATION WITH RML We introduce an approach that takes into consideration machine interpretable descriptions of data sources and mapping rules, which are used to generate rdf datasets, to also automatically generate its corresponding provenance and metadata information. Our approach relies on asserting statements from declarative descriptions of data sources and mapping rules. This allows our proposed approach to be applied on alternative mapping languages and be replicated in di erent implementations.
In our exemplary case, machine interpretable mapping
rules are de ned using the rdf Mapping Language (rml) [
Mapping rules are de ned using the rdf Mapping
Language (rml). rml [
rml documents contain rules de ning how the input data can be represented in rdf. An rml document (see Listing 1) contains one or more Triples Maps (line 5 and 13). A Triples Map de nes how triples are generated and consists of three main parts: the Logical Source, the Subject Map and zero or more Predicate-Object Maps. The Subject Map (line 6 and 14) de nes how unique identi ers (uris) are generated for the resources and is used as the subject of all rdf triples generated from this Triples Map. A Predicate-Object Map (line 7 and 15) consists of Predicate Maps, which de ne the rule that generates the triple's predicate (line 9, 17 and 19) and Object Maps (line 18 and 20) or Referencing Object Maps (line 10), which de ne how the triple's object is generated. <http://www.w3.org/ns/r2rml#>. <http://semweb.mmlab.be/ns/rml#>.
<http://xmlns.com/foaf/0.1/>. <#PersonMap> rml:logicalSource <#DCAT_LogicalSource> ; rr:subjectMap <#PersonSubjectMap>; rr:predicateObjectMap <#AccountPreObjMap>. <#PersonSubjectMap> rr:template "http://ex.com/{ID}". <#AccountPreObjMap> rr:predicate foaf:account;
rr:objectMap <#TwitterRefObjMap>.
<#TwitterRefObjMap> rr:parentTriplesMap <#TwitterAcount>.
A mapping document summarizes mapping rules de ned using the rml language. rml is serialized in rdf, thus a mapping document (<#MapDoc>) can be considered as an rdf dataset itself (void:Dataset). Therefore, it has its own metadata as any other rdf data can have. precise, a mapping document is a prov:Entity that can be associated with a prov:Agent, either a human agent or software. The Mapping Document is the result of a prov:Activity, which is informed, on its own turn, from di erent editing activities. @prefix dcterms: <http://purl.org/dc/terms/>. @prefix prov: <http://www.w3.org/ns/prov#>. @prefix void: <http://rdfs.org/ns/void#>. <#MapDoc> a prov:Entity, void:Dataset; prov:generatedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime; prov:wasGeneratedBy <#MapDoc_Generation>; prov:wasAssociatedWith <#RMLEditor>; prov:wasAttributedTo <http://rml.io/people/AnastasiaDimou>; dcterms:creator <http://rml.io/people/AnastasiaDimou>; dcterms:created "2016-01-05T17:10:00Z"^^xsd:dateTime; dcterms:modified "2016-01-05T17:15:00Z"^^xsd:dateTime; dcterms:issued "2016-01-07T10:10:00Z"^^xsd:dateTime. <#MapDoc_Editing> prov:startedAtTime prov:endedAtTime a prov:Activity; "2016-01-05T17:00:00Z"^^xsd:dateTime; "2016-01-05T17:10:00Z"^^xsd:dateTime . <#MapDoc_Generation> a prov:Activity; prov:generated <#MapDoc>; prov:startedAtTime "2016-01-05T17:09:00Z"^^xsd:dateTime; prov:endedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime; prov:wasInformedBy <#MapDoc_Editing>. <#RMLEditor> a prov:Agent; prov:type prov:SoftwareAgent. <http://rml.io/people/AnastasiaDimou> a prov:Agent; prov:type prov:Person; prov:actedOnBehalfOf <#DataOwner>. 5.3
One or more data sources might be considered for generating an rdf dataset. In our exemplary case, one data source is described by the <#DB LogicalSource> and the underlying database that contains the data is described by the <#DB Source> using the d2r vocabulary. Its description: @prefix rml: <http://semweb.mmlab.be/ns/rml#>. @prefix dcat: <http://www.w3.org/ns/dcat#>. @prefix d2rq: <http://www.wiwiss.fu-berlin.de/suhl/bizer/D2RQ/0.1#>. <#DB_LogicalSource> rml:logicalSource [ rml:query """SELECT * FROM DEPT WHERE ... """ ; rml:source <#DB_Source> ]. <#DB_Source> a d2rq:Database; d2rq:jdbcDSN "jdbc:mysql://localhost/example"; d2rq:jdbcDriver "com.mysql.jdbc.Driver"; d2rq:username "user"; d2rq:password "password". <#MapDoc> void:subset <#PersonMap>, <#TwitterAccountMap>. <#PersonMap> a prov:Entity, void:Dataset; prov:wasGeneratedBy <#PersonMap_Generation>; prov:wasAssociatedWith <#RMLEditor>; prov:wasAttributedTo <http://rml.io/people/AnastasiaDimou>. <#TwitterAccountMap> a prov:Entity, void:Dataset; prov:wasGeneratedBy <#TwitterAccountMap_Generation>; prov:wasAssociatedWith <#RMLEditor>; prov:wasAttributedTo <http://rml.io/people/AnastasiaDimou>. <#PersonMap_Generation> a prov:Activity; prov:generated <#PersonMap>; prov:wasInformedBy <#PersonMap_Editing>. <#TwitterAccountMap_Generation> a prov:Activity; prov:generated <#TwitterAccountMap>; prov:wasInformedBy <#TwitterAccountMap_Editing>. <#PersonMap_Editing> a prov:Activity. <#TwitterAccountMap_Editing> a prov:Activity.
Listing 3: Triples Map Metadata Description Listing 4: Database Source description
Similarly, a data source might be a dcat:Dataset and one of its distributions might be considered for generating the rdf dataset. Directly downloadable distributions contain a dcat:downloadURL reference. For instance: @prefix rml: <http://semweb.mmlab.be/ns/rml#>. @prefix dcat: <http://www.w3.org/ns/dcat#>. <#DCAT_LogicalSource> rml:source <#DCAT_Source>; rml:referenceFormulation ql:XPath; rml:iterator "...". <#DCAT_Source> a dcat:Dataset;
dcat:distribution <#XML_Distribution> . <#XML_Distribution> a dcat:Distribution;
dcat:downloadURL <http://ex.org/file.xml>.
Listing 5: DCAT source description
The data source retrieval can be considered as a prov:Activity attributed to a prov:Agent. Such a prov:Agent can be the data owner or an agent acting on his behalf, i.e. Data Generator. The data source consists of a prov:Entity which was derived from the data acquisition activity. The original data source description provides some information regarding the data source. Additional, provenance information is added for further clarity. The metadata for a data source, e.g., the <#DB LogicalSource> and the <#DCAT LogicalSource>, are described as follows: @prefix prov: <http://www.w3.org/ns/prov#>. <#DB_LogicalSource> a prov:Entity; prov:wasDerivedFrom <#DB_Source> ; prov:generatedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime . <#DB_Retrieval> a prov:Activity; prov:generated <#DB_LogicalSource> ; prov:used <#DB_Source>; prov:startedAtTime "2016-01-05T17:00:00Z"^^xsd:dateTime ; prov:endedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime . <#DCAT_LogicalSource> a prov:Entity; prov:generatedAtTime "2016-01-05T17:05:00Z"^^xsd:dateTime . <#DCATsource_Retrieval> a prov:Activity; prov:generated <#DCAT_LogicalSource>; prov:used <#DCAT_Source>.
Considering the aforementioned mapping document and the data source descriptions, rdf triples are generated.
Among the di erent approaches for capturing provenance and metadata information, rml can best be aligned with implicit graphs and rdf rei cation. Those two approaches generate metadata information independently of the generated rdf data. In particular, explicit graphs are not considered as they might coincide with the named graphs if any explicitly de ned for the actual rdf dataset.
Dataset level metadata information is associated with all triples generated considering all mapping rules in a mapping document. Named graph level metadata information is associated with triples which are generated considering Term Maps related to the corresponding Graph Map.
As far as partitioned rdf datasets is concerned, each partition is associated with di erent parts of one or more Triples Maps. To be more precise, source-level metadata information is generated for all triples which are derived from Triples Maps which share the same Logical Source. In the same context, subject-level metadata information is generated for each unique instantiation of one (or more) of the Subject Maps that appear in the mapping document. Similarly, predicate-level metadata information is generated for each unique predicate which appears in one or more Triples Maps. Last, objectlevel metadata anotations are generated for each unique object which is generated due to an Object Map. Whereas the aforementioned levels consider implicit graphs to represent provenance and metadata information, triple and RDF term level metadata information can only be captured considering rei cation statements.
Dataset level provenance and metadata information has as follows for the aforementioned running example: <#RDF_Dataset> a prov:Entity, void:Dataset; prov:generatedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime; prov:wasGeneratedBy <#RDFdataset_Generation>; prov:wasDerivedFrom <#DB_LogicalSource>,<#DCAT_LogicalSource>; prov:wasAssociatedWith <#RMLProcessor>; prov:wasAttributedTo <http://rml.io/people/AnastasiaDimou>; dcterms:creator <http://rml.io/people/AnastasiaDimou>; dcterms:created "2016-01-05T17:10:00Z"^^xsd:dateTime; dcterms:modified "2016-01-05T17:12:00Z"^^xsd:dateTime; dcterms:issued "2016-01-07T10:10:00Z"^^xsd:dateTime. <#RDFdataset_Generation> a prov:Activity; prov:generated <#RDF_Dataset>; prov:startedAtTime "2016-01-05T17:00:00Z"^^xsd:dateTime; prov:endedAtTime "2016-01-05T17:10:00Z"^^xsd:dateTime; prov:wasInformedBy <#MapDoc_Generation>; prov:used <#MapDoc>,<#DB_LogicalSource>,<#DCAT_LogicalSource>. <#RMLProcessor> a prov:Agent; prov:type prov:SoftwareAgent.
Listing 7: RDF Dataset Level Metadata
The rdf data generation might be triggered either by a
mapping document (mapping-driven approach ) or by a data
source (data-driven approach ) [
Specifying the data source where the rdf dataset was derived from becomes easy and can be automatically asserted thanks to the aligned mapping and data source descriptions. The rml mapping rules declaratively de ne the data sources used, in contrast to other mapping languages which do not explicitly de ne the data sources considered for ful lling the mapping activity.
However, as one can observe, it is de ned that the rdf dataset was derived from an extract of data from a database and an xml le published on the Web, but it is not explicitly de ned which triples are derived from each data source.
In order to address the aforementioned ambiguity regarding the rdf triples origin, rml metadata generation might be de ned based on the Predicate Object Maps, for instance the <#AccountPreObjMap> and the <#HomepagePreObjMap>. The metadata information of the generated rdf triples follows:
Even though it is easy one to observe that this resolves the ambiguity issue regarding the provenance in the case of triples generated considering the <#HomepagePreObjMap>, it is not the same in the case of triples generated considering the <#AccountPreObjMap>. In the later case, the rdf triples are formed generating the subject and the object from di erent data sources. To be more precise, the triple's subject is generated considering a value derived from the <#DCAT LogicalSource>, whereas the triple's object is derived from the <#DB LogicalSource>. If, even more detailed provenance is required, rdf term level should be preferred.
The rdf term level is the narrowest details level for metadata information. It is applicable in the cases of Referencing Object Maps, namely when the subject and the object of an rdf triple is derived from di erent data sources. rdf rei cation is the only approach for representing this metadata information. Considering the aforementioned running example, the metadata information of the rdf triples generated from the <#AccountPreObjMap> Predicate Object Map are dened as it follows: 1 _:ex12345 rdf:type 2 _:ex12345 rdf:subject 3 _:ex12345 rdf:predicate 4 _:ex12345 rdf:object 5 6 ex:item10245 prov:wasDerivedFrom <#DCAT_LogicalSource>. 7 <http://twitter.com/natadimou> 8 prov:wasDerivedFrom <DB_LogicalSource>.
In the case that the original data is published on the Web in the frame of a catalogue described with the dcat vocabulary, complementary metadata information can be generated to enrich it. If one of the dcat:Dataset distributions (dcat:Distribution) is considered to generate the corresponding rdf representation, complementary dcat metadata might be generated as well, to specify that the generated rdf dataset is another distribution of a certain dcat:Dataset published on the dcat:Catalog. For instance, in the case of the running example, the <#DCAT RDF> and the <#DB RDF> are source-level partitions of the rdf dataset. The rdf triples in the <#DCAT RDF> partition is an rdf distribution of the <#XML Distribution> for the <#DCAT Source>. The <#DCAT RDF> metadata information and the enriched <#DCAT Source> have as follows: 1 @prefix dcat: <http://www.w3.org/ns/dcat#>. 2 @prefix prov: <http://www.w3.org/ns/prov#>. 3 @prefix void: <http://rdfs.org/ns/void#>. 4 5 <#RDF_Dataset> void:subset <#DCAT_RDF>, <#DCAT_RDF>. 6 7 <#DCAT_RDF> a prov:Entity, void:Dataset; 8 prov:wasGeneratedBy <#RDFdataset_Generation>; 9 prov:wasDerivedFrom <#DCAT_LogicalSource>; 10 prov:wasAssociatedWith <#RMLProcessor>; 11 prov:wasAttributedTo <http://rml.io/people/AnastasiaDimou>. 12 13 <#DCAT_Source> dcat:distribution <#DCAT_RDF>.
Listing 10: DCAT Metadata Enrichment
METADATA & THE RML TOOL CHAIN We implemented the aforementioned approach at the rml tool chain, namely the rmleditor15 and the rmlprocessor16. The rml tool chain was con gured to support implicit graphs and rdf rei cation. The supported metadata can be further extended to take into consideration other metadata vocabularies too and generate corresponding metadata information. In more details:
The rmleditor [
The rmlprocessor has also been extended to automatically generate corresponding metadata regarding rdf dataset generation. It was extended to generate metadata information considering implicit graphs or rdf rei cation. To be more precise, the rmlprocessor was con gured to generate metadata information using implicit graphs for the metadata which are related with the whole dataset as well as for named graphs. The rmlprocessor was con gured to generate rdf rei cation triples if the metadata level is set on triple or rdf term level. The explicit graphs and singleton properties were not considered because they need to be de ned inline with the actual rdf data, together with the mapping rules. 7.
CONCLUSIONS AND FUTURE WORK The proposed approach aims to show how metadata of the fundamental activities for the generation and publication of rdf triples can be automatically generated. Our solution covers the rdf dataset generation, including metadata for the mapping rules de nition and the data descriptions. Based on the provided metadata information, it is expected that publishing infrastructures will enrich this information with complementary details regarding the rdf dataset publication activity. Moreover, it is expected that rdf publication infrastructures will re-determine certain properties regarding the metadata information, if the rdf dataset is reformed before it gets published. Moreover, the metadata can be enriched with additional information derived from other activities involved in the Linked Data publishing work ow.
Provenanve and metadata information can be
multidimensional and its consumption diverges across di erent systems.
Di erent applications require di erent levels of metadata
information to ful l their tasks, whereas diverse metadata
information might be desired. The presented work ow was
focused on the essential parts of the Linked Data
publishing work ow. However, any metadata information might be
considered as they accrue from other activities involved in
the Linked Data publishing work ow. In the future, we
consider including metadata information regarding the results
of the rdf validation [
Di erent aspects of the rdf data generation and publishing might in uence its quality and trustworthiness assessment. In most of the cases so far, the provenance and metadata information are manually delivered on dataset level. Automating the provenance and metadata generation relying on machine interpretable descriptions of the di erent work ow steps, allows to generate metadata in a systematic way. The generated provenance and metadata information becomes more accurate, consistent and complete. The metadata generation for certain rdf data is an incremental procedure that relies on the contribution of di erent activities in the Linked Data publishing work ow to enrich the information we have for the generated rdf dataset.
The described research activities were funded by Ghent University, iMinds, the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT), the Fund for Scienti c Research Flanders (FWO Flanders), and the European Union.