Existing knowledge graph construction mapping languages have a legacy of mapping over relational databases. A such, current mapping language join constructs conflate securing referential integrity with relating concepts across data sources. This leads to a significant amount of operations (resulting in performance bottlenecks), and loss of additional context of potential linking triples that are not generated due to a lack of referential integrity. We propose a reference condition next to the traditional join condition, allowing to express a relation between sources without imposing any referential integrity. In this short research paper, we describe the concept, its applicability, how it could be integrated in existing and future mapping languages, and a proof-of-concept implementation. Our evaluation based on GTFS-Madrid-Bench confirms the assumption that removing these integrity checks leads to much faster generation times, but we also find that using reference conditions results in exactly the same graph output, i.e. these alternative semantics do not influence generation results for cases where referential integrity is assumed within the source system. Adding the reference conditions keeps the best of all worlds: generation time is shortened where possible, you have more context in the resulting RDF graph, and the mapping file still supplies relevant metadata about relations between triples maps. For future work, we will further research relations between triples maps, and expand our implementation to more complex reference conditions. This will allow us to investigate similar performance gains with other benchmarks and other mapping engines.
Knowledge graph construction has been largely simplified by using established mapping languages and well-maintained mapping engines. The increasing maturity of this field is exempliifed by the activities of W3C’s Knowledge Graph Construction Community Group, and recent research detailing optimization strategies implemented in mapping engines.
Existing mapping languages historically extended W3C’s recommended Relational to RDF
Mapping Language (R2RML) [
As R2RML aimed at mapping data from relational databases, the expression of a relation between mapping rules in (R2)RML (by a referencing object map) is represented as an operation that secures referential integrity in the resulting knowledge graph (by imposing join conditions). Hence, the concept of securing referential integrity (which is the responsibility of data sources) and linking concepts (which is the responsibility of mapping languages) got conflated.
This conflation has a limiting efect on the usability of relations in mapping files. Join
conditions typically result in performance bottlenecks which in turn require joining optimization
strategies such as: use predicate-join index tables [
In this short research paper we argue that the default link between join conditions and the expression of relations between data sources is too restrictive for the task at hand, namely, the generation of RDF graphs.
We propose a reference condition next to the traditional join condition, allowing to express
a relation between sources without imposing any referential integrity. We provide a
proofof-concept implementation that replaces the join conditions which do not rely on external
information by reference conditions, and can be used as a first step of the knowledge graph
generation process by any (R2)RML engine. We evaluate this proof-of-concept implementation
and show the positive efects on the generation time when replacing join conditions by reference
conditions when this is possible. This change in semantics resulted in exactly the same output
when executing GTFS-Madrid-Bench [
After presenting the efects of conflating referential integrity checks with linking concepts during mapping (section 2), we describe the reference condition semantics and implementation (section 3), evaluate (section 4), and conclude (section 5).
After describing the benefits of being able to specify relations between triples maps (section 2.1), we explain the consequences of their conflation with referential integrity (section 2.2) and the efects of using an alternative mapping construction without explicit relations (section 2.3).
The referencing object map is the only construction in (R2)RML that explicitly expresses the
relation between triples maps. It allows using the subjects of another triples map (the parent
map) as the objects generated by a predicate-object map (the child map)1. As a running example
we use a part of the widely used GTFS-Madrid-Bench [
Expressing such relations between triples maps (i) provides relevant metadata and (ii) increases
the consistency and maintainability of the mapping file. (i) The relation between triples maps
is relevant metadata for the generation of documentation about the related knowledge graph,
e.g. deducting a richer SHACL shape from an (R2)RML mapping file [
According to the (R2)RML specification at least one join condition must be added to the
referencing object map if the logical sources of the parent map and the child map are not identical3.
Due to this join condition the expression of relating triples maps in (R2)RML conflates with
securing referential integrity in the knowledge graph. This conflation comes with following
disadvantages: (i) a potential loss of knowledge and (ii) a large performance hit. (i) The output
of the running example (fig. 2) is limited to seven triples. Although the id of agency 3 is available
in the R O U T E S . c s v , the triple expressing the agency of route 15 (fig. 3) will not be generated.
However, the missing triple can contain valuable information and can be used to solve queries
which do not require additional agency information: generating links between resources, even
if you do not have any additional information about them, aligns with the original vision of
3https://www.w3.org/TR/r2rml/#dfn-joint-sql-query
1 @ p r e f i x e x : < h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / > .
2 e x : m e t r o / r o u t e s / 1 d a g t f s : R o u t e ; g t f s : a g e n c y e x : a g e n c y / 4 .
3 e x : m e t r o / r o u t e s / 1 4 a g t f s : R o u t e ; g t f s : a g e n c y e x : a g e n c y / 5 .
4 e x : m e t r o / r o u t e s / 1 5 a g t f s : R o u t e .
5 e x : a g e n c y / 4 a g t f s : A g e n c y .
6 e x : a g e n c y / 5 a g t f s : A g e n c y .
1 < # r o u t e s > a r r : T r i p l e s M a p ;
2 r m l : l o g i c a l S o u r c e [ r m l : s o u r c e " R O U T E S . c s v " ; r m l : r e f e r e n c e F o r m u l a t i o n q l : C S V ] ;
3 r r : s u b j e c t M a p [ r r : t e m p l a t e " h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / m e t r o / r o u t e s / { i d } " ; r r : c l a s s g t f s : R o u t e ] ;
4 r r : p r e d i c a t e O b j e c t M a p [
5 r r : p r e d i c a t e g t f s : a g e n c y ;
6 r r : o b j e c t M a p [ r r : t e m p l a t e " h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / a g e n c y / { a g e n c y _ i d } ] ] .
7 < # a g e n c y > a r r : T r i p l e s M a p ;
8 r m l : l o g i c a l S o u r c e [ r m l : s o u r c e " A G E N C Y . j s o n " ; r m l : r e f e r e n c e F o r m u l a t i o n q l : J S O N P a t h ; r m l : i t e r a t o r " $ . [ * ] " ] ;
9 r r : s u b j e c t M a p [ r r : t e m p l a t e " h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / a g e n c y / { i d } " ; r r : c l a s s g t f s : A g e n c y ] .
10 < # a g e n c y 2 > a r r : T r i p l e s M a p ;
11 r m l : l o g i c a l S o u r c e [ r m l : s o u r c e " R O U T E S . c s v " ; r m l : r e f e r e n c e F o r m u l a t i o n q l : C S V ] ;
12 r r : s u b j e c t M a p [ r r : t e m p l a t e " h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / a g e n c y / { a g e n c y _ i d } " ; r r : c l a s s g t f s : A g e n c y ] .
designing RDF (”Anyone Can Make Statements About Any Resource”4). Use cases illustrating
the value of those missing triples are: mapping sample data, at the beginning of a project or for
testing a mapping file during the development process, and mapping data streams, where not all
data is simultaneously available. (ii) The performance hit when joining and checking referential
integrity is substantial, and is one of the reasons why the referential integrity constraint is
typically dropped in data warehouses [
Another way to relate triples is via carefully crafted URI templates (fig. 4): as long as the URI templates are consistent between an object map of a first triples map and a subject map of a second triples map, the nodes will match and triples will be related. This way of mapping related triples does not involve any referential integrity checking and avoids performance bottlenecks. Additionally, the notation is less verbose. However, the synchronization of the mapping rules becomes a manual process, and to get similar relevant metadata from the mapping file as with a referencing object map, an additional triples map specifying the class of the object of the 4https://www.w3.org/TR/rdf-concepts/#section-anyone ifrst triples map is needed (triples map < # a g e n c y 2 > ). Consequently, the manual synchronization eforts increase and the notation is again more verbose.
We introduce a reference condition for a referencing object map as alternative to the traditional join condition. A reference condition allows the use of a subject map of another logical source without imposing any referential integrity: the object of the child map is generated following the structure of the subject of the parent map without verifying or using data from the parent source. After describing its scope (section 3.1), we propose a language construct (section 3.2) and a proof-of-concept implementation (section 3.3). 3.1. Scope Not every join condition can be re-interpreted as a reference condition. We identify two cases for which join conditions remain needed: (i) data from the parent source is needed to build the child object URI, or (ii) the parent source is a selective dataset. Referring back to our running example, case (i) applies if the template of an agency uses the agency name: “ h t t p : / / t r a n s p o r t . l d . e s / m a d r i d / a g e n c y / { a g e n c y _ n a m e } ” . Case (ii) applies when the parent source A G E N C Y . j s o n is a selective dataset, e.g. there are two kinds of agencies, listed in two logical sources with each their own triples map and URI template. In that case the triple of fig. 3 would be wrong if agency 3 does not appear in AGENCY.json (fig. 1c).
The introduction of a reference condition (fig. 5) would require following adaptations to the (R2)RML specifications (emphasizing our proposed changes in bold) 5. (i) A referencing object map allows using the subject maps of another triples map for generating the objects of a predicate-object map. (ii) If the logical source of the child triples map and the logical source of the parent triples map of a referencing object map are not identical, then the referencing object map must have at least one condition, where all conditions are either reference conditions or join conditions. (iii) The joint SQL query of a referencing object map is: [...] If the referencing object map has at least one reference condition: S E L E C T c h i l d - c o l u m n 1 A S p a r e n t - c o l u m n 1 , c h i l d - c o l u m n 2 A S p a r e n t - c o l u m n 2 , . . . F R O M ( { c h i l d - q u e r y } ) A S t m p .
If a reference condition is used and the URI template requires data from the parent source (section 3.1 case (i)), the mapping engine should throw an error, similar to errors thrown when an unknown reference is used in the mapping file (e.g. test case R2RMLTC0002c 6).
Reference conditions enable the deduction of crafted URI templates, which can be handled by default by any (R2)RML mapping engine. This allows introducing reference conditions without 5https://rml.io/specs/rml/#logical-join and https://www.w3.org/2001/sw/rdb2rdf/r2rml/#foreign-key 6https://www.w3.org/2001/sw/rdb2rdf/test-cases/#R2RMLTC0002c 1 <#routes> a rr:TriplesMap ; 2 rml:logicalSource [rml:source "ROUTES.csv";rml:referenceFormulation ql:CSV]; 3 rr:subjectMap [rr:template "http://transport.ld.es/madrid/metro/routes/{id}";rr:class gtfs:Route]; 4 rr:predicateObjectMap [ 5 rr:predicate gtfs:agency; 6 rr:objectMap [rr:parentTriplesMap <#agency>;rr:refCondition [rr:child "agency_id";rr:parent "id"]] 7 ]. 8 <#agency> a rr:TriplesMap; 9 rml:logicalSource [rml:source "AGENCY.json";rml:referenceFormulation ql:JSONPath;rml:iterator "$.[*]"]; 10 rr:subjectMap [rr:template "http://transport.ld.es/madrid/agency/{id}";rr:class gtfs:Agency]. adding any overhead to the existing (R2)RML mapping engines. We published a proof-of-concept implementation converting referencing object maps with reference conditions to crafted URIs at https://github.com/RMLio/rml-loose-generator.
At this moment our implementation interprets every referencing object map that does not need data from the parent source with reference conditions instead of join conditions (i.e. it does not take case (ii) as described in section 3.1 into account) and is limited to referencing object maps with only one join condition, without use of functions, and where the subject of the parent is built with a template. Outside of these restrictions, the original semantics of join conditions are preserved.
We tested the efect of using reference conditions versus join conditions on
GTFS-MadridBench [
We tested these mapping engines for scales 1, 5, 50, and 100 (table 1), using a device with following specifications: 2 x Hexacore Intel E5645 (2.4GHz) CPU, 24GB RAM, 1x 250GB harddisk. RMLMapper and RMLStreamer cannot generate any output for the GTFS-Madrid-Bench within one hour when using join conditions. Using reference conditions, however, these mapping engines were able to generate correct output, with timings similar to using a state-of-the-art mapping engine like Morph-KGC. We note that RMLMapper cannot handle GTFS scales 50 and 100: the RMLMapper loads all data in memory during mapping, and the testing device ran out of memory during GTFS scales 50 and 100.
When comparing the resulting knowledge graphs from GTFS-Madrid-Bench we found no diference between the version generated with join conditions and the one generated with 7https://github.com/RMLio/rmlmapper-java, https://doi.org/10.5281/zenodo.3887065, and https://doi.org/10.5281/ zenodo.5543552, respectively. reference conditions. In hindsight, this makes sense: GTFS-Madrid-Bench’s testing data is also used to fill a relational database, where referential integrity is needed. We conclude that data joined in the GTFS-Madrid-Bench has perfect referential integrity and that none of the join conditions in the GTFS-Madrid-Bench mapping file impose any limitation to replacing it by a reference condition.
The referencing object map construct from (R2)RML implies that the resulting RDF graph must be referential integer, limiting the usability of relations in mapping files. Meanwhile, in use cases where referential integrity is needed, the data sources typically already validate this integrity by design (e.g. in the case of relational databases). As such, referential integrity checks during the mapping process can become redundant. For other use cases, e.g., mapping data sources that are not under the same governance, or mapping incomplete data (e.g., sample data or streaming data), dropping the notion of referential integrity can improve the completeness of the resulting knowledge graph.
With this paper we introduce reference conditions, an additional condition type that enables the expression of relations between mapping rules without joining the related data sources and imposing referential integrity on the resulting triples. With a very limited overhead, the use of reference conditions can accomplish an instant improvement for any RML or R2RML mapping engine on knowledge graph generation time, while the mapping file still remains consistent and well maintainable, and supplies the valuable metadata which can be deducted from expressed relations. We argue that, unless data from the parent source is needed to build the child object URI, or the parent source is a selective dataset, a reference condition should be preferred over a join condition because this construction fits the Open World Assumption supported by RDF.
Due to the completely correct generation within the GTFS-Madrid-Bench, we believe that many use cases can benefit from this additional condition type, embracing the Open World Assumption followed by performance gain without any significant side-efects.
For future work, we will research how to further optimize reference conditions. First, our current implementation only takes equality operators into account, however, we can expand this to other types of reference conditions (e.g. comparing the lowercased values between sources) by transforming the child reference. Second, we will verify if we can achieve similar performance gains with other benchmarks and other mapping engines.
The described research activities were supported by SolidLab Vlaanderen (Flemish Government, EWI and RRF project VV023/10) and the imec ICON project AI4Foodlogistics (Agentschap Innoveren en Ondernemen project nr. HBC.2020.3097). The authors want to thank the KGCW reviewers for their constructive feedback that helped improve the paper.