In this paper, we investigate the challenge of integrating heterogeneous, distributed data sources using semantic technologies to enable the answering of complex user queries on the combination of available data. Our solution uses a common OWL ontology that contains the concepts that represent the data in all the distributed sources providing maximum querying flexibility towards the user. Retrieval of specific data from distributed sources is done using a set of customized rules and built-in functions that call the data sources APIs triggered when a user issues a SPARQL query. We validated our system with forward and backward ruling strategies in a horticultural application domain. The most important lessons learned are that the approach is very flexible in the sense that only a few customized rules enable the querying of a large set of concepts. Using backwardreasoning rules largely improves the performance of the system, because the reasoner only executes those rules that it deems necessary to answer the user query.
In various business domains and sectors in society, data sources are distributed and heterogeneous by nature, because they come into existence separately and independently. As a consequence, combining and analyzing data from these heterogeneous, distributed data sources is a challenge. We have developed and studied a semantic technology solution that assumes that data is kept at distributed data sources and is only retrieved when needed to answer a user query. Our solution uses as a key instrument a common OWL ontology that contains the concepts that represent the data used from the distributed sources. This common ontology is made available via a triple store with a SPARQL engine that provides maximum querying flexibility on all the concepts in the common ontology. Retrieval of specific data from distributed sources is done using a set of customized rules defined in terms of the concepts and relations in the ontology as well as built-in functions that call the data sources APIs. The execution of these rules is triggered by a reasoner when a user issues a SPARQL query on the ontology and necessary data needs to be retrieved. In this paper, we discuss the validation of our system in an application domain with the usage of forward and backward ruling strategies for retrieving data using the Apache Jena’s generic rule reasoner.
This paper continues the research described in our previous work [
For our research we use Apache Jena’s Generic Rule Reasoner, because it integrates nicely with semantic technologies, like RDF and SPARQL. The rule reasoner uses RDF triples as its facts and also the syntax of the rules is tailored to working with these triples: [mortality-rule: (?x rdf:type :Man) -> (?x rdf:type :Mortal) ]. This reasoner supports both the forward data-driven and backward goal-driven ruling mechanisms, but also another important feature called built-in functions. Built-in functions can be included in the head or body of rules to perform actions that do not easily fit the rulebased paradigm, such as calculations or string manipulation. For example, in the rule [rule:(?n rdf:type :Man)(?n :cmLength ?o) -> CmInch(?o ?p)(?n :inchLength ?p) ] the built-in function CmInch(?o ?p) handles the conversion from cm to inch, where ?o is bound to a Man’s length in cm, and the ?p will be bound by the built-in function to the converted length of the Man in inches. Apache Jena’s rule reasoner keeps a register of standard built-in functions and allows developers to register custom built-in functions that can be used to realize the link with external data sources.
We have used the horticultural application domain to validate our solution as there
are many stakeholders that maintain data about food products that is however not
transparently available to all stakeholders in the supply chain. Therefore, we developed the
HortiCube platform that contains our solution using the Common Horticultural Model
(CHM) described in [
The body of the rule checks whether there are Comtrade individuals of countries ?exporting and ?importing, an apple ?product and a ?year in the CHM. In addition, the body of the rule contains a built-in function comtradeExport that takes these and returns the specific ?export value as result by executing an HTTPS-call to the Comtrade API. The head of the rule is meant to create a comtrade:Export individual in the CHM and add the retrieved ?export value to it as well as the ?exportingcountry, ?importingcountry, ?product and ?year for which the export value was requested.
In the forward chaining, data-driven modus the rule reasoner executes the rule for each combination of triples found that satisfy the body, which can lead to unacceptable execution times. Therefore, we also used the backward chaining, goal-driven modus by simply swapping the head and body of the rule. In that case, the reasoner will only try to prove the overall goal, based on the restrictions in the SPARQL query, which yields an enormous performance improvement. An important restriction of the Apache Jena rule reasoner is that it allows not more than one clause in the head. As a consequence, we split the backward-rule into six rules, one for each of the goals in the head.
Finally, we evaluated our system with respect to the design of the SPARQL query. We found out that the sequence of execution of the backward-rules by the reasoner very much depends on the order of the statements in the query. It turned out that using a “filter-as-soon-as-possible” policy for the ordering of the statements is best. In this way, the reasoner is instructed to first reduce the solution space by filtering on specific values in the triples. The other statements are then used to construct the result of the query. 4
We have learned various lessons with our experiments. First, we found that in some scenarios the goal-driven modus of the reasoner is preferable over the data-driven modus. In our test scenario, the goal-driven modus only called the external data source once, while the data-driven modus exhaustively requested information from the external data source for every combination of possibilities of data in the body of the rules. The performance difference between the two modi changes depending on both the size of the answer and the data. Second, the ordering of the clauses in the SPARQL query can have a considerable impact on query answering time, because it filtering and scoping of query results should be done first in the query. A similar behavior occurs with the order of the clauses in rules, where the location of the built-in can cause incorrectly derived triples. Both the user and the designer should be aware to correctly order the clauses in queries and rules. Third, designing a system like we describe in this paper consists of creating the following artifacts: (1) multiple ontologies, (2) mappings between the ontologies, (3) different rules to integrate the external data sources, (4) custom built-ins for each external data source. Designing such a system requires effort, but enables generic, flexible access to distributed data sources.
Although our solution efficiently and effectively operates, there are some possibilities for future work, such as cascading of OWL and rule-reasoners, aggregation of builtin calls, backward ruling with multiple heads, missing fact indications and parallelism in the reasoning process.