This paper describes research-in-progress work on data integration s ystem in Linked Open Data space. Proposed system uses concept of RDF identity links to interlink heterogeneous local data sources and integrate them into global data space. Academic supervisor: Vladimir Serebriakov, serebr@ccas.ru.
For the last few decades data integration has been one of the most actual problems of computer science. With the development of IT industry countless data sources emerged in the Internet. These data sources are heterogeneous in all possible ways. Effective usage of such data sources is impossible without automatical tools for data search, retrieval, publishing and transformation.
The original hypertextual web didn’t suit well for automatical processing of data from heterogeneous sources spread across the web. This led to emergence of various microformats and web APIs, and finally the concept of Semantic Web. Sematic Web implies usage of standard stack of data formats and technologies intended to support data accumulation, structuring and exchange across the web. The most important of these technologies are RDF, RDFS, OWL and SPARQL.
From the practical point of view one of the most
interesting Semantic Web initiatives is Linking Open
Data project [
To the moment, there are quite few solutions that
support variuos steps required to include one’s data into
Linked Open Data space, even though they are based on
existing hypertext web technologies. And there is no
system that includes all the functionality recommended
by LOD project. The most complex solution available
now is Virtuoso Universal Server [
However, both Virtuoso and D2R Server do not go
beyond simple RDF publishing. Their RDF-resource
interlinking capabilities are limited to URI generation
from templates. In many cases using such templated
URIs cannot expose identity relations between RDF
resources from different datasets and truly interlink
these datasets. There are several applications for RDF
data interlinking and link supporting, such as SILK
[
The possibilities of non-trivial usage of generated
linksets are yet to be explored. Very few applications
take advantage of this feature of Linked Open Data
space. It is worth to mention SPLENDID system [
This article proposes a concept of automated data integration system in Linked Open Data space. Proposed system should
• Form the single dataset from multiple heterogeneous sources of structured or unstructured information in similar knowledge domain and support/update formed dataset;
• Discover and store links between resources from system dataset and resources from different Linked Open Data sets available on the Internet in RDF format, as well as implicit links between reso urces within system dataset;
• Publish system dataset in the Internet in RDF format and provide access to it via user interface and API;
• Provide users and external applications with unified query interface to all of system’s data sources; • Support different data source types (including relational databases and SPARQL endpoints) and support on-fly connection of new data sources; • Include flexible ontology of knowledge domain that follows Linking Open Data project recommendations and can be extended to s upport new data sources.
Proposed system will follow modular architecture and will consist of following components :
• Ontology of informational objects and links of interest to system data consumers and providers ; • Linking subsystem, that should discover and store links between resources from system’s data sources and/or resources from external Linked Data sources;
• Publishing subsystem, which should provide users and applications with access to resources from system’s datas et according to LOD project recommendations ;
• Data integration subsystem, which will contain mechanism to uniquely identify system’s resources both within system and in Linked Open Data space and provide uniform access to all system’s resources. This subsystem will include a set of adapters that provide unified SPARQL access to system’s data sources of different types (relational databases, Web APIs, etc.); • Harvesting and extraction subsystem with a set of harvester components, which will gather data from system’s sources of unstructured data (text files, scanned documents, etc.), transform it into structured form and store it. This system is a subject of a future work.
The system uses OWL ontology to semantically organize objects and links that match the concepts of interest from knowledge domain of system’s data sources. Ontology consists of core terms and imported modules which can be added in case when some resources in newly added data source require more precise definition. In Linked Open Data space ontology serves as system’s data vocabulary, it is used to establish terminological outgoing links to external datasets and allows external applications to discover metadata to establish ingoing links. Following the principles of Linked Open Data, ontology is annotated in human language with such terms as rdfs:label or rdfs:comment. Ontology’s terms should be defined in URI namespace controlled by the system. Ontology adapts common Linked Data vocabularies such as Dublin Core, FOAF, vCard, PRISM, SIOC, Creative Commons, BibTex, Schema.org. Core of ontology is based on ENIP RAS ontology.
Publishing subsystem will serve as an entry point to the system for human users and Linked Data applications . It should dereference URIs of system’s resources, i.e. return descriptions of the object or concept identified by these URIs. It can be achieved by using a mechanism called content negotiation. Depending on HTTP GET request header, publishing subsystem will return either HTML representation or RDF/XML (as required by Linked Data applications) representation of the resource.
Publishing subsystem will receive data from data integration subsystem. For dereferencing resource URI, following information should be requested from data integration subsystem:
• All the literal values of resource, all incoming and outgoing RDF links. This information can be retrieved with simple SPARQL queries with patterns {<URI> ?x ?y} and {?x ?y <URI>};
• Most likely the results of these simple requests will contain URIs of other system’s resources. Linked Data applications often traverse URIs they find in RDF documents. Therefore to reduce the number of HTTP requests publishing subsystem should extend aforementioned requests to some depth, or by applying some explicitly stated rules;
• Information on ontology class to which the requested resource belongs and all its ancestors; • Information on the dataset to which this resource belongs;
All the information retrieved from data integration subsystem will be represented as a set of RDF triples. In case of RDF document these triples will be merged into resulting RDF/XML document and returned to client. In other case the triples will be published as HTML+RDFa document generated from template. These templates can be specified in general form and then redefined for specific classes.
A data integration subsystem will provide other subsystems or external agents with uniform access interface to all of the system’s data sources. Requested information should be specified with SPARQL query. This subsystem will be responsible for presenting system’s data as single dataset in Linked Open Data space. There are several approaches to data integration systems – data warehousing, data mediation, peer-topeer systems . Proposed system is supposed to work with multiple strongly autonomous data sources; therefore it adapts data mediation architecture. The drawback of such systems (e.g. Virtuoso) is huge amount in network interactions required to produce query answer. Proposed system uses Linked Open Data principles to reduce this drawback.
Data sources will be connected to the system via
adapters, which are SPARQL endpoints capable of
querying data sources in terms of system ontolog y.
These adapters should be generic, configurable
components (e.g. JDBC adapter, REST API adapter).
As opposed to existing data integration systems with
semantic capabilities (e.g. Virtuoso with its Sponger
cartridges), resources from different data sources won’t
be merged into single dataset by providing same URI to
identical resources. Instead, in the spirit of Linked Open
Data, every data source should be considered to contain
unique resources and get its own sub-namespace (like
http://<system_URL>/datasets/<source_id>). Adapter
should confront every resource from its data source
with HTTP URI from this namespace. Therefore we
will be able to track resource origin by its URI. When
new data source will be added to the system, its adapter
will be configured by specifying generic adapter
settings (e.g. JDBC connection string), general Dublin
Core description of the source, topic of interest
categorization, licensing information, etc. Adapter
configuration also includes the set of ontology classes
and properties to which data in these sources belongs.
This information can be entered manually or obtained
with SPARQL ASK request. Next, adapter
configuration will be published as voiD [
Execution of queries in data integration subsystem
will be carried out as follows . The first step is a
SPARQL query rewriting according to the axioms of
ontology, as described in [
Traditionally, the next step in data mediation process is construction of the physical query plan and its execution. During execution of query with atoms related to different data sources the results of subqueries to these data sources are joined. However, in the proposed system subquery results can be joined on literal field values and not on the URIs, because data sources are presented in a form of independent Linked Open Data sets and do not share common URIs . If subqueries to different data sources are to be joined on URIs we will have to use the sets of links generated by linking subsystem between these data sources. Each conjunctive query is a graph pattern with vertices being either literal values or the URIs or variables, and edges are labeled with predicates in terms of different data sources. If two adjacent edges are labeled with predicates from different sources, it is necessary to refer to the linkset for this pair of sources and select resource pairs that satisfy a given part of graph pattern. By performing this operation on all the links in the conjunctive query, we will obtain the set of resource URIs that satisfies part of the pattern that defines relationships between different data sources. Then the subquery parts related to specific data sources will be executed by adapters with corresponding join variables being replaced by URIs from linksets . On this step traditional query optimization techniques can be applied again.
RDF documents published in the Linked Open Data
space are required to contain outgoing links. These
outgoing links are RDF triples with the subject being
the URI of the resource from the local namespace and
the URI of the object and / or predicate belong ing to the
namespace of another dataset. The most important type
of outgoing links is identity links that point at URI
aliases used by other data sources to identify the same
real-world object or abstract concept. Identity links can
use such predicates as owl: sameAs, rdfs: seeAlso or
special SKOS terms. Although the uses of predicate
owl: sameAs in the LOD space are often contrary to the
semantics of OWL [
Linking subsystem will work as follows. In the first step, the two data sources to be interlinked are found. For this pair an initially empty voiD linkset is created and published. When new data sources is added to the system the linksets between this new data source and all existing data sets from other sources are automatically created. A linkset between internal dataset and external Linked Open Data set will be created in one of the following cases :
• The user can manually select a pair of datasets for linking;
• Relevant datasets can be discovered using
HTTP referrer technique described in [
• Relevant datasets can set be discovered by linking subsystem itself by traversing links in external dataset that is already linked to one of internal datasets.
When two target datasets for interlinking will have
been selected, the subsystem will clusterize datasets by
classes and determines pairs of clusters to be
interlinked. This should be done to reduce the number
of pairwise comparisons of datasets elements . In the
case of two internal datasets both of them are described
by the same ontology, so that pairs of clusters contain
instances of same ontology classes. In the case of
linking to an external dataset the subsystem might select
pairs of classes with help of different ontology mapping
techniques [
The third and final step of interlinking involves pairwise comparison of clusters elements to detect pairs of identity relations . These relations will be detected using SILK LSL language rules . In the case of internal data sources, rules will be declared together with the ontology and determine which instances of the same class are identical. In the case of an external dataset rules will be either specified manually, or derived from the existing rules and ontology mapping rules .
Complete binding is achieved by pairwise comparison of all elements of all datasets (both internal and external), but in practice such comparison is impossible. Link generation optimization requires additional study.
This paper proposes a concept of data integration system orientated towards Linked Open Data space. The novelty of this concept lies in its hybrid approach; the system proposed combines data mediation and data warehousing approaches by using locally stored linksets as indexes for a search engine hasn’t been implemen ted yet. To the author’s knowledge, such method hasn’t been implemented yet. Besides, while there are works dedicated to bringing single data sources into the LOD space or dealing with multiple already present sources in LOD space, the idea of bringing multiple data sources into LOD space via single data integration system has received very little attention.
Currently, the proof-of-concept system is being developed in CC RAS as a part of a practical project dedicated to integration of data on protected sites and animal species. While participating in a group on this project, the author is working on query answering algorithms in presence of linksets . As a result of this project, a large set of data on national parks should emerge in the LOD space, and if incoming links from external datasets appear, the project would be considered to be successful.
Future works on this project might include the study of link network generation and support algorithms. The system can also be extended with modules to access external Semantic Web resource aggregators (sig.ma) and semantic search engines (sindice.com). Also, additional studies in the management of licensing and data access in the context of the Linked Open Data are required.