The Deep Web is constituted by dynamically generated pages, usually requested through a HTML form; it is notoriously di cult to query and to search, as its pages are obviously non-indexable. More recently, Deep Web data have been made accessible through RESTful services that return information usually structured in JSON or XML format. We propose techniques to make the Deep Web available in the Linked Data Cloud, and we study algorithms for processing queries, posed in a transparent way on the Linked Data, on the underlying Deep Web sources. The tool we developed mainly focuses on exposing RESTful services as Linked Data datasets thus allowing a smoother semantic integration of di erent structured information sources in a global dataand knowledge-space.
Slowly but steadily, the Web has moved from a huge collection of unstructured
textual documents to a gigantic repository of structured data. At a rst stage,
the original static Web pages have been replaced by dynamic ones fed by
information coming from Deep web data sources which cannot be directly queried.
Then we assisted to the ourishing of new services that expose structured data
by exploiting standard Web technologies thus making possible their composition
for the creation of new integrated applications (mash-ups [
On the other side, the Linked Data (LD) approach bases on the idea that data can be delivered on the Web together with their explicit semantics by means of common vocabularies. Following the Linked Data principles2, datasets should be accessible through a SPARQL endpoint. Moreover, by using federated queries3 an agent is able to automatically integrate data coming from di erent sources thus creating a data space at a Web scale. Unfortunately, also the Linked Data approach comes with its drawbacks: { the e ort in setting up a SPARQL endpoint is felt more di cult than a RESTful approach from service providers. Nowadays, it is much easier to nd a JSON-based service than a LD-based one. { programmers are more used to JSON services than to SPARQL endpoints; { service providers are usually not interested in exposing all the data they have but only a small portion.
Based on the above points we can see that while from the practical point of view the champion is the RESTful approach, if we look at the knowledge point of view Linked Data represents a much better alternative. This was the leading observation that inspired us in the development of PoLDo. With PoLDo we can expose an already existing RESTful service, even a third-party one, as a SPARQL endpoint thus making it part of the Linked Data cloud. Thanks to a con gurable query planner, PoLDo is able to break down a SPARQL query into a sequence of RESTful service invocations. Starting from the retrieved data it then builds the answer to the original SPARQL query.
The remainder of this paper is structured as follows. In the next section we report on some relevant related work on accessing the Deep Web while in Section 3 we describe PoLDo together with an explanatory example. Conclusion closes the paper. 2
The term Deep Web (sometimes also called Hidden Web) [
As an example of Deep Web source, the whitepages.com website presents a
form where, when searching by name, the last name is a required eld. In order
to look for a person named Joseph Noto, living in New Jersey, we would ll
the form which in relational terms, corresponds to issuing a SQL query to the
underling database. Therefore, a Deep Web source can be naturally modeled as a
relational table (or a set of relational tables) where not all queries are allowed; in
fact, access limitations exist on such relations. In particular, they can be queried
only according to so-called access patterns, each of which enforces the selection
on some of the attributes (which corresponds to lling the corresponding eld
in the form with a value). It is believed that the size of the Deep Web is several
orders of magnitude larger [
Two main approaches for accessing the Deep Web exist [
The problem of answering a query over sources with access patterns falls into
the setting of vertical data integration, where a limited number of Deep Web
sources, whose interface is known (having been possibly understood
automatically), are queried in a structured way. But if we want to search via keyword
search the Deep Web together with the ordinary, \shallow" Web, we cannot rely
on a set of known sources, and therefore we have to adopt the surfacing approach.
Surfacing the Deep Web poses several challenges because it is not easy to get
a good coverage of the content of a source while at the same time limiting the
number of sampling accesses on it. Keyword search on relational databases has
been traditionally studied in several works in the literature (see e.g., [
PoLDo
The high level architecture of PoLDo is represented in Fig. 1. The engine is
responsible of getting the SPARQL query and breaking it down to a sequence of
RESTful calls to a remote service. The transformation is made possible thanks
to a mapping le that maps Linked Data URIs to the elements of the signature
of the remote call. While querying the remote service, PoLDo feeds an RDF local
triple store (Jena Fuseki in its current implementation) which is in charge of
processing the actual SPARQL query. More in details, we have:
PoLDo engine It receives the SPARQL query and extracts all the constants from
the graph template in the WHERE clause. Then, by using the algorithm in
[
Jena Fuseki The triple store is used to save a LD version of the data which are incrementally retrieved from the RESTful service. The availability of a third-party triple store makes PoLDo able to support the full speci cation of SPARQL query language. Furthermore, it is able to return the data in all the formats supported by Jena Fuseki.
PoLDo mapping le This le contains information about how to map the URIs of the SPARQL query to inputs and outputs of the service. Moreover, it also describes the entities represented by inputs and outputs as well as their mutual relations. 3.1
PoLDo mapping language PoLDo mapping le allows the designer to create a link between URIs contained in the SPARQL queries processed by the engine and, at the same time, to enrich their semantics by explicitly adding information about the corresponding OWL class or property that can be de ned also in an external vocabulary (e.g. DBpedia). Mapping rules can also be created to describe RDF triples containing information on how to relate values of an input parameter with the outputs of the service invocation. All the rules contained in the PoLDo mapping le are, in turn, represented as RDF triples which refers to a corresponding RDF-S ontology. The main element of the PoLDo ontology is the class poldo:Service (see Fig. 2). It describes each service in terms of its base URL (poldo:hasUrl), the
HTTP method GET or POST (poldo:hasMethod), the language of the answer
to the service call, e.g. XML or JSON (poldo:hasLanguage) and its inputs and
outputs (poldo:hasInput and poldo:hasOutput). The ontological description
of poldo:Input and poldo:Output are represented respectively in Fig. 3 and
Fig. 4. As we can see, both inputs and outputs of a services can be mapped
as instances of a class or as a property. Indeed, especially when the type of
the corresponding value is di erent from a string, we may have cases when the
parameter is better represented by a property rather than the subject or
object of a triple. We may think at a geographical service returning places based
on their coordinates. In this case, coordinates are better mapped to the
properties geo:lat and geo:long of the Basic Geo vocabulary4. The modeling of
poldo:Input and poldo:Output classes try to catch all possible cases in the
description of the inputs and outputs of a service. For instance, the
parameter poldo:hasFixedValue is used when we need a key to access the RESTful
service. As for poldo:Output we just highlight that it possible to model the
4 https://www.w3.org/2003/01/geo/
situation when the service returns a single value or a list of values by means of
the poldo:hasStructure and rdf:li statements.
The main component of PoLDo is its query planner implemented in the PoLDo
engine module. It uses the algorithm presented in [
For a better understanding of the overall approach, we now describe how PoLDo engine works by means of an example. Suppose we have two services related to music events in a city as in the following.
service input output http://example.com/api/returnSinger city title, singer http://example.com/api/returnPlace singer city, country Given a city, the rst service returns the title of the events in that city together with the performing artist. The second service returns the birth place of an artist. The corresponding mapping le will then contain the following triples: :returnArtist a poldo:Service ; poldo:hasURL ''http://example.com/api/returnSinger'' ; poldo:hasLanguage ''JSON'' ; poldo:hasMethod ''GET'' ; poldo:hasInput :city ; poldo:hasOutput :title ; poldo:hasOutput :singer . :returnPlace a poldo:Service ; poldo:hasURL ''http://example.com/api/returnPlace'' ; poldo:hasLanguage ''JSON'' ; poldo:hasMethod ''GET'' ; poldo:hasInput :singer ; poldo:hasOutput :city ; poldo:hasOutput :country ; Now suppose we want to know the name of singers born in a speci c city. The corresponding SPARQL query is then: SELECT ?singer WHERE { ?singer dbo:birthPlace ?city . ?city rdfs:label ''modena'' . ?singer a dbo:Person .
?city a dbo:Place . } LIMIT 1 The only entry points PoLDo engine has to query the services are constant symbols, in our case modena whose corresponding entity is declared to be a dbo:Place in the SPARQL query. By looking at the mapping le, the engine discovers it can query the service :returnArtist. Then, by using the outputs (constants) of the rst service, the engine may query :returnPlace and, if lucky, it can get that one of the artists returned by :returnArtist was born in modena. If this is not the case, the engine uses the constants returned by :returnPlace to query again :returnArtist thus continuing its search of the answer for the original SPARQL query. It is worth noticing that while iteratively querying the services, PoLDo builds RDF triples by creating fresh entities corresponding to the arriving constants and by enriching and connecting them thanks to the rules states in rows 15-20 of the above mapping le. All the triples are saved in the triple store which is then natively queried by means of the original SPARQL query. PoLDo engine stops querying the original services in the following cases: (i) the answer to the query is found; (ii) there are no more fresh constants and then the answer to the original query can not be found; (iii) the execution time exceeds a timeout set by the designer. 4
In this paper we presented PoLDo, that acts as a middleware between a RESTful service and SPARQL endpoint. By means of PoLDo we are allowed to expose the Deep Web data available via RESTful services as Linked Data that can be easily integrated in the so called Linked Data Cloud. The tool we developed adopts algorithms and techniques coming from the Deep Web literature to make possible the composition of services at a data level. Via a mapping le, PoLDo is able to interpret a SPARQL query in terms of a sequence of remote calls to external services and to translate the returned data in a temporary RDF graph which is locally stored in a triple store. The approach we developed is for sure a step forward the creation of a global, semantics-enabled, integrated, gigantic data graph as in the original view of the Semantic Web. Acknowledgments. Andrea Cal acknowledges partial support by the EPSRC project \Logic-based Integration and Querying of Unindexed Data" (EP/E010865/1) and by the EU COST Action IC1302 KEYSTONE.