This poster proposes a minimal, backward compatible and combinable restful interface for RDF Stream Engine. A number of RDF Stream Processors exists (e.g., CQELS [1], SPARQLstream [2], ETALIS/EP-SPARQL [3], Sparkwave [4], INSTANS [5] and C-SPARQL Engine [6]), but they do not talk each other. This hampers comparative evaluations: existing benchmark proposals [7, 8] had to create software adapters to test the various processors. In this condition, it is difficult to assess how much the benchmark results depend on the performances of the processors and how much on those of the adapters. Moreover, the lack of a shared protocol to transmit RDF streams hinders the combined usage of those processors. For instance, a user may want: a) to deploy SPARQLstream to natively process data streams1; b) to semantically enrich the resulting RDF streams using Sparkwave (or INSTANS); c) to aggregate the enriched streams in events using the C-SPARQL Engine (or CQELS); and d), finally, to detect complex events with ETALIS/EP-SPARQL. This poster proposes a restful interface for RDF Stream Processors that is: 1. minimal - more sophisticated interface can be envisioned, but in this attempt we would like to create a broad consensus, thus we avoid proposing controversial solutions. 2. backward compatible - we are reusing RDF and SPARQL standards wherever we can so to guarantee that adaptation of non-streaming clients for RDF and SPARQL is straight forward. 3. combinable - the proposed interface enforces that the output of a processor can serve as input to a processor (including the one that generates it). The remainder of the paper is organised as follows. Section 2 briefly presents the background required to understand the proposed interface. Section 3 proposes the interface. Section 4 shortly discusses two requirements that are not considered for this minimal proposal and how the interface can be extended to cover them. A proof of concept implementation of the proposed interfaces for the C-SPARQL engine is available for download at http://streamreasoning.org/ download.
From a conceptual point of view, existing RDF Stream Processors are homogenous. They define the notion of RDF Stream – an unbound list of tuples < t, τ > where t is an RDF triple and τ is a non-decreasing timestamp –, and continuous SPARQL query – a SPARQL query extended so that it can process RDF streams using continuous operators (e.g., windows to logically convert a portion of the infinite RDF stream in an RDF graph) and time-aware operators (e.g., sequence to ask that a graph pattern is detected before another one).
To the best of our knowledge, limited efforts was spent in defining a protocol
for: a) transmitting RDF stream across RDF Stream Processor on separated
machines, b) registering a continuous query in a processor, and c) observing
the continuously evolving results. The only existing solution are proprietary.
For instance, the C-SPARQL Engine is typically used within the Streaming
Linked Data framework [
A community effort is needed to propose a continuous SPARQL extension that can span across the existing proposals, but we believe this is the right time to propose a restful interfaces that processors can easily implement.
The following proposal specifies how to manage RDF streams, continuous SPARQL queries, and observers of continuous results (see Table 1 for details).
Complying to restful principles, users can register a new RDF stream σ in the processor using the PUT method on the resource /streams/. As a result the RDF stream /streams/σ becomes available in the processor. At this point, they can stream information on the RDF stream POSTing an RDF graph to /streams/σ) and they can unregister it using the DELETE method. The list of all registered stream is returned when GETting the resource /streams/.
It is worth to note that, learning from flexible time management in data
stream processors [
User can register a new continuous SPARQL query γ in the processor using the PUT method on the resource /queries/. The proposed interface is agnostic w.r.t. the language used to declare the query and leaves to the processor to parse the query in the body of the request. Nonetheless, it requires the query to refer only to RDF streams already registered in the processor. If the user tries to register a query on streams that have not been registered, yet, the service must refuse to register the query. If the registration is successful, the processor starts the continuous execution of the query and the query /queries/γ appears in the list of queries that can be retrieved GETting the resource /queries/. As for the RDF streams, the query /queries/γ can be unregistered using the DELETE method. The method POST on the resource /queries/γ is used to start observing the query results, to pause the query and to restart it.
Access to query results follows an observable-observer design pattern. In order to start observing the results of a query γ, a user has to POST a callback URL to /queries/γ. The created observer ω is identified by an URL of the form /queries/γ/observers/ω. The user can stop observing the query by DELETing this resource. Multiple observers per query are possible. Whenever γ computes new results, the processor notifies all the observers by invoking the provided callback URLs.
If the query is of the forms SELECT or ASK, results must be formatted according to SPARQL 1.1 query results2, thus allowing for backward compatibility with existing SPARQL resultset parsers.
If the query is of the forms CONSTRUCT or DESCRIBE, the processor must POST an RDF graph containing the result. As a result our proposal is not only backward compatibility – it is conform to SPARQL 1.1 result formats –, but it is also combinable – the results of a query can be POSTed to another registered RDF stream. The callback URL passed as parameter in starting an observer simply has to be the URL of an existing RDF stream3. 4
The proposal, being minimal, ignored important requirements w.r.t. time modelling, access control, and transmission overhead. 2 Our implementation supports http://www.w3.org/TR/2013/REC-sparql11-results-json/ 3 In order to avoid the overhead to stream on HTTP an RDF stream that is consumed by the same processor, when a query γ of the forms CONSTRUCT or DESCRIBE is registered, an RDF stream, whose identifier is /streams/γ, is automatically registered. The result of the query γ is internally streamed on it.
Adding the application time to the protocol is only a matter to POST a
timestamp together with the RDF graph. However, as explained in [
The proposed interface lacks access control, but it is ready for HTTP-based access control. An HTTP server, between the user and the Restful service container, can handle access to /streams and /queries. Moreover, only the owner of a query γ can start observing the results of γ or is allowed to list all the observers (i.e., GETting /queries/γ/obsevers/). However, investigating OAuth-based access-control is on our research agenda.
Last, but not least, we recognise that the transmission overhead of the proposed solution can reduce the processor throughput if the user frequently POSTs RDF graphs containing only few triples. In our future work, we intent to explore the streaming of RDF triples in N-quads format on a Web-socket.