The RSP-QL model explains and uni es RDF Stream Processing (RSP) approaches into a more general semantics. It was successfully applied to model C-SPARQL, CQELS-QL and SPARQLstream taking into account the operational semantics of the existing implementations. In this paper, we present Yasper 1.0, an RSP engine that implements RSP-QL semantics. Moreover, we present the challenges we found during our implementation experience and we discuss the research opportunities they imply. For each of these challenges, we also formulate some hypotheses that will drive our future work empirical studies on RSP.
In this section, we present the background information required to understand the content of the paper.
We adopt a generic de nition of RDF Stream, i.e. a sequence of pairs (Oi, ti), where ti is a non-decreasing timestamp and Oi is either a named RDF Graph or a timestamped RDF triple.
RSP-QL [
A time-based sliding window operator W is a Stream-to-Relation (S2R)
operator [
A streaming dataset SDS is a set composed by an optional default element A0, n (n 0) named Time-Varying Graphs and m (m 0) named sliding window operators over k (k m) streams.
RSP-QL evaluation semantics is de ned as eval(SDS(G), SE, t), where SDS is a streaming dataset having G as active Time-Varying Graph, SE is an algebraic expression and t is a time instant. Accordingly, the extension to multiwindow queries involves more than one active Time-Varying Graphs, i.e. eval(SDS(Gi ... Gn), SE, t). The evaluation is computed over the instantaneous graphs Gi(t), i.e., eval(SDS(Gi; t), SE). An RSP-QL query is continuously evaluated against a SDS by an RSP engine. The set ET of evaluation time instants is determined by the RSP engine according to a reporting policy that, in RSP-QL, consists of the composition of the following strategies: CC Content Change { the engine reports if the content of the current window changes {, WC Window Close { the engine reports if the current window closes {, NC Non-empty Content { the engine reports if the current window is not empty {, and P Periodic { the engine reports at regular intervals.
The output of the RSP-QL query evaluation is an instantaneous multiset of
solution mappings for each evaluation time instant in "ET". Time-aware
operators called Relation-to-Stream (R2S) are required to transform the instantaneous
multiset of solution mappings into a Time-Varying multiset of solution mappings.
RSP-QL comprises the following R2S operators: the RStream, which emits each
solution mappings; the IStream, which emits the di erence between the
current solution mappings and previous ones, and; the DStream, which emits the
di erence between the previous solution mappings and the current ones.
C-SPARQL [
Triple. Queries written C-SPARQL are executed by an RSP engine3 that pipes a Data Stream Management System (DSMS) with a SPARQL engine. The CSPARQL engine delegates the window execution to the DSMS and the remainder of the query to the SPARQL engine. C-SPARQL reporting policy is on window close and non-empty content (WCNC). C-SPARQL provides only the RStream operator.
SPARQLstream [
CQELS-QL [
In this section, we introduce Yasper's architecture and how it implements
RSPQL concepts. Figure 1 shows the following modules of Yasper 1.0 Streams,
Windowing, SDS, Querying and Reasoning. Modules that exposes Yasper as a REST
service [
The Stream module, Figure 1 (a), contains the classes to represent a Stream.
Yasper adopts a generic data stream model [
Thanks to this representation, Yasper can replicate the two RDF stream
models4 studied in RSP, i.e. unbounded sequence of timestamped RDF triples
as in C-SPARQL vs. unbounded sequence of RDF graphs as in SLD [
The Windowing module, Figure 1 (b), is based on Esper5, an open-source DSMS that relies on the Event Processing Language (EPL) and special objects called listeners that continuously receive the EPL queries outputs. We represent the RSP-QL Time-Based Sliding Window Operator (henceforth referred to as window operator) as an EPL statement on an RDF Stream plus a listener that continuously receives its results. Each window operator maintains a
Time-Varying Graph that is responsible for generating an Instantaneous RDF Graph. The windowing is performed by means of temporal annotations of the StreamItems. Therefore Yasper can refer either to the ingestion{time { the time of arrival of a Stream Item { or to the event{time { the time each Stream Item is annotated with. Yasper works by default using Event Time that, notably, does not guarantee total ordering of Stream Items and, thus, requires further synchronization mechanisms in an open Web environment.
A streaming dataset SDS, Figure 1 (c), contains all the Time-Varying RDF
Graphs. For each evaluation time t, each window operator w pushes StreamItems
into a Time-Varying RDF Graph tvg, which reactively generates an
Instantaneous RDF Graph g The streaming dataset SDS can be consolidated into an
equivalent instantaneous dataset DI, i.e. a set of instantaneous RDF Graphs.
The content of an instantaneous RDF graph gi belongs to the current window
identi ed by their window operator wi at time t; wi might be a sub-window [
As described in Section 2, RSP-QL comprises four strategies to de ne an
RSP engine reporting policy. As of the time of the submission, Yasper reports
the results on Window Close (WC) and Non-empty Content (NC). However,
how policies work with multi window queries is not obvious. RSP-QL [
WINDOW ?w2 f? s ?p ? og FILTER ( ?w1 != ?w2) g
The Querying module, Figure 1 (d), contains the elements for query
instantiation and continuous execution. At this stage of development, Yasper accepts
only SELECT and CONSTRUCT queries; the R2S operators R-, I- and DStream
are available [
The reasoning module, Figure 1 (e), supports continuous query answering under entailment regime. For each evaluation time instant t, before evaluating an RSP-QL query q, Yasper computes the closure under the entailment e of each instantaneous RDF Graph in SDS(t) using the Jena Generic Rule Reasoner. Future work in this direction comprise the integration of RDFox and Ontop. 4
In this section, we present the challenges unveiled by our implementation
experience. Yasper relies on some design decisions that require to be discussed with
the SR/RSP community. In particular, we found issues about the streams item
representation, stream item order, and multi-window queries evaluation, which
relates to the notion of RSP-QL dataset SDS. We summarize these issues with
the following questions:
(Q1) What is the best way to model the stream content?
(Q2) How does the time model impact the processing?
(Q3) What does de ne the current SDS at time t?
(Q4) Is there an e cient way to maintain the SDS?
(C1) Stream Content. RSP-QL assumes that stream items are timestamped
(named) RDF Graphs. Actually, this assumption contrasts with how most of the
existing RSP approaches work in practice. Indeed, [
Although the theoretical equivalence between the two approaches has been
proved [
{ Hp.1 The cost of translating a stream of timestamped RDF statement to a
stream of named RDF graph is negligible.
(C2) Stream Ordering. RSP approaches typically assume the stream items
arrive with non-decreasing timestamps. Although this assumption is reasonable
for monolithic infrastructure, it does not hold for distributed or federated
architectures [
We name the problem SDS Consolidation and we introduce the notion of
Active SDS { i.e. the SDS against which an RSP-QL query q is evaluated at
time t. Dell'Aglio et al. [
In order to better understand the impact on the results of the di erent SDS consolidations semantics, let us consider the example illustrated in Figure 2: :w1 and :w2 are two window operators applied to two streams. By de nition, they de ne many windows with opening and closing time instants (o,c). Let us consider now the SDS at three relevant time instants: t = 30 where only a window de ned by :w1 is closed; t = 35 where both :w1 and :w2 de ne two { Hp.6 Once xed the SDS consolidation approach, the Maintenance Strategy does not in uence the correctness of an RSP-QL query. { Hp.7 the performance of the delta Maintenance Strategy is always comparable to those of the snapshot.
rebuild SDS.
Challenges C1 Stream Content C2 Stream Ordering C3 SDS Consolidation
SDS Maintenance
RSP-QL Variants Named Graph RDF Statement Non-Decreasing Monotonic
Current Closed / Snapshot
Virtual Late Items
Cached Deltas In this section we discuss the research opportunities that each of the presented challenges highlights.
Challenge C1, and partially C3, indicate that the RSP solution space is not
yet systematically explored and there is room for optimization driven by
RSPQL insights. The community e orts on empirical research should continue, also
supported by new resources like Yasper and the recent RSPLab [
Challenge C2 suggests that some typical SR/RSP assumptions are inadequate for distributed or federated environments. The management of out-of-order data arrival is an open challenge for mature systems like Spark8 and Flink9 and so should be for the SR/RSP community. Federated RSP is another appealing idea; networks of engines that exchange data and queries require to relax these assumptions and rethink the model.
We advocate the use of reasoning to deal with the challenges, i.e.
out-oforder data arrival and federation and planning. Moreover, correctness must be
rede ned to include relevant notions such as maximum delay, watermark and
multiple active windows [
Challenge C3 reveals that designing a syntax that takes into account all the aspects of RSP is hard. Especially because we, as a community, did not investigate what is the intended semantics of a query. A universal syntax might not exist, while task-speci c syntaxes that capture fragments of RSP-QL semantics can better t the user needs.
Speci cally for querying, we must focus on usability. For a query language, we need to reduce ambiguity and improve e ciency as much as possible even if it means trading expressiveness and usability if necessary.
Our intuition is that some window operators are easier to use in combination with speci c policies and our proposal is to re ect these preferred relations in the RSP-QL syntax. In Listing 1.2 we present three alternative syntaxes that we believe are tightly coupled. The window at line 1 suggests that the reporting will happen on window close (i.e. when the window slides). The window at line 2 is meaningful only when the system reports on content change. The window
at line 3 might be a possible syntax that highlights that sub-windows of a given size are allowed. 1 FROM NAMEDWINDOW : a [RANGE 5s , STEP 2 s ] ON STREAM : s t r 1 2 FROM NAMEDWINDOW : b [RANGE 5 s ] ON STREAM : s t r 2 3 FROM NAMEDWINDOW : c [RANGE 5s , SLIDE 5s , SPLIT 1 s ] ON STREAM : s t r 3
6
In this paper we presented Yasper 1.0, i.e. an RSP-QL compliant RSP engine, and the challenges we faced during our implementation experience.
At the moment of writing, Yasper supports the proposed syntax [
Regarding the presented challenges and our hypotheses, our goals is to
perform a descriptive study on RSP engines. To this extent, we will formulate
statistical tests to investigate our hypotheses and we will extensively evaluate
Yasper 1.0 and the other relevant RSP engines [
Acknowledgments. We thank Daniele Dell'Aglio for the supporting discussions and brainstorming regarding RSP-QL.