RSPLab is an infrastructure that reduces the e ort required to design and execute reproducible experiments as well as share their results. It provides a programmatic environment to deploy RDF Streams and RSP engines, interact with them and continuously monitor their performances and collect statistics. In this demo well show how to deploy RDFStream and RSP engines in the cloud and how to interact with them registering and unregistering continuous queries lively. We will show how these action impact the performances and we transform the results in visual plot on a chart.
In the RDF Stream Processing (RSP) community particular interest has arisen
on empirical research on benchmarks and evaluation methodologies. These
comprehend studies on query language expressive power, performance, correctness
of results, memory load and latency [
Beside community e orts, the evaluation of RSP engines is still not
systematic. This mostly because RSP engines have di erent semantics, interfaces, and
processing models. Moreover, existing test drivers [
In this paper, we present RSPLab [
We will show how it enables design of experiments by the means of a programmatic interface that allows deploying the environment, running experiments, measuring the performance, visualizing the results as reports, and cleaning up the environment to get ready for a new experiment.
The remainder of the paper is structured as follow: Section 2 shows the RSPLab architecture, Section 3 describes how the demonstration will be carried out and nally Section 4 concludes and provides information on where additional material regarding the resource can be found.
RSPLab was developed using Docker, i.e. a lightweight virtualization framework1. This allows to fully control the available resources enabling experiments execution at scale and it guarantees components isolation. Moreover, it fosters reproducibility by making the execution hardware-independent.
Streamer. The data provisioning tier publishes RDF streams from existing sources.
It is implemented using a modi ed version of TripleWave [
Collector. This tiers is in charge of monitoring the execution of a benchmark, in particular it includes: (1) a distributed continuous monitoring system3, that collects performance statistics, (2) a time-series database4 which is used to store the data collected and (3) a RSPSink that persists query results on a cloud le systems, allowing for post-hoc correctness analyses.
Controller. The control and analysis tier allows the RSPLab user to interact with the whole environment by deploying streams and engines and monitoring the performance of the latter. A programmatic interface, implemented using iPython Notebooks5 includes wrappers for RSP services, TripleWave and RSPsinks. Assisted data visualization dashboards are accessible, in real time, via Grafana6, i.e. a dashboard that reads directly form the time-series database. Experimental reports annotated with VOID can be generated also using the library.
2 https://github.com/streamreasoning/triplewave/tree/rsplab 3 https://github.com/google/cadvisor 4 https://www.influxdata.com/ 5 https://ipython.org/notebook.html 6 https://grafana.com/
1 Q1 = rsp . BenchmarkQueries .
CityBench . Q1 2 e = rsp . new experiment ( ) 3 e . add engine ( " http : / / c q l e s . rsp lab . org " , 80 , rsp . D i a l e c t s .
CQELS) 4 e . add KPIs( rsp . KPI .
Memory Consumption , rsp . KPI . CPU Load) 5 e . add query ( "CB. Q1" , rsp .
QueryType . Query , Q1 , rsp .
D i a l e c t s .CQELS) 6 e . add tbox ( "CB. Q1" , name=" c i t y t r a f f i c . owl " , base=" rsp lab . org " ) 7 e . add graph ( "CB. Q1" , name=" Se ns orR ep osi to ry . r d f " , base =" rsp lab . org " ) 8 e . add stream ( "CB. Q1" , "
AarhusTrafficData158505 " , base=" rsp lab . org " ) 9 e . add stream ( "CB. Q1" , "
AarhusTrafficData182955 " , base=" rsp lab . org " ) 1 #WARM UP 2 3 rsp = RSPEngine ( ehost , eport ) ; 4 5 f o r d i n experiment . graphs ( ) : 6 rsp . register graph ( d ) 7 f o r s i n experiment . streams ( ) : 8 rsp . register stream ( s ) 9 f o r q i n experiment . q u e r i e s ( ) : 10 rsp . register query ( q ) 11 rsp . new observer ( query , ' d e f a u l t ' ) 12 spawn sinks ( experiment ) 13 14 # OBSERVE 15 wait ( experiment . duration ( ) ) 16 17 f o r q i n engine . q u e r i e s ( ) : 18 f o r o i n engine . o b s e r v e r s ( q ) : 19 rsp . unregister observer ( o ) 20 rsp . unregister query ( q ) 21 f o r s i n engine . streams ( ) : 22 rsp . unregister stream ( s ) 23 rsp . report . publish ( experiment )
Step-1 Deployment and con guration of RDF Streams: we show how to deploy and con gure the Streamer component in order to stream the included datasets. Step-2 Deployment and con guration of RSP Engines: we show how to deploy and con gure the Consumer tier, with particular focus on the C-SPARQL and CQELS engines.
Step-3 Continuous query of the streams and collection of the results: we show
how to issue continuous queries to the streams using the programming interface.
We use the included Python library7 that allow to fully control all the tiers
included in RSPLab. In particular it allows to (1) create and launch experiments
following the structure described in [
Step-4 Visualization of statistics in real time: we show how to use Grafana on the Controller tier in order to visualize the performance indexes in real-time (an example is shown in Figure 2). Moreover we will show how with RSPLab is possible to observe the di erent phases of an experiment: the warm-up phase, where the engines are deployed and the streams and the queries are registered, and the observation phase, where the engines are in steady state, consuming the streams and answering the queries. 7 https://github.com/streamreasoning/rsplib
With this demonstration we aim to show the e ectiveness of RSPLab in facilitating the deployment and execution of RSP benchmarks.
We will run both the Fig. 2: A example of dashboard showing CPU load and included benchmarks Memory usage. (e.g., Citybench and LSBench) and custom variation, by adding and removing queries in real-time, in order to highlight how these actions impact on the performance. 4
The main contribution of this demonstration paper is the proof of RSPLab e ectiveness. The Docker-based architecture and the programmatic interface facilitate the process of deploying the environment, running experiments, measuring the performance, visualizing the results as reports.
RSPLab is released as open-source8, examples, documentation and tutorials are available on GitHub9, hosted by the Stream Reasoning organization.
9 https://github.com/streamreasoning/rsplab