Recently, there has been e orts in lifting the content produced by stream sources, e.g. sensors, to a semantic level. In particular, there is ongoing work in representing stream data following the standards of Linked Data, creating what it is called Linked Stream Data. The advantages of Linked Stream Data are manyfold: adding semantics allows the search and exploration of sensor data without any prior knowledge of the data source, and using the principles of Linked Data facilitates the integration of stream data to the increasing number of data collections that form the Linked Open Data cloud, enabling a new range of applications. However, the highly dynamic and temporal nature of Linked Stream Data poses many challenges in making Linked Stream Data a reality that users and applications can bene t from. In this position paper we address the challenges in Linked Stream Data processing. We will focus on data representation and storage, query model and query processing, highlighting the main di erences compared to Linked Data processing and looking at the approaches that currently address these challenges, showing what has been done and what is still needed, suggesting ideas for future research.
Stream data sources, in particular sensors, are very popular nowadays and can
be found everywhere, for instance in mobile phones (accelerometer, compass,
etc.), in weather observation stations (temperature, humidity, etc.), in the health
care domain (heart rate, blood pressure monitors, etc.), in devices for tracking
people's and object's locations (GPS, RFID, etc.), and in the Web at large, with
online communities services such as Twitter and Facebook delivering real time
data on various topics (RSS or Atom feeds, etc.), where users play the role of
citizen sensors [
Sensed data is often archived or streamed as raw data, but rarely associated with enough metadata describing its meaning. Meaning of sensor data includes the feature of interest, the speci cation of measuring devices, accuracy, measuring condition, scenario of measurements, location, etc. Such metadata is essential for search and exploration when the user is confronted with large numbers of sensors and gigabytes of sensor data. The lack of metadata also makes the integration of sensor data with other data sources a di cult and labour-intensive task.
There have been a lot of e orts in employing Semantic Web technology to
semantically enrich sensor data [
In this paper we address the challenges of Linked Stream Data processing, focusing on data representation and storage, query model and query processing. We highlight the main di erences compared to Linked Data processing which prevent standard techniques to be directly applied. Then, we move on the approaches that currently address these challenges, showing what has been done and what is still needed, suggesting ideas for future research. The remainder of this paper is organized as following. The section 2 focuses on the data representation and the need of new query models. The query processing and integration with other data sources is addressed in section 3. Section 4 concludes the paper and gives some nal remarks on the topic. 2
Linked Stream Data follows the standards of Linked Data, therefore we believe
that it should be represented based on RDF, a widely used standard for Linked
Data. A RDF representation of stream data, or RDF Stream, extends RDF by
adding temporal information. There is already ongoing work that follows this
principle: for stream data, CQL [
With the RDF Stream de ned, we now need to model queries over Linked
Stream Data. Similar to stream data, queries are expected to be continuous,
i.e. they are likely to be valid for a certain time period. We suggest a query
model based on CQL. CQL consists of query fragments inherited from relational
query models, plus three new data mappings operators: relational-to-relational
mapping, stream-to-relational mapping, and relational-to-stream mapping.
Following the same idea, we suggest to de ne operators to map RDF temporal
graphs to RDF graphs and vice versa. The idea of \snapshots" of RDF temporal
graphs enables the creation of nite RDF graphs from a temporal graph [
The suggested query model is simple yet quite powerful. To demonstrate it we use the following query example: \whenever a car is within 2km of a junction for which a speed sensor and a tra c camera is available, report the car's average speed and camera image". For that we assume we have sensors streaming images captured by tra c cameras and sensors that can track the speed of cars passing by. We also assume that we have these two types of sensors allocated along di erent streets, and that cars contain GPS sensors that can stream the car's current location. Finally, there is a metadata graph http://sensors.deri.org/metadata containing all other information about these sensors, such as geographic location. Figure 1 shows the example query written using SPARQL query patterns and window-based graph patterns, where http://sensors.deri.org/streams/mygps/ is the URI of the GPS location stream and spatial:distant is a built-in function returning the distant between two coordinates in kilometers.
SELECT ?junctionName ?snapShot AVERAGE(?speed) as avgSpeed FROM NAMED <http://sensors.deri.org/streams/mygps/> [now] as ?gps FROM NAMED ?trafficcamera [now] as ?junctionImage FROM NAMED ?trafficsensor [RANGE 30 seconds] as ?carSpeed FROM NAMED <http://sensors.deri.org/metadata> WHERE {
GRAPH ?junctionImage {?camera cam:hasSnapShot ?snapShot} GRAPH ?gps {?car geo:lat ?carLat.?car geo:long ?carLon} GRAPH ?carSpeed {?car traffic:passbySpeed ?speed} GRAPH <http://sensors.deri.org/metadata> { ?trafficcamera geo:locatedAt ?junctionLoc. ?traffisensor geo:locatedAt ?junctionLoc. ?junctionLoc geo:lat ?juncLat. ?junctionLoc geo:long ?juncLong. ?junctionLoc geo:name ?junctionName.
FILTER {spatial:distant(?carLat,?carLon,?juncLat,?juncLong)<=2} } } GROUP BY ?speed
The query rst gets the car's current location (given by the GPS), and joins
it with the graph containing the metadata, which provides the identi ers for the
speed and tra c camera sensors at the junctions. Since the URIs of the tra c
camera streams and the tra c sensor streams needed are unknown and subject
to change (since they depend on the car's location), they are represented as
variables in the graph query pattern. The sliding-window operators, [NOW] for
current snapshot and [RANGE] for snapshots within a time range, are applied
over the continuous tra c camera and speed stream. The result of these
operators are materialized and represented as RDF graphs that can be processed by
the SPARQL query processor. Details of the proposed formalization for RDF
Streams and the model for continuous query over Linked Stream Data are
presented in [
Even though our proposed query model allows queries to be executed using
standard query processors for triplestores, the execution is very ine cient, since
they do not support continuous queries. That means that each query would
have to be repeatedly issued as often as the updates on the streams, for as long
as the query is valid, every time checking if the new values satisfy the query's
conditions. In some cases, as in the example query from previous section, the
query is valid for a long period of time, which make this approach prohibitive.
StreamingSPARQL has addressed this issue by having translation rules that
translate continuous queries to SPARQL algebras and sliding-window operators.
Although it gives a solution for handling continuous queries, this approach is
still quite ine cient, since triplestores are mostly based on relational database
storage, which are proved to be ine cient for data with high update rates [
A major issue in continuous query processing is memory consumption. It
is common that Linked Stream Data processing involves a large amount of
data that is likely not to t into main memory. Therefore, intermediate results
need to be stored on disk and later reloaded for further processing. Since disk
reads/writes are generally expensive minimizing such operations becomes very
important. One approach to the problem is to apply dictionary encoding, which
is commonly used by triplestores [
In applications that involves a combination of many data sources, especially if
some of them are not stream sources, the performance of the query processor can
be greatly improved if some of the intermediate results are cached, for instance,
for the input data that do not change very often during the duration of the query.
Results reported in [
Traditional relational databases are equipped with query optimizers which
are responsible for nding the best execution plan. Such feature is also desirable
in Linked Stream Data processing. A query optimizer typically computes the
optimal query plan in the compiling phase using the statistical distribution of
the data. However, in context of stream data, this optimizing technique does not
yield satisfying results, as the distribution of the data changes during run-time.
The CQELS system [
Further ideas to improve continuous query processing are controlling the
sampling rate of the input stream data and building stochastic/statistical model
for predicting data series. Both ideas aim at reducing the number of data that
needs to be retrieved for query evaluation. The former suggest to sample the
data from stream source in a slower rate than the data is produced. While this
results in lost of information, there are many applications in which sampling
might su ce. The latter consists in modeling the stream source, such that the
data values can be predicted, avoiding access to the stream source. In both cases,
a combination of prior knowledge derived from historical data, human knowledge
in the form of processing rules and reasoners is needed. In particular, reusing
domain knowledge represented as ontologies and rules, and performing reasoning
in continuous query processing is an open and interesting research area [
This position paper provides an overall picture of a new emerging research area, Linked Stream Data processing. We have addressed the main challenges in this area regarding data representation and storage, query model and query processing. We have shown why standard Semantic Web technologies can not be directly applied, and highlighted research that is currently being carried out to solve these issues. However, there is still several open issues and our paper have also suggested ideas for future research.
This work has been supported by the Science Foundation Ireland under Grant No. SFI/08/CE/I1380 (Lion-2) and the Irish Research Council for Science, Engineering and Technology (IRCSET).