The amount of information available as online streams is increasing steadily. A number of RDF stream processing systems have been developed in an attempt to leverage existing Semantic Web technologies, and to support typical stream operations, but very little attention has been paid to the way in which event objects (i.e. data records representing events) are streamed. In this position paper, we present the issue of respecting event object boundaries in RDF streams, and discuss some pros and cons of the various solutions.
The amount of information available as online streams is increasing steadily, and
a number of domains already rely heavily on streams of data, e.g., market feed
processing and electronic trading [
The Semantic Web community has attempted to lift streaming content to
a semantic level [
While benchmarks have been developed to evaluate system performance [
In this position paper, we present the issue of respecting event object boundaries in RDF streams, and present some possible solutions with respect to existing state of the art, and lists some of the pros and cons of the di erent solutions. Finally, we discuss the open challenges for future research. 2
The authors in [
Contrary to queries executed against static RDF data, the constant ow of information in a streaming data context means that no nal answer can ever be returned. Queries must instead be executed continuously as new data becomes available. Typically, windows over streams are used to allow outdated triples to be ignored, which also makes it possible to support common aggregate functions in queries.
There are, as of yet, no standards for how RDF data should be represented as
streams, but a number of state-of-the-art RDF stream processing systems have
assumed that the data is delivered as individual RDF triples [
(hsubji; predi; objii; i) (hsubji+1; predi+1; obji+1i; i+1)
:::
Since the sequence of pairs is ordered the timestamps are strictly non-decreasing.
Triples with the same timestamp, although sequenced in a speci c order in the
stream, should be regarded as having \occurred" at the same time. This view is
similar to that which is often used in Data Stream Management Systems [
Event objects (i.e. data records denoting events) [
In sensor streams, the issue of event object boundaries can often be handled without any great di culties. For example, if synchronization issues are ignored, a sensor reporting a xed number of values, or the intervals between streamed events, is known streams can be viewed through a window that slides over the stream. But when the input is not predictable, a window over a stream, by itself, cannot guarantee that all the triples of an RDG graph are within window boundaries at any given point in time, thus, we could risk querying a partially streamed graph.
The solutions presented in the following section are aimed both towards RDF stream processing systems (i.e. event consumers), and towards the RDF streams (i.e. event producers). For brevity, queries will be discussed only in general terms, rather than as full SPARQL queries. 4
Depending on the complexity of the RDF graphs that are transferred in an RDF stream, di erent solutions are possible. As mentioned in the previous section, if the events in a stream have predictable event object boundaries, these can be respected simply by having a window that slides with a xed amount between query evaluations. However, when event objects are less predictable, this is no a su cient solution. To support dynamic, and heterogeneous, streams of event objects, a solution must be able to handle streams for which neither the intervals between events, nor the number of triples in an event, is known. Header A common approach to making transferred data interpretable is to provide a message \header", containing meta data about the message. In the case of RDF streams each event object could be preceded by a header, providing information about the RDF graph being streamed. A header would have to be generated by the event producer for every new event object. The event consumer (i.e the RDF stream processing system) would subsequently have to interpret each header, ensuring that registered queries are not executed against a partially transferred RDF graph. The header element could be represented in various ways, e.g., as a triple, a set of triples, or as a dedicated type. A simple example of a header can be seen Figure 1.
Past 4 2 7
Future
Commonality A common attribute, or feature of \commonality", can be used as an indicator for the boundary of an RDF graph. A very basic feature of commonality could be that triples belonging to an RDF graph are streamed in sequence. A trailing boundary triple could then be used as a marker to denote that the entire RDF graph has been streamed (see Figure 2).
A more robust solution would be to attach the trailing triple to a node that is unique for the particular graph, i.e., some node that can be used to refer to the speci c event object. This could provide support for graph boundaries in situations where graphs can overlap each other in a stream.
RDF Graph Stream RDF data streams are usually considered to be composed of individual RDF triples, but we can also consider streaming RDF graphs directly. Streaming sets of RDF triples together, rather than the individual triples separately, would greatly simplify the task of the event producer, since neither the order of decomposition of event object graphs, nor the addition of triples, needs to be considered. Figure 3 illustrates a stream of RDF triple sets, where each set represent a complete event object graph.
Past
Future
Customized Decomposition Controlling the order, by which the individual triples in an RDF graph are streamed, can sometimes provide a more e cient way of streaming RDF graphs. If an RDF graph can be transferred in an order that implicitly respects event object boundaries correctly, results can be returned even before the a complete RDF graph has been received. This can be quite valuable in contexts where latencies must be kept as low as possible. There is, however, no general way of employing this solution for arbitrary RDF graphs.
Passing a header, as the rst element of a decomposed RDF graph, can provide meta data about event objects. For example, before the arrival of an event object the header may convey the size of the arriving RDF graph, allowing the RDF stream processing system to adapt windows over the stream accordingly. A header could, on the other hand, require a di erent way of consuming data streams, if the header information is to be handled automatically by the RDF stream processing system. One of the bene ts of the header solution is that, if the header was handled by the system internally, queries would not need use any special patterns to respect event object boundaries.
Providing a trailing triple, generated for the purpose of separating two RDF graphs, provides a simple way of separating event objects in a stream. The triple could be generated in any number of ways, but the most robust solutions would require it to be connected, in some way, to a unique event object identi er in the preceding graph. Even if we assume that event object triples may overlap each other in the stream, the trailing triple could still be used to determine that the entire graph has been streamed.
The streaming of RDF graphs, rather than individual triples, is a feasible solution for adding support for event object boundaries. Streaming graphs, rather than triples, would require a di erent way of consuming data streams than in the case of triples. RDF stream processing engines, that rely on evaluation of all queries for every new incoming triple, would either be forced to suppress results based on partial graphs, or would need to support adding multiple triples at a time. The bene t of streaming graphs directly is that, since the event object boundaries never become an issue, engines could provide a built-in support for event object boundaries.
The nal solution, suggested in this paper, involves customizing the decomposition of a graph, so that the boundaries are implicitly respected by the order of the streamed triples. In domains where real-time execution is a priority this would enable queries to be executed as soon as a match can be found. However, since this solution is only guaranteed to work in some contexts, and requires a customized solution, it cannot be viewed as a general solution to the issue of event object boundaries. 6
While it is rarely discussed in research literature, the way in which RDF graphs are decomposed into streams of RDF triples has a direct impact on the ability to distinguish graph boundaries. In streams emitting predictable RDF graphs, it is usually su cient to let the boundaries of event objects be de ned by windows that slide across the stream. However, this solution is not su cient if neither the time interval between events, nor the number triples in each event object, is known ad remains static.
In this paper, we have presented four possible solutions, ranging from adding a single trailing triple to the end of each graph, to solutions that require modi cations to the way in which RDF data streams are represented, and the way in which they are typically consumed by RDF stream processing engines.
Future challenges involve practical evaluation, and standardization, of the
various solutions. It would, e.g., be necessary to develop vocabularies to express
various types of headers, and trailing triples. It is also necessary to study the
efciency of the various solutions empirically, to ensure that low latency, necessary
in a number of domains [
The various solutions can also be combined in various ways. As an example, domains requiring near real-time execution could bene t from event object decomposition that optimizes the stream for the most common type of query, even if this solution alone does not provide support for event object boundaries.