Future Ubiquitous Sensor Networks (USNs) are expected to sense and combine multiple descriptions of user contexts, providing a huge potential for the development of revolutionary context-aware applications bridging the physical and digital worlds. Current Sensor Web initiatives aim to support ubiquitous access to sensor networks, but to make them really useful for context-aware applications there is still a strong need to develop new mechanisms for enriching sensor data with high-level information, which would constitute a step towards Semantic Sensor Networks. In this paper, state-charts technology, described according to the W3C SCXML language, combined with the functionalities arising from Semantic Web technologies, are proposed as an efficient, simple and flexible means to make progress towards these objectives. Working with Sensor Web data, we propose SCXML-based semantic processing to produce high-level information through automatic embedding of semantic annotations, as well as the identification of relevant contexts. Both of these approaches are useful in a wide range of application domains. The capabilities and limitations of an SCXML processing architecture are also discussed in the context of its implementation over an OSGi framework and its integration into an experimental Telco USN Platform based on OGC® Sensor Web Enablement guidelines.
Many efforts on the Internet are currently focusing on a Content-Centric Future
Internet, where users not only consume but also create (prosumers) logical data as text,
audio and video [
Up to now, the challenges for the integration of real world information into the
digital world have been mainly addressed in two different research fields: SANs and
Context-aware Services [
Semantic Web technologies, as an evolving extension of the current Web, are
intended to provide well-defined and machine accessible information, shareable across
applications. Based on Ontologies, or formal descriptions of concepts and their
relationships, applications may benefit from a shared semantic representation of sensor
data [
Using the Semantic Web to progress from Sensor Frameworks to Context
Frameworks is not new It has been proposed in the recent Semantic Sensor Web initiative
[
Several approaches have already been proposed to carry out these two tasks
automatically. The Semantic Web Architecture for Sensor Networks (SWASN), presented
in [
The motivation to use state-chart machines and the W3C SCXML language can be
summarized in the following main points:
• Beyond conventional finite-state machines, which are inappropriate for the
behavioral description of complex control systems, state-charts can aggregate multiple
inputs or process them individually rather straight-forwardly. As introduced by
Harel [
W3C [
The rest of the paper is organized as follows: Section 2 explores SCXML’s
capabilities to semantically annotate sensor data, process it and identify contextual
situations; an in-car driving scenario is used as an example. In Section 3 we put these
possibilities into practice, defining a high-level architecture where a two-layer
SCXML processing strategy is used to generate semantic and contextual information
from Sensor Web data. Section 4 presents how the proposed processing architecture
has been implemented over an OSGi framework and integrated into a pilot Telco
USN Platform based on the guidelines offered by the OGC® Sensor Web Enablement
initiative [
In this paper we propose the use of state-charts, as implemented in W3C SCXML
language [
SCXML is a general language for control processes, but the addition of several functionalities beyond conventional finite-state machines makes SCXML suitable for integrating different sources of information rather straightforwardly. SCXML allows the definition of state-charts hierarchically, so a state-chart can itself be a state within another chart. Depending on the associated semantics, this construction can be interpreted either as a functional aggregation of charts, a compositional or dependency network, or as establishing generalization (i.e. subsumption and/or implication) relations between states. Also, several state-charts can run in parallel (i.e. simultaneous states). States that can be active at the same time are said to be functionally disjoint (generally independent), while those sharing a common parent but which run sequentially are, in general, mutually exclusive.
Just as for finite-state machines, in SCXML relations between states are mainly stated as transitions, which serve to describe an entire network of dependencies across the state-chart. Transitions can be defined in different ways. The simplest transition in SCXML is direct implication ( p ⇒ q ): the transition to the next state q is taken as soon as the first p is entered. Nevertheless, transitions in SCXML are almost always driven by events, and sometimes controlled via guard-conditions. In both cases a similar interpretation holds: the direct implication clause is replaced by a conditional implication, which depends on the actual state p, the guard-condition c, the expected event e, or a combination of both (i.e. p ∧ c ⇒ q , p ∧ e ⇒ q , p ∧ e ∧ c ⇒ q). SCXML can also define when events are fired based on the evaluation of a set of conditions ai (various combinations could be possible), thus introducing an additional relation ( a1 ∧ a2 ∧ … ⇒ e ) between states. These processing capabilities make SCXML suitable to define deductive analysis over sensor data and events producing high-level interpretations or the identification of particular real-world situations.
The semantic interpretation of raw sensed data is also usually based on decisions depending on past events and facts. Thus, it is necessary to support the logging of past information to support this kind of processing. Additionally, this allows tracing the evolution of states and measures. SCXML supplies a history pseudo-state which represents the state configuration in which a certain state was in the last time the machine transitioned from it. A transition with this history pseudo-state as its target is in fact a transition to the set of descendant states, i.e. those that were active the last time this state was left. This allows designers to recover a previous configuration, and, furthermore, to dynamically re-design the flow or analyze unexpected results. SCXML also allows defining probabilities for state transitions, so some statistical modelling is possible for dealing with sensor data uncertainty.
In addition to these processing capabilities, SCXML introduces a <datamodel> element to allow a uniform access method to both local and remote data. The <datamodel> can encapsulate any number of <data> elements, expressed in a XML, defining variables fetched from an external source or specified in-line. This data model can be used both to evaluate guard conditions – when a transition in the machine is required – and to save data when a new state is entered.
Turning back to guard conditions, these are defined as conditional expressions evaluated either when a transition event is triggered or just after the state is entered. The set of operators in these conditional expressions varies, including numeric quantifiers and logical operators, stated in the data model. Logical operators including less than, equal to or greater than can be used, which extend the expressive possibilities of propositions.
The properties and functionalities of SCXML just described allow the use of rules of logic for semantic processing. Accordingly, as long as programming is consistent and structured, it is possible to annotate and process data, as well as to identify particular situations, following the deterministic, deductive flow specified in SCXML. In Figure 1 several state-charts graphs are used to illustrate how input from sensor data (accessed through different interfaces to Sensor Networks) can be processed towards other state-charts for further, higher-level processing. This hierarchical and successive processing and combination of information between states can end in a “stable state” describing or representing a particular context or real-world situation. The resulting information can be regarded as flowing from new “virtual sensors” generated by the Semantic Sensor Networks. It is worth noting that state-charts can also be used to control actuators or actuation processes, but we have not addressed this aspect in this paper.
A context-aware application for an in-car driving scenario will serve as a motivating example to illustrate the possibilities of annotating data and identifying higher semantic situations using SCXML. In the driving context a great variety of heterogeneous data sources are available, some coming from multiple in-vehicle sensors (e.g. vehicle conditions, road conditions, the driver’s state of attention and ability, etc.), others provided by “external” Sensor Webs (traffic, weather information, etc.). The integration of both types of sources constitutes a true challenge. As discussed in Section 1, the designer of an in-car HMI can take advantage of all those pieces of information and become a prosumer of Semantic Sensor Networks. as he/she can both consume traffic information (e.g. suggest a different travel plan to the driver), and at the same time produce semantic data by offering the particular car travel plan chosen for external traffic modelling. This kind of information is considered to be vital for future traffic management systems.
The literature identifies three independent domains describing the global context
for driver-vehicle interaction: driver, vehicle and environment [
To annotate the collected sensor data we have followed the Sensor Web
Enablement initiative guidelines [
In our in-vehicle scenario, when a certain feature changes a sensor, it fires an event towards the context-aware application that is subscribed to it. This event allows the associated sensor data, encoded using O&M, to be processed through the SCXML execution environment to update the state machines representing the driver, vehicle and environment contexts. Each one of these domains can contain other domains (and therefore other state machines) which are paired with the sensors defined in the context. Thus, for instance, the vehicle domain could simply be described by a velocity state machine and a steering wheel angle state machine, and the environment domain by a traffic road state machine and a weather state machine. As the XML example below illustrates, this structure for the driver, vehicle and environment contexts can be easily defined using the SCXML <parallel> element. Note that each machine has to be subscribed to events coming from those sensors related to the local context, so only the data associated to relevant events has to be handled.
<parallel id=”context”> <parallel id=”driver/> <parallel id=”vehicle”> <state id=”velocity”/> <state id=”steeringWheelAngle”/> </parallel> <parallel id=”environment”> <state id=”trafficRoad”/> <parallel id=”weatherConditions”> <state id=”speedWind”/> <state id=”precipitation”/> </parallel> <state id=”roadType”/> </parallel> </parallel>
We now show how a particular state can be defined as another state-chart. Taking, for example, the previous precipitation state, the following SCXML code shows how it can be defined by a new state machine using a set of states and transitions driven by a precipitation.change event. <state id=”precipitation”> <state id=”weakPrecipitation”>
<invoke targettype=”semanticAnnotation” src=”weakPrecipitation”/> </state> <state id=”moderatePrecipitation”>
<invoke targettype=”semanticAnnotation” src=”moderatePrecipitation”/> </state> <state id=”heavyPrecipitation”>
<invoke targettype=”semanticAnnotation” src=”heavyPrecipitation”/> </state> <state id=”torrentialPrecipitation”>
<invoke targettype=”semanticAnnotation” src=”torrentialPrecipitation”/> </state> <transition event=”precipitation.change” cond=”precipitation lt 5” target=”weakPrecipitation”/> <transition event=”precipitation.change” cond=”precipitation gt 5 and lt 20” target=”moderatePrecipitation”/> <transition event=”precipitation.change” cond=”precipitation gt 20 and lt 45” target=”heavyPrecipitation”/> <transition event=”precipitation.change” cond=”precipitation gt 45” target=”torrentialPrecipitation”/> </state>
In this simple example we can also see how the state machine, using the cond
attribute (guard condition), checks the precipitation value to transition to the
corresponding state (weak, moderate, heavy or torrential precipitation), which will be a
more precise description of the actual precipitation state (this is equivalent to the
subsumption construction described in subsection 2.1). This simple semantic
interpretation can be used to enrich raw precipitation measurements by linking semantic
information to them. In the SCXML code above this is done through the <invoke> tag
that executes an external semantic annotation process each time a particular state is
entered. Besides the control flow, using the <invoke> element, SCXML can invoke
external processes. In our approach the external process is invoked to annotate the
raw O&M sensor data with semantic information using a domain-specific ontology.
We use RDFa (or Resource Description Framework –in- attributes) to embed
semantic annotations into XML data. RDFa [
rdf:about=”http://.../weather-ont.daml#windSpeed”> <swe:uom code=”mph” rdf:about=”http://.../weather-ont.daml#mphSpeed”> <swe:value> 90.0 </swe:value>
</swe:Quantity> </swe:field> </swe:DataRecord> <swe:DataRecord> <swe:field name=”precipitation”> <swe:Quantity definition=”urn:ogc:def:property:OGC:1.0:precipitation” rdf:about=”http://.../weather-ont.daml#precipitation> <swe:uom code=”mm/h” rdf:about:=”http://.../uom.owl#mmph/> <swe:value> 40 </swe:value> </swe:field> </swe:DataRecord>
Once sensor data is semantically enriched, the SCXML data model provides a standard way to store it and share it with other state machines. In the precipitation example, once the RDF annotation process is completed, an event could be fired to announce its availability so that different state-charts may use it. On top of a first annotation level, other processes may be defined, which, based on higher levels of abstraction, could detect specific situations useful for context-aware applications. In the following example, precipitation and windSpeed are used to define a higher level domain of WeatherConditions. <state id=”normalWeatherConditions”> <transition cond=”Data(environment, weather/windSpeed) eq ‘strong’ and Data(environment, weather/precipitation) eq ‘heavy’” target=”adverseWeatherConditions” /> </state> <state id=”adverseWeatherConditions”>
<send event=”adverseWeatherConditions.situation”/> </state>
This upper domain is composed of two states representing two different and mutually exclusive situations: normalWeatherConditions and adverseWeatherConditions. The state machine will remain in the normalWeatherConditions state until the transition condition is fulfilled. Just before this, the lowest domains will update the speedWind and precipitation fields in the data model sending an event to the upper domains signalling a change in one of its variables. Then, when the guard condition is met, a transition to the adverseWeatherConditions state occurs, causing this new situation to be annotated in the data model, as well as the firing of an update event (for the sake of simplicity, the invoke tag to the annotation process is not shown in the example).
Finally, we may identify meaningful situations for context-aware applications by integrating information from different domain levels. The following SCXML fragment shows an example where environmental information (weatherConditions and roadType) together with vehicle (velocity) information are used to define a dangerous driving situation and generate the corresponding warning event. <state id=”normalDriving”> <transition cond=”Data(environment, weatherConditions) eq ‘adverse’ and Data(environment, roadType) eq ‘lane’ and Data(vehicle, velocity) eq ‘excesive’” target=”dangerousDriving” /> </state> <state id=”dangerousDriving”>
<send event=”dangerousDriving.situation”/> </state>
Based on the SCXML capabilities described in the previous Section, Figure 2 represents a two-layer architecture for using SCXML to process sensor data (Sensor Web) and provide semantic annotations and contexts to both Semantic Sensor Networks and Context-Aware applications.
Beginning with our lower layer, the Domain-Specific Sensor Annotation layer (DSSA), heterogeneous sources of sensor data (Sensor Web) are processed by domain-specific SCXML machines. In our previous example (Section 2.1) the DSSA included the driver, vehicle and environment state machines. These SCXML machines have to be designed relying on local ontologies, so they can semantically annotate events related to specific sensor data to which they are subscribed. In a second layer, the Semantic Sensor Processing layer (SSP), the functionalities supplied by SCXML are used to process information from different local domains, generating higher levels of semantic information. Going back to the example in Section 2.1, the SSP domain would include the higher semantic level of weatherConditions. Also, just as in the lower DSSA layer, the state-char machines in the SSP layer have to be designed on top of specific ontologies, although at this level they must represent more global or wider domains. The SSP layer can provide three main main types of outcome: 1. Semantically annotated sensors: As the SSP layer processes sensor data already annotated in the DSSA layer, one possible outcome in the SSP layer can be just to extend the already semantically embedded information in DSSA with higher-level semantics from SSP. 2. Semantically annotated virtual sensor data: As SSP can integrate or process semantic sensor data from different local domains at the DSSA level, it can produce new information related to the real-world. This information can be regarded as data from a virtual sensor. 3. Detection of specific situations: Driven by the information from the DSSA layer, the particular states of the SCXML machines in the SSP layer can be interpreted as specific higher-level situations derived from local domains. Thus, certain meaningful patterns can be associated with the actual situation.
As shown in Figure 2, semantic annotation in the DSSA and SSP layers (outcomes 1 & 2) can be regarded as a mechanism for exposing sensor data from the Sensor Web to the Semantic Sensor Web. The detection of context and interpretation of the situation, provides valuable high-level information that can be fed to context-aware applications. It is worth pointing out that the annotated semantic sensor data could also be useful for context-aware applications that develop their own strategies based on semantic queries, reasoning or inference engines.
We produced an experimental implementation of our SCXML processing
architecture in order to perform an exploratory analysis of its capabilities and limitations. Our
implementation ran on an OSGi framework, and was then integrated into a pilot Telco
USN Platform based on OGC® Sensor Web Enablement initiative (SWE) guidelines
[
The Open Service Gateway Initiative (OSGi) is a consortium of technology to create
open specifications for facilitating the interoperability of applications and services
over a variety of networked devices in homes, cars and other environments. OSGi is a
service platform for the Java programming language that allows applications to be
developed from small, reusable and collaborative components called bundles.
Our SCXML processing architecture was implemented with the 3.4 release of the
OSGi platform [
As it was originally designed to support networked devices in homes, cars, etc., our
implementation of the SCXML architecture on an OSGi platform is a highly efficient
solution for closed or “local” environments, but it is more limited for wide open or
“distributed” Sensor Web scenarios. Furthermore, our IVIS application domain is
currently being developed in the context of the connected cars of the future. In this
scenario, in-car sensor data has to be combined with Sensor Web information (traffic,
weather information, etc.). For this purpose our OSGi SCXML semantic processing
architecture was integrated into Telefónica’s experimental USN Platform based on
OGC® Sensor Web Enablement (SWE) initiative guidelines [
Telefónica USN [
• USN-Gateways are the entry points for different sensor networks to the all-IP core network. Two main situations are addressed: a) sensor networks already connected to the Internet, as Sensor Web infrastructures; and b) non-IP sensor • networks using proprietary technologies (such as ZigBee). In both cases USNGateways, as functional components of Telefónica USN, guarantee both unified information (i.e. sensor data) modelling, and communication protocols for heterogeneous Sensor Network infrastructures.
The USN Service Enabler assists different applications and services in gaining
access to the information provided by ubiquitous Sensor Networks. Following
Service Oriented Architecture (SOA) principles, the USN Enabler was designed
as a Sensor Networks service enabler in a Service Delivery Platform (SDP),
where IP-based telecommunications can be accessed by third-party service
providers with information technology (IT). In particular, the Telefónica USN
Enabler was developed according to the common abstraction layer OMA Service
Environment (OSE), specified by the Open Mobile Alliance (OMA) [
For information modelling, as in OGC® SWE [
O&M language) are stored to allow subsequent retrieval or information extraction. • Data Processing: this functionality provides those capabilities needed to implement processing and filtering operations on sensor data.
Also relevant to our work is the development of Publish/Subscribe mechanisms (based on both SIP and HTTP) that allow services to subscribe not just to the observations provided by sensors but also to complex conditions involving other sensors and previous observations. We decided to implement both SIP and HTTP protocols in order to allow the development of convergent Telco/IT services in an all-IP environment.
As discussed in Section 1, one of our motivations for using SCXML was its
successful application to service orchestration in Service Delivery Platforms (SDPs) [
Our SCXML OSGi implementation was integrated into the Telefónica USN Platform in a similar fashion. As depicted in Figure 4, the SCXML semantic processing was integrated as a natural extension of the OGC® Data Processing entity. Several implementation issues were addressed: • First, the EventAdmin OSGi Service of the SCXML architecture had to be bridged to the USN Publish/Subscribe functionality, so that when a sensor notifies an observation to the USN Enabler, the corresponding event is communicated to the DSSA layer, thus driving the SCXML semantic processing (as described in Section 3). In this adaptation mechanism also the USN sensor data, modelled using O&M language (and possibly from the Observation Storage Entity), was stored into the proper SCXML data model. • The semantic annotation process was also extended so that when the representation update was completed, the enriched sensor data (RDFa annotated), stored in the SCXML data model, were moved back to the Observation Storage entity. In that way annotated data can be shared with third party applications through the USN Publish/Subscribe entity. Consequently, not only sensor data but also semantic links between data and formal (and often universal) descriptions based on ontologies are published. Thus it can be said that the USN Platform becomes a Semantic Sensor Network Platform. • A final implementation issue is related to the publishing of high-level real-world contexts or situations obtained as outcomes of the SCXML processing in the SSP layer (as described in Section 3). A new OSGi communication bundle was implemented so these situations or contexts could be published as “virtual sensors” to interested applications using the USN Publish/Subscribe entity. This new publishing function was implemented as an external process that could be invoked inside SCXML states.
To illustrate the integration protocol between the SCXML processing architecture and the USN Platform, Figure 5 presents a call flow where a sensor event is notified to the SCXML framework (already subscribed to this event) and both semantic annotation and context identification are performed and notified to a given application.
The flow in Figure 5 starts when the SCXML processing component receives a sensor event notification from the USN Publish/Subscribe entity. The DSSA layer enriches the received O&M sensor data by embedding semantic annotations and stores them into the Observation Storage entity. Then, through Publish/Subscribe, the availability of this new information is notified to the interested application (already subscribed to it). Finally, changes in the lower DSSA layer induce the SSP layer statechart machines to move so that they are able to detect relevant contexts. Once again, the Observation Storage entity is updated and, through USN Publish/Subscribe, the new context information is notified to the same context-aware application.
In this paper we have considered the use of SCXML as a possible semantic processing component contributing to bridge the gap between Sensor Web and Semantic Sensor Networks. SCXML capabilities have been reviewed to explore its possibilities for processing sensor data and providing high-level semantics for context-aware applications. A two-layer SCXML processing architecture has been presented. In the first layer (DSSA) raw sensor data is semantically annotated using domain-specific SCXML machines relying on local ontologies, while in the second layer (SSP) information from different local domains are combined to generate higher levels of semantic information. The implementation of the proposed SCXML architecture over an OSGi framework has also been presented, as well as its integration into a Telco USN Platform based on OGC® Sensor Web Enablement initiative (SWE) guidelines. This has allowed us to explore and present some experimental experiences on the capabilities and limitations of our SCXML semantic processing architecture acting on wide Sensor Web environments.
Based on these analyses and experimental results, future work will address the main limitations we have found with our practical implementations: • Even though SCXML provides high flexibility and rapid development for moderate sensor data processing, combining multiple sources of information becomes a tough task when the number of sources and the amount of information is potentially very big. This limitation to both managing and designing is particularly important since no powerful graphic development environment is yet available. • The semantic annotation process needs to be improved. Its efficiency could be improved by exploring the latest SCXML draft on using JSON (ECMAScript) for binding expressions for access to the data model. New mechanisms should also be developed to assist in the mapping between ontologies and SCXML machines or particular states. The modelling power of SCXML could be greatly improved if it were possible to associate metadata on states and events. • Specific improvements are needed to avoid redundancies in the two main mapping processes: between the OSGi EventAdmin Service and the USN Publish/Subscribe entity, and between the SCXML data model and the Observation Storage entity. • Finally, besides the good performance of the SCXML execution environment, scalability and latency demand further analysis and research efforts. Latency, mainly when processing distributed sensor data from the USN platform (Sensor Web), has been shown to impact the event-driven SCXML processing very negatively. Therefore, specific time-synchronization mechanisms should be sought. Execution times could be reduced by applying SCXML script compilation and by efficient caching of these scripts. Also compact binary representations for the SCXML data model should be considered. Scalability is therefore extremely important for this kind of infrastructures, as well as the granting access to Semantic Sensor Networks both to retrieve and share information.
The activities described in this paper were funded by the Spanish Ministry of Science and Technology as part of the TEC2009-14719-C02-02 project.