The recent years have seen a vast increase in the amount of environmental sensor data that is being published on the web. Semantic enrichment of sensor data addresses the problems of (re-)use, integration, and discovery. A critical issue is how to generate semantic sensor data from existing data sources. In this paper, we present our approach to semantically augment an existing sensor data infrastructure, in which data is published via a RESTful API as inter-linked json documents. In particular, we describe and discuss the use of ontologies and the design and development of seraw, a system that transforms a set of json documents into an rdf graph augmented with links to other resources in the Linked Open Data cloud. This transformation is based on user-provided mappings and supported by a library of purpose-built functions. We discuss lessons learned during development and outline remaining open problems.
Environmental sensors play an important role in applications such as weather
forecasting, bush re monitoring, and irrigation scheduling. Recent years have
seen a vast increase in the amount of environmental sensor data that is being
published on the web. To make the most of the available data, it is
important to enhance our ability to discover, integrate, and contextualise the data.
Standardization e orts like the Open Geospatial Consortium (ogc) Sensor Web
Enablement (SWE) [
The Commonwealth Scienti c and Industrial Research Organisation (csiro)
collects and archives data from a large number of terrestrial and marine sensors
that measure parameters such as rainfall, temperature, and salinity. Parts of the
data are made available via an in-house sensor data infrastructure, referred to as
Sensor Cloud. One of the main components of this infrastructure is a Web API
implemented following RESTful principles [
To overcome these limitations, we propose a conceptual framework to
semantically augment the Sensor Cloud. Generating semantic sensor data from
our existing Sensor Cloud infrastructure involves several steps. First, we model
sensor data through the Sensor Cloud Ontology (SCO), which extends
existing ontologies including the Semantic Sensor Network Ontology (SSNO) [
In this paper we discuss the main components of our framework to semantically augment the Sensor Cloud, named Semantic Sensor Cloud and in this context we present our approach to generate RDF from JSON, implemented in SERAW. Given that there is a widespread use of APIs that serve data as json documents (e. g., geospatial data 2), we believe that this approach can be generalised to other applications e. g., the API provided by the centralised sensor data repository Xively 3.
In summary, the main contributions of this paper are: { We create an ontology for the Sensor Cloud, which extends standard ontologies to model environmental sensor data. 1 http://www.json.org 2 http://www.opengeospatial.org/projects/groups/gservrestswg 3 https://xively.com { We develop seraw, a system that transforms a set of inter-linked json documents into an rdf graph based on mappings between document elements and concepts and properties in our ontology. { We enhance the generated rdf graph with links to resources in the lod cloud through purpose-built functions.
The remainder of this paper is structured as follows: We start by describing our conceptual framework in Section 2. We present our ontology to model Sensor Cloud data in Section 3. In Section 4, we describe our mechanism for transforming a set of json documents into rdf. Implementation details are given in Section 5. Section 6 outlines how we augment the resulting rdf graph with external information. We discuss related work in Section 7. Section 8 concludes the paper and gives an outlook into future work. 2
In this section, we describe the conceptual framework for the Semantic Sensor
Cloud, which integrates semantics into the the existing Sensor Cloud. Figure 1
shows the main components of the framework, where the components on the
right side (shaded in gray colour) belong to the semantic enrichment part, while
the components on the left belong to the Sensor Cloud. Following the layered
architecture for web accessible sensor data in [
The Sensor Cloud is a sensor data infrastructure developed within csiro. The main goal is to provide a centralised point of access for environmental sensor data collected from various sensor networks in the organisation. Scalability and support for application development were the main drivers that in uenced design decisions. The infrastructure consists of a number of components. A detailed description of the infrastructure is beyond the scope of this paper. Here, we only give a brief description of the main components. The RESTful API (scapi) provides data access and manipulation. The data management component addresses how data is structured, identi ed, and partitioned into documents. Sensor Data Adapters transform data from external sources into the json format. json possesses several bene ts for application development, especially when developing web applications using JavaScript. MongoDB 4 is used as the data storage backend, primarily for its exibility regarding data structure.
Within the infrastructure, sensor data is virtually structured hierarchically, following N etwork ! P latf orm ! Sensor ! P henomenon ! Observation. This hierarchical structure resembles the deployment of sensors in the physical
Semantic Sensor Cloud
Data Management JSON
Mongo
DB
RESTful
API
SPARQL
SERAW Sensor
Data Adapters
LoD Cloud
SCO
RDF
Store
world. Each sensor network consists of platforms, with each platform having one
or more sensors attached; each sensor observes one or more phenomena. Besides
observations, information such as sensor device characteristics and calibration
history can also be accessed, which in the infrastructure is currently represented
using a json pro le of StarFL [
The hierarchical structure of the data is re ected by the urls used to access data. Networks, platforms, sensors, and phenomena have names to identify them. These names form a relative key structure. That is, the name of a network uniquely identi es it in the infrastructure. The name of a platform is unique among other platforms in the same network. There may, however, be a platform with the same name allocated to a di erent network. urls are currently the only way for external applications to access documents in the Sensor Cloud. An example for url http://www.sense-t.csiro.au/sensorcloud/v1/network/TIA/ platform/Tamar_Ridge/sensor/RIMCO_7499/phenomenon/rainfall is shown in Figure 2. The returned json document refers to the network \TIA", platform \Tamar Ridge", sensor \RIMCO 7499", and phenomenon \rainfall". The content of the le is determined by the data management component. In this example it shows, among other data, the location of the platform in terms of its coordinates, date and number of observations, and links to the related observations (time series) and to the sensing procedure.
The main goal of our work is to semantically augment the Sensor Cloud with knowledge management components that enable the automated generation of rdf from the sensor data. At the heart of our work is seraw, a system that transforms documents accessible via the sc-api into rdf. seraw makes use of an ontology developed as extension of existing ontologies particularly for the Sensor Cloud. By making use of a set of purpose-build functions, seraw allows to include references to external resources and information. The generated data is maintained in an rdf triple store and accessible via a SPARQL endpoint. Figure 3 shows a screen shot of the query interface for seraw. In the following sections, we describe components in the Semantic Sensor Cloud in more detail. 3
In order to semantically describe sensor data from the Sensor Cloud we created the Sensor Cloud Ontology (SCO)5. The principle behind the design of the ontology is to use and extend existing ontologies, meanwhile aligning with the Sensor Cloud terminologies. Accordingly, classes and properties were created and mapped to the ones in existing ontologies, as described below. The advantages are that sensor data can be queried according to the original terminologies while their consistency can be checked against SCO. We use the ontology to facilitate syntactic-to-semantic transformation in seraw (Section 4).
Following best practices, we reused several ontologies, namely, the SSN ontology (SSNO), an ontology for the OGC's Observation and Measurements (OM) 6, and the Basic Geo (WGS84) (GEO) vocabulary 7. The main criterion to use these
7 http://www.w3.org/2003/01/geo/wgs84 pos ontologies is that they are derived from standardisation e orts. Figure 4 shows the main classes and properties of SCO, where the pre x of a class, e. g., ssn of ssn:Observation, indicates the ontology the class comes from.
We extend these ontologies in several ways. In SCO we de ne sco:Network, sco:Platform, and sco:Sensor to make explicit the fact that networks, platforms, and sensors in the Sensor Cloud are ssn:Systems (contrary to the de nition of platforms and sensors in SSNO). sco:Sensor, for example, is de ned as subclass of ssn:SensingDevice which in turn is a subclass of ssn:System. To describe sensor observations, we introduce concepts sco:ObservedPhenomenon, sco:ObservationResult, and sco:TimeSeriesObservatedValue as subclasses of ssn:Observation, ssn:SensorOutput, and ssn:ObservationValue, respectively. These concepts re ect the particular use of terms phenomenon and obhasSensingDevice subclass subclass subclass subclass subclass sco:Network hasPlatform sco:Platform hasSensor sco:Sensor ssn:observedBy sco:Observed
Phenomenon ssn:observationResult sco:Observation
Result hasLocationCoordinate hasLocoantPiolantCfooromrdinate hasMetadata hasMetadata hasMetadata ssn:hasValue sco:Location Coordinate subclass geo:Point om:Metadata ssn:Observation
Value subclass sco:TimeSeries ObservedValue ssn: System ssn:Sensing
Device ssn:Observation ssn:Sensor Output servation in the hierarchical document structure of the Sensor Cloud. To be compatible with GEO and OM, we introduced sco:LocationCoordinate as a type of geo:Point, and used om:Metadata to describe metadata of several classes (e.g. sco:Sensor). In doing so, we are then able to use standardised (ISO) vocabularies for coordinates, deployment, and quality that are left open in SSNO. Furthermore, we introduced some properties that are speci c to the Sensor Cloud, e. g., those describing the number of time-value pairs of time series, and the rst or last observation time.
Besides the above ontologies, we also used existing vocabularies such as dcterms:source (for the url of the original json documents). In addition, we align with UCUM 8 for the units of measure, MUO 9 for the physical qualities, and Climate and Features 10 for the properties. This alignment is enabled by subclassing sco:ObservedProperty and dul:UnitOfMeasure accordingly.
The SCO data instances can be queried through the interface shown in Figure 3. The shown query returns longitude and latitude for all platforms in the Sensor Cloud. Another example is the following query that retrieves observation quality information represented as observation metadata.
10 http://www.w3.org/2005/Incubator/ssn/ssnx/cf/cf-property
SELECT DISTINCT ? sensingDevice ? observation ? obsMetadata ? qualityInfo ? methodType WHERE { ? sensingDevice a ssn : Sensor . ? sensingDevice sco : hasObservedPhenomenon ? observation . ? observation sco : hasMetadata ? obsMetadata . ? obsMetadata md : dataQualityInfo ? qualityInfo . ? qualityInfo dq : report ? report .
? report dq : evaluationMethodType ? methodType .
} 4
Within this section we describe how seraw generates an rdf representation for sensor data accessible via the sc-api. Our main goal is to create a single rdf graph from a set of inter-linked json documents. Transformation is guided by user-provided url expressions and transformation scripts. url expressions associate documents of di erent type (e. g., Network, Platform, Sensor) with transformation scripts. Each script then describes how document elements are mapped to concepts and properties in a given ontology, i. e., SCO in our case. For seraw we developed a simple mapping language to express transformation scripts. As with any formal language, there is a trade-o between functionality and complexity of use. Languages such as xslt are powerful but di cult to use, especially for domain experts with limited computer programming experience. Furthermore, most approaches focus on transforming a single document without considering the problem of creating and maintaining relationships between resources in di erent input documents. We start by de ning url expressions (Section 4.1) before describing our mapping language (Section 4.2). 4.1
urls in sc-api follow patterns that re ect the structure of data in the Sensor Cloud. seraw exploits these patterns by means of url expressions to match urls. The syntax of url expressions is similar to that of urls. For simplicity, we assume that a url is composed of two parts: a service identi er (a string consisting of protocol, server name, domain name, and optional port information), and a (potentially empty) array of strings, called path names. A url expression also has a service identi er, and a (potentially empty) array of strings which could be path variables or path names. A path variable is a regular expression. We reference variables from 0 to n based on the order of their occurrence. A url matches a url expression if the service identi ers are equal, the path arrays are of same size, and each path name in the url matches the corresponding element in the url expression. When a url matches an expression we bind the variables in the expression to the path names in the url. Figure 5 shows an example url expressions. Regular expressions are enclosed in square brackets and follow Java's regular expression syntax (e. g., `: ' matches any character sequence). Both urls http://sense-t.csiro.au/sensorcloud/v1/network/ TIA/platform/Tmar_Ridge and http://sense-t.csiro.au/sensorcloud/v1/ network/SouthEsk/platform/Ben_Lomond match the pattern in Figure 5. In the rst case, variable 0 is bound to path name TIA and variable 1 is bound to Tmar Ridge. We use a set of url expressions to identify relevant documents. Only documents having a url matching one of the given url expressions are considered by seraw. The use of url expressions gives us exibility in that we can restrict the set of documents that are transformed (down to transforming only a single document). Starting from a set of urls, seraw traverses the documents accessible via the sc-api by following links between them. To detect cycles in the document graph, we keep track of all urls visited. With each url expression we optionally associate a transformation script. For each document, we execute all transformation scripts that are associated to any url expressions matched by its url. Associating transformation scripts with url expressions allows us to handle each document type di erently. By executing translation scripts, we add individuals to the ontology, generating the rdf representation for the documents accessible via the sc-api. 4.2
Transformation scripts are programs in our mapping language. The language is
inspired by formal mapping languages (e. g., [
Transformation scripts take the document (i. e., a json object) and a list
of path names bound to variables in the matched url expression as input.
These path names are accessible within scripts by referencing the corresponding
variables using double square brackets. In our example, [[0]] and [[
Lines 3-8 process the elements under path platform/location. Note that we use a path that is relative to the json element identi ed by the surrounding forstatement. There might be multiple elements in a document that match a given path expression. The for-statement is executed for each of them. The statement in line 3 creates an instance of sco:LocationCoordinate which has datatype properties geo:lat, geo:long and geo:alt. The statements in line 5-8 create instances of these datatype properties. The pre x '@' indicates that the property value comes from the primitive json element identi ed by the respective relative path (e. g., latitude). An optional type information (e. g., ^^double) indicates how the value is represented in rdf. The statements in lines 9 and 11 are examples for creating instances of object properties. It is important to note that we may reference an individual before it is created. In line 11, for example, it is possible that the referenced sco:Network with identi er `Network TIA' has not already been created (i. e., it is not in the lookup table). If that is the case, we defer creation of references until referenced individuals have been created.
Implementation of SERAW seraw is implemented in Java and uses Jena API to generate rdf. We run seraw as a web service using Apache Tomcat. The crawler is a separate process within the system. It crawls documents accessible through the sc-api and transforms them into rdf. Execution times of seraw depend on the number and complexity of transformation scripts, as well as on network latency. Given the large number of documents accessible via sc-api, the overall time for transformation is several minutes. We conducted initial experiments to evaluate the impact of document access and transformation on overall execution time. Here, we focus on transforming documents representing sensing platforms using the script in Figure 6. All experiments were performed on a standard Dell Latitude laptop with a 2.53 GHz Intel Core i5 CPU and 4GB of RAM. All times are averaged over ten runs. There are a total of 74 platforms in sc-api with an average document size of 497 bytes. Our experimental results indicate that network latency (as expected) has the major in uence on execution times. The average time for accessing a document was 37.55 ms. Transforming a document, on the other hand, only took 1.02 ms on average.
seraw creates an rdf version of data accessible via the sc-api. Derived data raises the issue of timeliness. As data in the Sensor Cloud is updated, our rdf outdates over time. There is an obvious di erence between update frequencies of di erent document types. While sensor observations get updated frequently, sensor metadata and deployment information is less likely to be updated. Depending on the type of information that is transformed into rdf and the application requirements we choose the frequency with which we re-run the transformation process. When doing so, we overwrite results from the previous run. The user interface also allows the process to be run on demand to generate an up-to-date rdf version of the data.
The seraw user interface provides the ability to create and edit transformation scripts. It further provides the opportunity to test the results of created scripts on a restricted set of urls. Figure 7 shows part of the interface that allows a user to specify a set of urls and run the transformation scripts on the documents returned by these urls. The interface displays the transformation result and any error messages generated during the transformation. 6
The motivation for our work was not only to generate an rdf version of the data
in the Sensor Cloud, but also to augment the resulting rdf graph with external
information. As such we consider including links to external resources in the rdf
graph. Several authors have recently described approaches for publishing
semantic sensor data that includes links to resources in the lod cloud [
One example function is the generation of address information for a given location. The Sensor Cloud contains location information in form of latitude and longitude. We implemented a function, called revGeoCode, which uses Google's Geocoding API 12 to transform location coordinates into a humanreadable address. The API returns a list of results containing postal addresses and other geographical names for a location, with the most speci c one returned rst. Our implementation of revGeoCode returns the string value of the rst formatted addresses eld in the result list (if any), e. g., 619 Auburn Road, Kayena TAS 7270, Australia for the sensor platform at Tamar Ridge (Figure 7). 12 https://developers.google.com/maps/documentation/geocoding/ Other examples are functions that query DBpedia 13, Geonames 14, or Freebase 15 to retrieve a uri for a given location name. The following SPARQL query basically represents the function that queries DBpedia for location uris, called DBpLocation, where var is the location name provided as parameter with the function call:
SELECT ? city WHERE { ? city rdf : type <http :// dbpedia . org / class / yago / TownsInTasmania > .
FILTER ( regex (? city , var ) }
We further combined functionalities into a function DBpLatLonLocation that takes latitude and longitude values as input, retrieves a place name for the location from Google's Geocoding API (from the rst address component of type locality in the result), and uses the place name (if any) to retrieve a uri from DBpedia. We implemented similar functions for Geonames and Freebase, called GeoLatLonLocation and F BLatLonLocation respectively. Figure 8 outlines the two-step process of using latitude and longitude information to generate a link to a location resource.
In transformation scripts, we make function calls when generating datatype properties. For example, adding the following line to the script in Figure 6 (anywhere in between lines 4-8) will take values of latitude and longitude in the json element and add location information to the individual identi ed by y2.
We added location information for 56 of 74 platforms in the Sensor Cloud using DBpLatLonLocation. We also added information from Geonames and Freebase. DBpedia, Geonames, and Freebase give only general information about a particular place. This information, nevertheless, boosts up the knowledge that can be used by other applications (i.e., Agriculture Decision Support Systems). In the future, we will extend our library of functions, e. g., for retrieving soil types as provided by the European Environment Agency 16. We also plan to add functionality to enable linking to geographical resources by querying services such as the Geographical Service 17 and LinkedGeoData 18. 7
We designed seraw to complement existing Web APIs that publish data in json
format by providing semantic enhancements and query capabilities that are not
available otherwise. While we use the Sensor Cloud as our main motivation,
13 http://dbpedia.org
14 http://www.geonames.org
15 http://www.freebase.com
16 http://www.eea.europa.eu/data-and-maps/data/soil-type
17 http://geoservice.psi.enakting.org
18 http://linkedgeodata.org
Fig. 8. Function to Retrieve Location uri based on Latitude and Longitude
Information
the transformation mechanism described in this paper is applicable to other
APIs. The semantic sensor web [
In [
seraw has an extendable set of purpose-built functions for creating links to the lod cloud. Having a library of functions facilitates reuse of functionality. We see our approach towards link generation as a step in the right direction that hopefully will nd adoption in other systems. 8
In this paper we explored ways for transforming sensor data available via an existing data infrastructure as a semantically enriched rdf dataset. We provide Knowledge Management components that access components of our existing Sensor Cloud and generate rdf that can be queried via a SPARQL endpoint or re-used by applications for di erent purposes. The requirements for semantic web technologies within a sensor data infrastructure can vary according to the architecture level ranging from sensor devices to analytics to applications. We believe that seraw is a generic tool that can be used across di erent levels in such infrastructure since it allows mappings rules and services to be combined.
One area of future work is the development of a more sophisticated solution to update our rdf copy of the Sensor Cloud data. Instead of transforming the whole set of documents, the goal is to transform only those that have been modi ed recently. We are able to identify documents that have been modi ed from the log les of the web server running the Sensor Cloud. We can then selectively transform only these documents. To do so requires to keep track of provenance in the rdf graph to identify those parts of the graph that where derived from a particular json document.
A similar problem arises with the generation of links to external resources. We generate links to external resources every time we run the transformation process. For functions that use information about the location of a platform, for example, the results are likely to be the same in every run (unless the platform is a mobile platform). One path of future work is to cache results from calls to external services. Provenance, again, plays an important role here, i. e., keeping track of parameter values (e. g., latitude and longitude) to determine whether a document has change in between runs or not. 19 http://triplify.org/Overview 20 http://d2rq.org
The authors would like to thank the development team of the Sensor Cloud for their support. In particular, we would like to thank Chris Peters, Chris Sharman, and Peter Taylor for their assistance and feedback during the development of seraw. We would also like to thank Michael Compton for his valuable feedback during preparation of the manuscript.
The Intelligent Sensing and Systems Laboratory and the Tasmanian node of the Australian Centre for Broadband Innovation is assisted by a grant from the Tasmanian Government which is administered by the Tasmanian Department of Economic Development, Tourism and the Arts.