In this demo we present the Trusted Tiny Things system that can be used to interrogate Internet of Things (IoT) devices and present users with information about their characteristics and capabilities. The system consists of a mobile application used to retrieve information about IoT devices supported by RESTful web services. In order to infer IoT device capabilities our services perform reasoning over the provenance of devices characterised using a number of Semantic Web technologies. In this demo we illustrate the use of the system with two distinct IoT devices: an NFC tag used at bus stops to provide a means to access real-time bus timetables, and a black box device installed into vehicles by insurance companies to track driving behaviour.
The vision of the Internet of Things (IoT) is a dynamic global network based on
standard and interoperable communication protocols where physical and virtual
`things' have identities, physical attributes, and capabilities and are seamlessly
integrated into the existing internet infrastructure. The IoT is thus built upon
a range of sensors and other devices [
Questions that a user might ask include: What kind of data does the thing
collect? Is the data transmitted? If so, how and to whom? For what purposes are
the data used? What control do I have over any aspects related to the generation
and use of this data?. These questions are re ected in the \TRUSTe Internet of
Things Privacy Index - GB Edition1" study where more than 80% of the 2,005
people interviewed were concerned about such issues. Our proposed solution is
largely based on participatory design activities we conducted during the course of
the project. To date, we have conducted a number of participatory design events
involving a total of 77 participants with di erent technological backgrounds[
These questions are particularly relevant in the context of Smart City
development. Bartoli et al.[
The Trusted Tiny Things (T3) project is investigating how Semantic Web
technologies can be used to describe the context surrounding IoT devices (e.g.
manufacturer, owner, security method) and to reason about device capabilities.
As 'things' become more interconnected this context should also include
provenance information: a record of the entities (devices or services) and processes
(data transmission, data analysis, decision making) involved in the creation and
use of data. A formal representation of provenance has been identi ed as
essential to support users (and machines) to better understand and trust data[
In the remainder of this paper we discuss our T3 software system and the supporting semantic framework. We conclude by presenting a list of features being demonstrated about the system through the use of two case studies (incar black box and bus stop).
We have developed a software infrastructure and mobile app (see Figure 2) that can be used to query, update and register IoT devices and to notify the user of any changes in the capabilities of a registered device. 2.1
There are two user groups concerned with usage of our system. { Device owners and manufacturers - this user group is concerned with registering IoT device into the system and updating device characteristics and provenance records through our Restful API according to our guidelines2. { Users of the IoT system (devices and services) - These are the users of the IoT device and associated services and they interact with our system through NFC-enabled Android mobile application. 2.2
The system is composed of multiple layers (see Figure 1): { Storage layer - it is used for storing and retrieving device metadata, provenance record and information about users. We are using two di erent Jena TDB3 repositories. One repository holds publicly available provenance of IoT devices. The other repository holds a record of user preferences and accepted devices. We also utilise MySql Database to store con dential user data for increased security(e.g smartphone IDs used to identify users, associations with the IoT devices and statistics how users are using our system). { Ontological layer - it contains a number of ontologies used to support the metadata in the Storage layer. These ontologies are discussed in Section 3. { Service Layer - it de nes methods for querying, updating, and synchronising data from the devices. It is also capable of notifying users of any changes in an IoT device or service (e.g. a new organisation is using information provided by an existing IoT device). { RestFul API Service Layer - it provides uniform access to the system from external applications such us our Trusted Tiny Things mobile app. 2.3
The system consists of ve core Java EE based services:
Service Layer Ontology Layer Storage Layer
RESTful API Services Notification
Capability Reasoning
Register T3 Annotations and Capabilities
W3C PROV
Jena TDB
IoT Device Provenance User Accepted Capabilities and
Preferences
Update
IoTA
MySQL Smartphone IDs and Device Capabilities
Query FOAF { The Registration service is used to register new devices into the system. It provides methods for generating links between devices and the physical entities they represent using the IoTa domain model. When user registers device he/she is given unique ID, which should be used over the whole IoT system to send generated or intended provenance data to our system. For example, if the device is communicating with the server and server is acting upon the data that was collected from IoT device, provenance of the transactions or processes that were carried out externally should be captured and send to our system. { The Update service provides methods for updating information about devices in the system. This includes the ability to update the provenance of the device and its use. On how to update provenance and other IoT device information reader can refer to our guidelines. This service also provides a method for associating smartphone IDs and devices when users accept (or decline) the capabilities of the device via our app. { The Capability Reasoning and Filtering service is designed to reason about the provenance information associated with devices in order to infer direct or indirect capabilities. This service makes use of a rule-based reasoner implemented using the TopBraid SPIN API4 to evaluate rules of the kind described in Section 3.1. Since the smart devices, their services and third-party smart application can generate vast amount of provenance metadata, it was necessary to implement an e cient mechanism to lter redundant repeated capabilities that are already being recorded. For example, public security camera is capable of motion detection. For each motion event activation the camera would generate provenance, which would be sent to our system. A user may only wish to know that camera has motion capabilities. { The Query service is used for extracting information about an IoT device based on the metadata stored in our system (including provenance and capability inferences). 4 http://topbraid.org/spin/api/ { The Noti cation service checks for changes in the capabilities of registered devices every hour. If the service detects such changes push noti cation messages are sent to the relevant users (using the smartphone IDs associated with the device) informing them of the new capabilities. 2.4
The services described above are accessible from our mobile app via a RESTful API layer and JSON5 is used as the data interchange format. Devices in our API are recognised by a custom URL http://t3domain/devices/fDeviceIDg where the DeviceID represents the device identi er encoded in our Trusted Thing NFC tag. Using the devices URL we can support di erent types of GET and POST actions to retrieve or create information about devices. The app listens for NFC events and intercepts the signal in order to determine if the device was registered with our system. If so, it would retrieve characteristics and list of inferred capabilities derived from past behavior of the device. The app also summarizes in pictorial representation all the organizations associated with it and the type of data that are being collected. If user is happy with the capabilities, he/she has the option to either Accept or Decline the use of the smart device or service associated with it. If accepted, user can store the device under his/hers own nickname so the user can reference to it in the future. Once user accepted the device capabilities, he/she also subscribes in our system to be noti ed about any changes in capabilities (e.g. capability changes or new capability is detected).
In order to inform the design of a semantic framework for IoT devices we have conducted three participatory design events involving a total of 14 participants with di erent technological backgrounds. Participants were asked to discuss issues surrounding the capabilities of IoT devices. Questions were posed such as: What do you think are the capabilities of this device? and What kind of capabilities would you want to be aware of before interacting with this kind of device?. Guided by user requirements we have designed an ontological framework to support inferences about device capabilities using provenance.
In order to retrieve information about IoT devices (characteristics, provenance, capabilities, etc.) it is necessary to be able to identify things (e.g. in car black box,smart fridge) and their IoT components (tag, device, sensor or service).
Bandara et al. [
The Internet of Things Architecture6(IoTa) is another project working
towards building a common architecture for the future Internet of Things. They
have developed a conceptual model [
In order to reason about device capabilities we need to be able to represent the provenance of an IoT device (e.g. processes, agents and the data being used and generated). In our model we describe such provenance using the emerging W3C PROV-O7 ontology as it is designed to be applicable to a wide range of applications and domains. PROV-O de nes concepts such as: prov:Entity (physical, digital, conceptual); prov:Activity (something that occurs over a period of time and acts upon or with entities.); and prov:Agent (something that bears some form of responsibility for an activity). Using this ontology it is possible to describe who is the agent responsible for a speci c process (prov:Activity ) taking place (e.g. who collects my position location data) or what information
iota:represents iota:VirtualEntity ComIpooTnent “BlackBox Controller
Implementation” iota:Sensor “3G Transmitter” iota:isAssociatedWith iota:Sensor “GPS Sensor” iota:User “Car Holder” iota:interactsWith iota:PhysicalEntity “INSURANCE HOLDER
CAR” iota:isAttachedTo iota:Device “BlackBox” iota:contains are being created generated (e.g. device generated information about the speed). We were able to align PROV-O with IoTa Domain Model so we can be more descriptive about IoT devices. Moreover by combination of these two semantic models we can better capture the creation,usage and ow of information in smart infrastructures.
Figure 4 shows an example of our framework being used to represent a in-car black box device. With the provenance support provided by PROV-O we are able to identify not only the high level concepts of IoT system, but speci c data and relationships between them. In this case there is a process (prov:Activity ) which used the GPS sensor to calculate current speed of vehicle and generated another prov:Entity containing calculated information about driver's speed. These information are then used by external process which calculates insurance premium. Note that both black box controller and insurance smart service acted on behalf of the same company. Participants during our design exercises highlighted the need to provide contact information about agents (individuals or organisations) responsible for certain devices and therefore we use the FOAF8 ontology. The class foaf:Organization was de ned as a subclass of prov:Agent. 3.1
As discussed earlier in this paper, PROV-O allows us to answer some of the issues related to transparency of IoT devices in smart environments, such as who is responsible for a device, what data is generated and where such data is transmitted. However, in PROV-O it is not possible to distinguish between personal and non-personal data, neither it is possible to describe the purpose behind the use of such data.
We designed an ontology (T3 ontology) to support two important aspects of the reasoning required to infer and detect device capabilities. Firstly,the ontology
prov:actedOnBehalfOf prov:SoftwareAgent
iota:Service “Insurance Company Smart
Service” iota:isAssociatedWith prov:wasAssociatedWith prov:wasGeneratedBy prov:Entity “New Premium”
prov:Activity “Calculate Insurance Premium” prov:used prov:Agent “Insurance Company” prov:Entity iota:Device “BlackBox” iota:contains prov:Entity iota:Sensor “GPS Sensor” prov:used
prov:actedOnBehalfOf iota:represents prov:SoftwareAgent ComIpooTnent iota:VirtualEntity “BlackBox Controller
Implementation” prov:wasAssociatedWith prov:Entity “40 km/h” prov:wasGeneratedBy
prov:Activity “Calculate Current Speed of Vehicle” provides annotations over PROV-O concepts capturing the kind of information identi ed by our participants. These annotations include: { The ttt:PersonalData class is used to identify if a prov:Entity represents data that can be personal to individual. Human-readable text of the data description can be attached to the prov:Entity via ttt:description property. { The ttt:purpose property is used to provide an human readable explanation of why certain entities (described as personal data) are being generated or used. This property is associated with prov:Activity (process,task) via prov:Usage class. { The ttt:Capability class de nes di erent kinds of capabilities (e.g. ttt:DataCollection, ttt:DataGeneration, ttt:DataSharing,ttt:DataGeneration) that can be associated with iota:Devices. These associations (described by the ttt:isCapableOf property) are made on the basis of a number of inference rules described later in this section. { Properties such as ttt:securityDescription, ttt:deviceDesription, ttt:deviceName were de ned to describe and further support transparency in human readable format.
Secondly, in order to infer the capabilities of IoT devices using our ontological framework we can associate rules to speci c classes of ttt:Capability. We make use of the SPIN ontology9 to support the use of SPARQL to specify rules and logical constraints necessary to reason about capabilities. The rule is designed to traverse a PROV-O provenance graph starting from an instance of an iota:Device. To date, we implemented ve inference rules, which are closely related to personal information: check if any IoT device or service is capable of generating automatic bills for a user, check if there are personal data being collected, rule to see if personal data are being used and why, detect if personal
data are shared with third-party organisations, and detect what data is the device or service generating. Implementation of some of these rules can be seen in Figure 5.
If any of the rule activates, it constructs new inferred facts about the device in questions and forms them into capabilities (subclasses of ttt:Capability ). If certain capability is already attached to the device (e.g. PersonalDataSharing) ltering service (see Section 2.3) makes sure it won't recreate the same one, unless there is new purpose or actor (e.g. there is a new agent the personal data are being shared with). In this case the same type of capability is attached to the device, but with changed properties.
If data are uniquely identi ed and the provenance of the devices and services associated with it is being captured and sent to our framework, system can track down how far certain information (e.g. personal data) ows in the whole IoT system or smart infrastructure. This will allow to identify all the agents, purpose and processes, that played any part in creation, generation, modi cation and aggregation of personally identi able data.
We have also implemented rules that are used to determine what kind of provenance has been used to infer a speci c device capability and if the capabilities are direct(on the device) or indirect(capability of external entities). However, this is out of the scope of this demo paper. 4
We organise this demo in two parts. The rst part demonstrates how our mobile app can be used to query and visualise provenance information about IoT devices in the bus stop scenario. The second part of our demonstration explores a scenario, in which provenance is generated and sent to our system in real-time from IoT devices. We demonstrate how a user is noti ed about changes when the capabilities of an associated device change. 4.1
Near Field Communication (NFC) tags are being deployed on the bus stops in Aberdeenshire, Scotland UK so that user can retrieve real-time timetable information. These passive IoT devices embed a URL that is used to redirect and NFC-enabled smartphone to a service providing timetable information for a speci c city location. Users may think that this service is provided by Aberdeenshire Council, but instead it delegates to third-party organisation (RSL Ltd). Moreover, both organisations are collecting certain kinds of personal data (e.g. IP Address, smartphone model and OS, and the version of the web browser users). Users deserve to know what data they are providing and to whom and they should also have an option to refuse to use the service. To this end, the Trusted Tiny Things tags and system have been deployed to more than 2300 bus stops in Aberdeenshire in 2014.
Demonstration content: { For the purpose of this demonstration we built a replica bus stop. The NFC tag attached to the bus stop will be scanned using an NFC-enabled smartphone. { We explain how the system checks if the device was previously accepted buy the owner of the smartphone. We demonstrate how the meta-data associated with the IoT device (bus-stop tag) and the meta-data associated with the timetable services is used in order to infer capabilities. { Using the mobile app, we show what companies are associated with the device, who is the owner, manufacturer and list of capabilities, which will be explorable by clicking on them to nd out further details about them. We will also demonstrate how the capability of the device can be accepted and recorded into the system. { Finally, we will demonstrate how the system can be used for devices that have already been accepted where the app will show non intrusive dialog for few seconds informing the user about the time and date he trusted the device and how the user is immediately redirected to the service.
For a video presentation of this case study, please visit our website10. 4.2
The second part of the demonstration shows how our system can make use of real-time provenance data being generated by IoT devices. We simulate an insurance company that is installing tracker devices (black boxes) into their customer's vehicles so they can use the data collected to tailor the insurance premiums based on real driving behaviour. For this case study, we developed a replica black box and associated IoT services (insurance company service and a car manufacturer service) which can communicate and exchange information in a IoT ecosystem. 10 http://t3.abdn.ac.uk
This scenario will be demonstrated in two stages:
In the rst stage of this demonstration we will deploy a replica black box device to detect the location of a vehicle and the speed the vehicle is travelling. We will demonstrate how the insurance provider uses the information generated by the device to understand the driving patterns, current location and speed for premium calculation.
Using the Trusted Tiny Things mobile app we will scan the tag attached to the black box and visualise an initial set of capabilities. Such capabilities are de ned based on the ow of data between the device and the supporting services. The ow of data can be seen in Figure 6.
In the second stage of this demonstration we will introduce new actor (car manufacturer service ) to the IoT ecosystem. Previously, the insurance company was calculating premiums based on the information collected from the black box. In this scenario, the insurance company will begin to share accelerometer data with the car manufacturer's service in order to determine out how the policy holder's services their car.
We will illustrate how the new capabilities of the service are determined when the new service is activated. In this example, we will illustrate how the policy holder will be noti ed that personal data were shared with a third-party company. The ow of data associated with this scenario is illustrated in in Figure 7.
Black Box
Braking and Acceleration Data
This research is supported by the Research Council UK Digital Economy IT as a Utility Network+ (EP/K003569/1) and the dot.rural Digital Economy Hub (EP/G066051/1).