In the past few years, the Internet of Things has started to become a reality; however, its growth has been hampered by privacy and security concerns. One promising approach is to use Semantic Web technologies to mitigate privacy concerns in an informed, exible way. We present Carlton, a framework for managing data privacy for entities in a Physical Web deployment using Semantic Web technologies. Carlton uses context-sensitive privacy policies to protect privacy of organizational and personnel data. We provide use case scenarios where natural language queries for data are handled by the system, and show how privacy policies may be used to manage data privacy in such scenarios, based on an ontology of concepts that can be used as rule antecedents in customizable privacy policies.
In the past few years, the Internet of Things (IoT) has started to become a reality. Advances like a ordable Bluetooth Low Energy devices, the Physical Web protocol, and ubiquitous devices such as smart-phones and smart-watches make it possible for people to communicate with devices and places in the world around them usefully and intuitively. Unfortunately, the lack of good privacy and security solutions for ubiquitous sensors and interconnected devices is a continuing concern. Existing approaches to managing privacy tend to assume a closed system, in which known users with known needs can be matched to prede ned policies; the open, mobile nature of the IoT requires more semantically informed solutions. It is a system where devices need to advertise and discover others in their vicinity, interoperate with them in a given context, and publish and honor their privacy and security constraints.
The UN specialized agency responsible for issues concerning information and
communication technologies have de ned the Internet of Things as a global
infrastructure for the information society, enabling advanced services by
interconnecting (physical and virtual) things based on existing and evolving
interoperable information and communication technologies [
Over the past few years, we have observed a rise in usage of IoT devices in
various domains. However, the IoT domain has also been criticized for privacy
and security issues that plague these devices. One major cause for concern stems
from a lack of interoperability among manufacturers and the devices they
create. Most IoT systems today are built in a bottom-up manner|that is, deployed
sensors talk to custom and proprietary gateways, which in turn expose
proprietary services; at the highest level manufacturers provide intelligent applications
based on data from those services. This vertical architecture, which is controlled
by manufacturers, hampers horizontal interoperability [
Semantic Web technology uses the general Web architecture, e.g., URIs to
access data, with such access potentially controlled based on rules de ned in
a knowledge base (KB). Semantic Web has been used to de ne access control
rules in various domains, e.g., mobile applications, smart meeting rooms, RDF
triple stores, Social Media, etc. [
Our proof-of-concept study involves a deployment at our organization, where we study a set of use cases pertaining to queries about persons, places and events associated with the organization. Our goal is to protect the privacy of organizational and personnel data. We examine cases where a user's data privacy is managed by their personal privacy policies. We also study scenarios which handle data privacy in the absence of a personal privacy policy, or in case of queries about places and events. Privacy policies in our framework are contextsensitive; as such, they are dependent on user context, where a user is either (1): a person whose data is represented in the system; or (2): a person who is querying the system. In the following sections we will describe the Carlton system and some initial scenarios, followed by a discussion of ways in which a similar architecture could support more sophisticated privacy reasoning. 2
Role Based Access Control (RBAC) [
Using policy based security is not a new technique. Kagal et. al. [
KAoS [
The hierarchical notion of context de ned in this paper is an extension of
our previous work [
A model for context-sensitive policy based access control for IoT was
presented in our previous work [
Finally, we take a look at indirect vs obvious privacy implications. Ma et.
al. [
The prototype Carlton application targeted in this work is a pervasive information system that uses beacons (low powered, battery e cient devices that broadcast content over Bluetooth), xed-position kiosks, mobile devices, and a NLP engine to respond to natural language queries. Beacons are deployed in key locations in the department, like o ces, classrooms, and labs. The application is intended to provide `Help Desk'-style information about an academic department, allowing people to ask questions and get spoken responses, sometimes augmented with an appropriate display.
One of our goals is to design a system that could be installed by other
university departments and customized by providing a database of key information
about its people and places. Both kiosks and mobile devices use beacons to know
their locations, using the Nearby Messages API [
The Carlton system architecture is shown in Figure 1. The system is
invoked by a user by asking queries as either natural language text or speech
through our kiosk or our Carlton Android app. Spoken language is converted
to text using a SpeechRecognizer service in the Android app, that is part of
Android SDK[
Context is retrieved from an ontology as well as from external sources such as location. Users have the option of identifying themselves; users who lack credentials, or who do not identify themselves, will potentially receive more limited data, depending on privacy policies. A reasoning engine uses inferred context, the user's speech act and de ned privacy policy as input, to reason Additional Context User identity; room calendar Conference ITE conroom ference room
Main deDr. A Joshi partment User identity; o ce target disam
biguation;
Hall out- target Dr. K Joshi side faculty location o ce
Response \No; shall I book it for 2PM-3PM?" \No. See the front desk for room reservations." \The chairman is not in the o ce right now." \No. Would you like to hear her o ce hours?" User
Query
Target
Location
Faculty 1 member \Is this room booked 2 Student from 2PM
3PM?"
Sta 3 member \Is Dr.
Joshi 4 iSntucdlaesnst here?" 5 6
Visitor \Where is
Dr. Tim Finin's o ce?" 7 Student
ITE build- Building o ce \Please see the ing third map; target front desk in ITE
oor disambiguation 325 for directions." Dr. T Finin
Admin building
Target biguation
\Please see the
pubdisam- licly available
campus directory or the
information desk."
User identity;
campus direc- \His o ce is in the
tory; target ITE building."
disambiguation
over, and infer the granularity at which data or descriptions will be shared in
the response. After the response is generated, a natural language query response
is generated and presented, either as text or speech. Table 1 shows some examples
of queries, context, and responses.
Carlton uses context-sensitive privacy policy rules de ned in the Semantic Web
Rule Language (SWRL) [
Antecedents in our rule speci cation consist of the context of a requesting entity, along with the entity type that is being requested, and may also include context for the entity whose data is requested.
A more abstract representation may be considered as a triple (U, C, Q) which contains: U, that represents requested user's context, that is the user whose data is represented in the system. C is requesting user's context, that is the user who is querying the system. Q is the query that is received by the system. The following is an example policy rule with instantiated values, based on our ontology, which demonstrates how information is controlled by our system. Imagine a scenario where Professor Xavier shares information about his present precise location with a person, if they are member of his lab and he supervises them, given he is not speaking at a talk at the moment. The rst 8 predicates in this policy rule represent the requester student's context. The next 8 relate to the requested party's context. The query is about location request and response from the system is to share the location.
Example 1. @prefix crltn :< https :// www . ebiquity . org / ontologies / carlton /0.1 >. @prefix swrlb :< http :// www .w3. org /2003/11/ swrlb >. crltn : student (? requester ) ^ ( crltn : supervises (\Xavier00 ,? requester ) _ ( crltn : affiliatedWith (? requester ,? labName ) ^ crltn : leads (\Xavier00 ,? labName )) ) ^ crltn : hasCurrentLocation (? requester ,? aBldgLocation ) ^ crltn : room (? aBldgLocation ) ^ crltn : sitsIn (\Xavier00, aBldgLocation ) ^ crltn : currentTime (? currTime ) ^ swrlb : Exists (? anEvent ) ^ crltn : speakingAt (\Xavier00 ,? anEvent ) ^ ( ( crltn : startTime (? anEvent ,? eventStartTime ) ^ swrlb : greaterThan (? eventStartTime ,? currTime )) _ ( crltn : endTime (? anEvent ,? eventEndTime ) ^ swrlb : greaterThan (? currTime ,? eventEndTime )) ) ^ crltn : hasCurrentLocation (\Xavier00 ,? aLocation ) ^ crltn : Location (? aLocation ) ^ crltn : requestLocation (\Xavier00) =) shareLocation (? aLocation ) On the other hand, if a student is simply a liated with the same department as he is, Professor Xavier might share a generic location using the privacy policy rule shown below.
Example 2. @prefix crltn :< https :// www . ebiquity . org / ontologies / carlton /0.1 >. @prefix swrlb :< http :// www .w3. org /2003/11/ swrlb >. crltn : student (? requester ) ^ crltn : affiliatedWith (? requester ,? deptName ) ^ crltn : affiliatedWith (\Xavier00 ,? deptName ) ^ crltn : hasCurrentLocation (\Xavier00 ,? aLocation ) ^ crltn : Location (? aLocation ) ^ crltn : partOf (? aLocation ,? city ) ^ crltn : City (? city ) ^ crltn : requestLocation (\Xavier00) =) shareLocation (? city ) 3.2
User Context Speci cation
Context has been de ned by Dey and Abowd [
An interesting feature of our system is that we wish to protect the privacy of
the user querying the system, as well as the information being queried. Identi
cation does have privacy repercussions, and therefore we have designed our system
to be capable of working without identifying a user. When context information
is not available, or if only generic context information is available (e.g., `the user
is a student'), we evaluate data privacy rules accordingly by using rules with
more generic contextual antecedents. We are able to do this using our ontology,
which allows location and activity generalization by using partOf relationships
among location entities and activity hierarchies. The generalization technique
described here is an extension of our work in data privacy in the mobile
domain [
This technique helps us preserve privacy by sharing data with varying levels of granularity, as speci ed by policies. It involves replacing a value with a less speci c but semantically consistent value. As an example of location generalization, a user might de ne a policy stating that \My building level location can be shared with my colleagues," which allows partial location sharing with anyone who is identi ed as a `colleague', based on rules written in our system. In this case, if an exact room location of a user is known, the ontology allows us to generalize to a building using the \Part Of" transitive property. In our ontology Location is a super class of the Room, Building, City, State and Country classes. The various sub-classes are used to denote di erent levels of abstractions for the location.
Activity generalization is similar. Consider a case where we have a hierarchy of work activities: working partOf meeting partOf department meeting
lab meeting partOf
This hierarchy lets a user de ne policies like \Share with team members when I am in a team meeting" and \Share with team non-members when I am in a meeting." This is especially useful if we consider that a team meeting might be accessible only to team members, as it allows obfuscating the data to just a \meeting" when the party querying the data is allowed some access but not access to con dential team data.
We look at identity based generalization in this paper as an extension of the generalization technique. Our ontology contains a Person class with a property affiliatedWith that de nes if a requester is \a liated with" a university, department or research group. If the user is not a liated with any instances of these classes that match the a liation of an entity whose data was requested, we generalize the response accordingly. This allows us to de ne rules like \Share with non-members of the organization only my city level location info". 3.3
Semantic Knowledge The Carlton ontology de nes entity relationships between persons, locations and events in the organizational structure. We are able to make inferences of the form described in Section 5 based on the entity relationship properties de ned in the ontology. See Figure 2 for details of the ontology. The four primary classes de ned in our ontology include Person, Location, Organization and Event. A Person in our ontology might be a Faculty, Staff, Student or Visitor to the organization. Organization is further broken into University, Department, ResearchGroup classes. Event have the sub-classes Talk, Meeting and Course. For the Location class partOf relationship allows hierarchical location context de nition. The same applies to the Organization sub-classes. The Person class allows us to infer a liation with an Organization. A Visitor is a Person in our ontology and might have a a liation too, with an Organization but the instance of a visitor's Organization would not match Organization information for Faculty, Staff or Student from a particular University that the Visitor does not belong to. We are able to infer Location of an Event through the relationship heldAt between Event and Room classes. Obviously a Talk will be held in a ConferenceRoom, a Course will be taught in a Classroom etc. where the rooms are types of the Room class and therefore using the heldAt relationship we are able to infer triples that state such a relationship. Similar inference maybe drawn for a liation of a Faculty, Staff or Student with a Department or ResearchGroup. 4
The system described in this paper takes steps towards ful lling the promise of the Internet of Things: it allows easy access to appropriately contextualized knowledge, making it possible for users to nd appropriate information quickly and easily while reducing administrative overhead. Consistent with that, however, it also showcases some of the privacy concerns associated with precisely those capabilities. In this section, we look at some of the ways in which privacy comes into play in such a system, and discuss how using a semantically informed knowledge framework allows for informed, appropriate behavior. 4.1
Privacy Stakeholders One consideration when discussing privacy concerns is the correct identi cation of stakeholders, individuals or groups whose privacy may be compromised. Some stakeholders are easily identi ed, while others may be more subtle. The same is true of the possible impact of di erent kinds of information access, that is, privacy failures may range from obvious to indirect. We consider three possible stakeholders and, for each, discuss selected privacy concerns.
The most obvious stakeholder group is query targets: individuals about whom queries are posed. Information about query targets may be very direct, as in location, or more general, such as working at a particular research lab. In addition, this information varies in consistency across time: location presumably changes frequently, whereas membership in the Faculty group is usually permanent. Broadly speaking, query targets can be assumed to have some (but not complete) control over their own information; as an example, the department may make information available about professors' o ce hours, but not otherwise provide information from calendars, while an individual may choose to make their meeting times available to their own students.
Less obviously, target privacy may also apply to entire organizations or subgroups. While an academic department may be relatively public, some types of information may still be protected. As an example, what research groups exist within the department is likely to be public information, but the physical location of labs engaged in controversial research may not be. Similarly, a research group may or may not choose to make their group meeting time public, depending on member preference and research topic.
The nal stakeholder whose privacy must be considered is the Carlton
user. The pattern of a user's queries may reveal unwarranted information|
for example, a professor might look askance on a student continually asking
about lab meeting times, even if the student is punctual; if a faculty member
consistently asks about the chair's location and then manages to \just miss" him
or her, active avoidance might be deduced. Even a user who does not identify
him or herself may be de-anonymizable after relatively few queries. [
Semantically Informed Access and Risks Having a formal, semantic representation of the underpinnings of information to be accessed and actions to be taken allows for the formal representation of policies as rules. Carlton combines information about users, contexts, and query targets as antecedents to rules that allow users to express complex sets of constraints on information that can be shared. Information can be controlled based on the user or the query target (as shown in Example 2), the user's location (as shown in Table 1), or more generally|for example, the department might write default rules that prohibit the sharing of information about peoples' locations. Such a rule should include an antecedent specifying that it applies only if there is no more permissive target-speci c rule.
Semantic hierarchies also make it possible to provide \fall-upward" information. Example 1 shows a case in which only the City of the query target is explicitly provided, rather than the speci c location. However, because rules can be expressed with arbitrary complexity, such rules can be speci ed implicitly in both antecedent and consequent. A generalized policy might, for example, say \For members of the academic community, give a summary of my daily schedule; otherwise, provide no information." Such a policy would chain with more general rules about a liation and summarization; this is explored further in Section 5 below.
One notable category of privacy failures in a rule-based system is combinatorial failures, that is, failures in which multiple pieces of allowed information lead to a conclusion that is not intended to be public. This is a concern for overlapping rules with varying privacy implications. As an example, Professor Grey does not allow anyone to see her location or calendar. Dr. Xavier does not allow anyone to see his location, but allows his students to see his calendar, meaning that they can nd out that he is in a meeting with Dr. Grey and Dr. Hank McCoy. Dr. McCoy does not allow access to his calendar, but makes his location available to anyone. With a small number of queries, Dr. Xavier's students can nd the location and current calendar event of all three professors.
This is a di cult problem to address. Determining whether any combination of rules will allow for unintended access would be excessively restrictive. More importantly, it is isomorphic to a complete truth maintenance problem, and not tractable for a knowledge base of reasonable size. While it is possible to write additional rules that explicitly restrict information access in addition to explicitly exposing it, this solution is not logically complete and is, additionally, prone to logical con icts unless rules occur in a strict precedence ordering. The current best available solution is to write relatively few, fairly general rules, and consider the output in di erent cases carefully; while unsatisfying, this is currently the best case for achieving a usable system. 5
We are developing and testing the Carlton prototype in an academic Computer Science department. A number of beacons have been deployed in our University's Information Technology and Engineering (ITE) building's front o ce suite, ofces of faculty members, labs and conference rooms. Carlton is capable of handling a variety of natural language queries, e.g., \Who is Foggy Nelson?", \Where is Ben Ulrich's o ce?", \What does Matt Murdock teach?", etc. Since we have installed our system in an academic organization, we look at use cases that make sense in such a setting. We will look at use cases in which personal data privacy policies have been set up by Professor Jean Grey.
Use Case 1: Share speci c information with people who are nearby and have a relationship with the query target. Prof. Grey prefers certain private information to be shared with people she knows who are near her o ce. She shares her private calendar data with students from a course she is teaching and with people she advises. This may be de ned using the rule: \Share my calendar information with people I teach or supervise if they are near my o ce." Such a query would have relevance in case a student is standing in front of her o ce and wants to know when she will be available for a meeting.
Use Case 2: Share only summary information with people who are far away and have a relationship with the query target. Prof. Grey does not want her students to skip lab meetings irrespective of whether she is busy in another meeting or not. Therefore, she sets up a policy stating \Share only summary information from my calendar with people I supervise if they are far away from my o ce." According to this policy her students will only get to know that Prof. Grey is available today and has some meetings unless they are already in the building. Use Case 3: Share only publicly available data with unknown people. Carlton allows Prof. Grey to block anyone who is not part of her group or school from getting any information from her calendar or any other source that is private to her group or school. This is a special use case which not only takes care of data privacy for the professor but it also takes care of privacy implications from a requester's perspective. We explain this further in Section 6. Use Case 4: Share information about organization's entities with people who are nearby and if they belong to the group or organization. Now we look at privacy of a physical or virtual entity of the organization. The entity in this scenario maybe a meeting room, a lab or an event. Prof. Grey has a lab called PhoenixForce. PhoenixForce owns a meeting room denoted as ITE-1, two labs named ITE-4 and ITE-5, and organizes an event called FallWelcomeBack. Only researchers from her lab are allowed to book the meeting room, get access to the lab, or join the event. These require that the requester has information about when the meeting room is available, whether lab has empty seats, and where the event is. Carlton is thus able to use privacy policies like \Share meeting room availability with members of PhoenixForce," \Share lab empty seat count for ITE-4 if user is near ITE-4 and AND a member of PhoenixForce," and nally \Share event location with members of PhoenixForce if they are on campus." Use Case 5: Share public information about an organization's entities with unknown people. In this use case Carlton unaware of a requester's identity, and therefore shares only publicly available event information, public lab info and public meeting room data|for example, that PhoenixForce does Robotics research and meeting room ITE325 is used for dissertation defenses. These use cases are intended to showcase how data privacy for an individual or organization's entities are managed using Physical Web context discovery and by using privacy policies de ned in Semantic Web technologies. 6
Although our query client includes a login option, it is not necessary for a user to log in in order to query our system. One may observe that our use cases include scenarios where a user's identity is unknown. In these cases we do share less information but this acts as a feature of our system. If a user logs into our app then they have authenticated themselves to our system and we may authorize access to entities as per the privacy policies that have been de ned. However, this means that the system constantly knows who the user is and where they go and what they ask etc. This may be considered as a violation of privacy of the requester.
In case of the physical web we may be able to determine where a particular user is at all times, given enough number of beacons are used to ensure organization-wide coverage. We have thus created a privacy issue for users requesting data from our system, as a side e ect of actually trying to manage data privacy for individuals and entities of the organization. How do we address this issue? We have already alluded to a potential restriction on our system in Use Cases 3 and 5 that will allow us to reduce some of the privacy concerns. By not making the login step mandatory, we have reduced these privacy concerns. Queries are still responded to in this case but we generalize our response based on the a liation information that we might have.
Is it possible to authenticate a user without knowing the actual identity
of the user? Yes, we have implemented Authentication without Identi cation
using Zero-Knowledge Proof as described by [
The Carlton mobile app does not require any permission other than:
{ android.permission.INTERNET
{ android.permission.ACCESS FINE LOCATION
{ android.permission.BLUETOOTH
These permissions are required to talk to the beacons over the BLE protocol
and to obtain beacon data and call back-end server for query responses over
the internet. The location access is required by the Nearby Messages API [
In this paper, we have presented Carlton, a framework for managing data privacy for entities in a Physical Web deployment using Semantic Web technologies. Our system used context-sensitive privacy policies to protect privacy of organizational and personnel data. We have provided few use case scenarios where natural language queries for data are handled by Carlton. We have explained how privacy policies may be used to manage data privacy in such scenarios, based on an ontology of concepts that can be used as rule antecedents in customizable privacy policies.
Although we have presented a few use cases in this paper, the number of scenarios are small relative to fully functional, in-real-life system. The purpose of use cases that we have presented, was to demonstrate the utility of Carlton. However, many more scenarios are possible and we intend to explore them all eventually. A challenge that we constantly faced during this project was managing the beacons when they are in close proximity. This problem was solved in a trivial manner by reducing the signal strengths of the beacons. However, this still doesn't handle con ict resolution in presence of multiple beacons in close-proximity. Handling, such issues could be a challenging goal to pursue, in the future. 8
We gratefully acknowledge the support of the Google Internet of Things (IoT) Technology Research Award pilot, which provided the hardware used for this research.