Wearable computing devices like Google Glass are at the forefront of technological evolution in smart devices. The ubiquitous and oblivious nature of photography using these devices has made people concerned about their privacy in private and public settings. The FaceBlock3 project protects the privacy of people around Glass users by making pictures taken by the latter, Privacy-Aware. Through sharing of privacy policies, users can choose whether or not to be included in pictures. However, the current privacy model of FaceBlock only permits simple constraints such as allow versus disallow pictures. In this paper, we present an extended context-aware privacy model represented using OWL ontologies and SWRL rules. We also describe use cases of how this model can help FaceBlock to generate Privacy-Aware Pictures depending on context and privacy needs of the user.
Google Glass is a wearable device with an optical head-mounted display that
enable users to take pictures just by saying \OK Glass, take a picture" or
winking. Therefore, the device has raised privacy concerns because it has a readily
available camera which can take pictures without anyone noticing. We
developed FaceBlock [
FaceBlock allows users to state their policy about being photographed (i.e.,
\I don't want my picture to be taken") by other people. To start with, FaceBlock
generates an eigenface [
While being helpful in safeguarding privacy, such an all-or-nothing model does not help in many real-life situations. A person's preferences would depend on the context of the situation (e.g., time, place, activity and participants), who is taking the picture and with whom it may be shared. For example, policies like \I am okay being photographed by people I know at a private event" or \I do not like to be photographed when I am at public places". Therefore, to handle such policies, the system should understand the semantics of concepts such as \public place", \people I know" or \private event" as well as other elements describing situation of the user.
FaceBlock's current privacy model allows a user's device to specify whether she wants her image obfuscated in pictures. Privacy-aware pictures are disabled by default privacy policy when there is no active policy sent by the other users. The current simple privacy model does not allow statements that photography is permitted, using the approach of \whatever is not explicitly prohibited, is permitted". Instead of asking everyone around if it is okay to take a picture, a Glass user might prefer or require having a positive statement from the subjects that the image can be displayed in public. Therefore, including a way for a person to grant permission is also an obvious way to improve FaceBlock. 2
Arguably one of the most accepted de nition of the concept \context" was
suggested by Dey and Abowd [
The payload of FaceBlock's current policy includes the face identi er and
privacy policy (allow vs. disallow pictures). However, more ne-grained privacy
policies can be added to the FaceBlock system using Semantic Web technologies.
The use of ontologies to represent privacy policies has been studied before in [
Location-aware policies. The basic assumption for the FaceBlock application
is that both concerned parties (the Google Glass user and a mobile user) are
at the same location. This is justi ed by use of peer-to-peer (P2P) networks
for sharing the policy between two users. In our scenario we treat location
context semantically as in \at the bar", \at the University campus" or \inside my
home". We de ne location hierarchy by referencing existing entities from linked
data ontologies such as DBpedia [
Activity-aware policies. Activity adds another dimension to the meaning of
location. For example, a classroom used for private meetings versus public
lectures. Activity recognition is included in current smart devices at some extent.
For example, Google recently introduced through Android API version 8 and
above the ability to recognize simple activities like \in a vehicle", \on a
bicycle", \running", \still, \tilting", and \on foot". For other complex activities,
there are systems [
Time-aware policies. While time is ingrained into other aspects of context, it allows for an all-embracing notion of privacy in pictures without worrying about the location and activity of users. For example, users could mention in their privacy policy that no pictures of them should be taken after 5:00 pm on Fridays irrespective of the activity or location. Time driven policy examples could be \don't allow my picture on Friday after 5:00 pm till Monday until 8:00 am", \don't take my picture during holidays". Continuing from the last example, as Spring Break would be classi ed as holidays, FaceBlock will be active during the time.
Composite policies. Most user situations demand a combination of various types of context-aware policies described above. Complex policies could possibly include rules that take into consideration more than one or all aspects of context as de ned above. A possible example of such a scenario would be \do not allow my social network colleagues group (identity context) to take pictures of me (identity context) at parties (activity context) held on weekends (time context) at the beach house (location context)". We propose to prede ne these rules using the Semantic Web Rule Language (SWRL) based on FaceBlock's context ontology (see Figure 2). For example, Figure 3 shows the SWRL rule that models the previous policy. Notice that FaceBlock creates an maintains updated an instance of the Context class with the current context of the user. Also, to model whether a user is allowed to take pictures or not of another person we use the data property FaceBlockPictures(Person,xsd:boolean). 3
We describe the cross device process for generating privacy-aware pictures as
shown in Figure 4. The user's context is represented using an OWL ontology and
privacy policies are described using SWRL rules. Finally, we use a Description
Logics reasoner to infer if the current context of the user matches with any of
the privacy policies de ned. Using wireless communications, such as Bluetooth
and WiFi, FaceBlock creates peer-to-peer networks to share the privacy policies
among devices. The user device holds the ontology and reasoner and is in charge
of checking the policies that should be applied (in [
We use an example with two users, Primal and Roberto, to explain how the FaceBlock system works using the privacy policies. Primal is the user who wishes to protect his privacy and Roberto is the user with the device for taking pictures, that is Google Glass. Initially, Primal takes a picture of himself using FaceBlock and it generates a face identi er (step 1 in Figure 4). He also speci es the context constraints for his pictures using FaceBlock. At the Beach House, Primal's smartphone detects and shares the face identi er with Roberto's Google Glass (step 2) along with a unique identi cation. Later, both devices periodically check whether the other device has left the surroundings by greeting/acknowledgment messages. Roberto's device receives this information, stores it and sends back his UID and an acknowledgment of the previous message (steps 3 and 4).
Afterwards, FaceBlock on Primal's device continuously collects information about his context and checks if any rule should be triggered by using the reasoner (step 5). In this case, the context has changed (the party started) and the rule presented in Figure 3 gets triggered requesting Roberto to FaceBlock pictures of Primal (step 6). The corresponding privacy policy for Primal is shared with Roberto's device (step 7). Each privacy policy has a Time To Live (TTL) associated with it during which the policy should be applied to the pictures of the user. Currently, we are using a uniform TTL for every policy. Roberto's device accepts the privacy policy from Primal's device (step 8) and whenever he takes a photo FaceBlock converts it into a privacy-aware picture (step 9). For that, if faces are detected in the recently taken picture, FaceBlock checks if Primal is present in the picture by comparing the detected face with Primal's face identier and obscures it (step 10). Thus, FaceBlock creates a privacy-aware picture for Roberto and protects the privacy of Primal. 4
In this paper, we have described a new policy management module for
FaceBlock, an approach to preserve user privacy when taking pictures with smart
devices. We have an implementation of the basic FaceBlock concept for trivial
allow/disallow policies that works on Android-based devices, including Google
Glass. We have also done work in our research group on context modeling,
learning, acquisition, use and sharing for mobile devices [
From a security and privacy perspective, Faceblock's framework has both advantages and disadvantages. On the plus side, a device does not need to identify its user to participate nor must it reveal any information about its context model. The use of peer-to-peer networking reduces exposure to identi cation by network location, especially if a device spoofs its MAC address. A device wishing to protect its user needs to provide only an eigenface and periodic allow picture and disallow picture messages. Eigenfaces enable a privacy preserving method to share face identi ers which retains enough information for FaceBlock to perform face recognition while making it di cult for humans. We acknowledge face recognition techniques can be inaccurate which might mistakenly apply the policy to someone who looks similar. We are further investigating di erent face recognition techniques to reduce the number of false positives and false negatives. Our architecture also assumes that each user device computes and maintains its own context, so the cost of this task is distributed.
On the other hand, like any privacy policy based solutions FaceBlock might su er from malicious policies. Since Google Glass acts as the server for privacyaware picture requests, a device could also launch a kind of denial of service attack on a Glass device by sending it many requests to block di erent eigenface images that appear to be from di erent devices via MAC address spoo ng. Even though face identi ers and unique identi cation lessen the privacy loss involved in con rming the identity of users during the generation of privacy-aware pictures, we are exploring the possibility of a zero-knowledge protocol. Lastly, the picture taking device follows policies voluntarily; there is no mechanism to guarantee, or even incentivize, enforcement.
We are exploring mechanisms to assist users in de ning their policies, which involve using Graphical User Interfaces (GUIs) and forms to generate SWRL rules. We are additionally exploring the possibility of supporting location-based beacons that can broadcast organization's policies. These are the same as contextaware policies but do not require the exchange of a face identi er between participants. For example, many museums have a policy that no pictures are ever allowed (location-aware). A church might have a policy that pictures are allowed when a service is not taking place and disallowed when one is (activity-aware). This will enable FaceBlock to generate privacy-pictures for inanimate objects.
FaceBlock has to make sure that the privacy policy under consideration
reects the current context of the user. An interrupt driven approach, in which
user actively enters context change into the application, would be more
accurate and less power consuming. But this requires user to actively participate
in the generation of privacy-aware pictures and might fatigue them. A sensor
based polling approach can automatically detect context change without
interference from the user at a higher power cost. We are currently exploring both
these approaches to evaluate the tradeo s of accuracy versus e ciency [
The current FaceBlock protocol relies on P2P networks for information exchange between two users. But in real-life scenarios there would be more than one user in the vicinity of a Google Glass user who wishes to protect his privacy. With higher number of communicating devices, relying on a synchronous connection-oriented link over P2P networks (e.g., Bluetooth) might result in degradation of quality of service. Therefore, other wireless mechanisms, such as WiFi, or even a centralized cloud-based approach could be considered. Also in cases where multiple devices share con icting policies, FaceBlock would require a con ict resolution mechanism.
Mechanisms for protecting user privacy in social circles have to balance
between privacy requirements and the easiness of utilizing them. Privacy preference
models such as P3P [