Privacy policies are legal documents used to inform users about the collection and handling of their data services or technologies with which they interact. Research has shown that few users take the time to read these policies, as they are often long and difficult to understand. In addition, users often only care about a small subset of issues discussed in privacy policies, and some of the issues they actually care about may not even be addressed in the text of the policies. Rather than requiring users to read the policies, a better approach might be to allow them to simply ask questions about those issues they care about, possibly through iterative dialog. In this work, we take a step towards this goal by exploring the idea of an automated privacy question-answering assistant, and look at the kinds of questions users are likely to pose to such a system. This analysis is informed by an initial study that elicits privacy questions from crowdworkers about the data practices of mobile apps. We analyze 1350 questions posed by crowdworkers about the privacy practices of a diverse cross section of mobile applications. This analysis sheds some light on privacy issues mobile app users are likely to inquire about as well as their ability to articulate questions in this domain. Our findings in turn should help inform the design of future privacy question answering systems.
Privacy policies are the legal documents which disclose the ways in which a company gathers, uses, shares and manages user data. They are now nearly ubiquitous on websites and mobile applications. Privacy policies work under the ”notice and choice” regime, where users read privacy policies and can then choose whether or not to accept the terms of the policy, occasionally subject to some opt-in or opt-out provisions.
However due to the length and verbosity of these
documents
In this work, we explore the idea of an automated “privacy assistant”, which allows users to explore the content of a privacy policy by answering their questions. This kind of question-answering approach would allow for a more personalized approach to privacy, enabling users to review sections of policies that they are most interested in. The successful development of effective question-answering functionality for privacy requires a careful understanding of the types of questions users are likely to ask, how users are likely to formulate these questions, as well as estimating the difficulty of answering these questions. In this work, it is our goal to explore these issues by providing a preliminary qualitative analysis of privacy-related questions posed by crowdworkers.
There has been considerable interest in making the content
of privacy policies easy to understand. These include
approaches that prescribe guidelines for drafting privacy
policies
Several large-scale reading comprehension/answer selection
datasets exist for Wikipedia passages
We would like to gain a better understanding of the kinds of questions users are likely to ask, and what legally-sound answers to them would be. For this purpose, we collect our data in two stages: first, we crowdsource questions on the contents of privacy policies from crowdworkers, and then we rely on domain experts with legal training to provide answers to the questions. We would like to note that our methodology only exposes crowdworkers to public information about each of the companies, rather than requiring them to read the privacy policy to formulate questions. This includes the name of the mobile application, the description of the mobile application as presented on the Google Playstore as well as screenshots from the mobile application. This approach attempts to circumvent potential bias from lexical entrainment, and more generally the risk of biasing crowdworkers to ask questions only about the practices disclosed in the privacy policy.
In this study we intentionally select mobile applications
from a number of different categories, specifically
focusing on apps from categories that occupy 2% of mobile
applications on the Google Playstore
3We choose to focus on the privacy policies of mobile applications given the ubiquitousness of smartphones. However, our study design is limited to Android mobile applications. In practice however, these mobile applications often share privacy policies across platforms set of questions such that we range from mobile applications which are well-known and likely to have carefully constructed privacy policies, all the way to applications which may have smaller install bases and less sophisticated privacy policies. We sample applications from each category using the Google Playstore recommendation engine, such that only half of the applications in our corpus have more than 5 million installs 4. We collect data for 27 privacy policies across 10 categories of mobile applications. 5
An important objective of this study is to elicit and understand the types of questions users are likely to have when looking to install a mobile application. As discussed earlier, we present information similar to the information found when looking at the application in the Google playstore (Figure 2). We use Amazon Mechanical Turk to elicit questions about these privacy policies. Crowdworkers were asked to imagine they installed a mobile application and could talk to a trusted privacy assistant, whom they could ask any privacyrelated question pertaining to the app. They were paid 12$ per hour to ask five questions for a given policy. We solicited questions from Turkers who were conferred “master” status, and whose location was within the United States and our task received favorable reviews on TurkerHub. For each mobile application, crowdworkers were also asked to rate their understanding of what the app does on a Likert scale of 1-5 (ranging from not being familiar to understanding it extremely well), as well as to indicate whether they had installed or used the app before. We also collected demographic information regarding the age of the crowdworkers.
We are not just interested in collecting data on what questions users ask, but also a corpus of what good answers to these questions would be. For this purpose, given questions 4We choose 5 million installs as a threshold on popularity of the mobile application, but this choice is debatable. Mobile applications with fewer than 5 million installs could also represent applications of large corporations and vice versa.
5The Playstore categories we sample applications from
include: Books and Reference, Business, Education, Entertainment,
Lifestyle, Health and Fitness, News and Magazines, Tools, Travel
and Local, and Games.
for a particular application, we recruit four experts with legal
training to formulate answers to these questions based on the
text of that application’s privacy policy. The experts annotate
questions for their relevance, subjectivity and also identify
the relevant OPP-115
We would like to explore the kinds of questions that users
ask our conversational assistant. We analyze questions based
on their question words, as well as by having our expert
annotators indicate whether they believe the questions are
related to privacy, whether they are subjective in nature and
what categories they belong to in the OPP-15 ontology
7All analyses in this section are presented on the ’All’ data split unless mentioned otherwise is/does what will how can are who where if 27.9 % 13.5 % 11.9 % 10.1 % 8.6 % 4.5 % 4.4 % 1.3 % 1.8 % Question Words We qualitatively analyze questions by their expected types, based on the first word of the question. Note that while the question word can give us some information about the information-seeking intent of the user, the different question words can often be used interchangeably. For example, the questions ’will it collect my location?’ can also be phrased as ‘does it collect my location’. Keeping these limitations in mind, we perform a qualitative analysis of the elicited questions to identify common user intents. The distributions of questions across types can be found in Table 2. By far, the largest proportion of questions can be grouped into the ‘is/does’ category where, similar to the ‘are’ category, users are often questioning the assistant about a particular privacy attribute (for example, ’does this app track my location?’ or ‘is this app tracking my location?’). The next largest category includes ‘what’ questions which include a broad spectrum of questions (for example, what sort of analytics are integrated in the app?’ or ’what do you do with my information’). The ‘will’ and ‘can’ questions are usually asking about a potential privacy harm (for example, ‘will i be explicitly told when my info is being shared with a third party?’ or ‘will any academic institutions or employers be able to access my performance/score information?’ or ‘can the app see what i type and what i search for?’). ’How’ questions generally either ask about specific company processes, or abstract attributes, such as security, longevity of data retention etc (for example, ‘how safe is my password’ and ‘how is my data protected’). Relevant ‘where’ questions are generally related to data storage (for example, ‘Where is my data stored?’). Questions that begin with ‘who’ are usually asking about first party or third party access to data (for example, ‘who can see my account information?’ or ‘who all has access to my medical information?’). Finally questions in the ‘if’ category typically establish a premise, before asking a question. Such a question needs to be answered based on both the contents of the policy as well as assuming the information in the premise is true (for example, ‘if i link it to my Facebook will it have access to view my private information?’ or ‘if i choose to opt out of the app gathering my personal data, can i still use the app?’).
questions asked to our privacy assistant are ‘relevant’ i.e are related to privacy, and how many are subjective in nature.
In the real-world it isn’t necessary that users will only ask our privacy assistant questions related to privacy. Thus, it is important for us to be able to identify which questions we are capable of attempting to answer. We analyze the test-set where each example features multiple annotations from our expert-annotators. We consider the majority-vote to be the judgement of whether a question is relevant or subjective. We find that 78.85% of questions received by our privacy assistant are relevant, with 6.28% being subjective. Table. 3 gives us more insight into this phenomena. We observe that the majority of questions (74%) are relevant but not subjective (for example, ‘what information are they collecting?’). 4.86% of questions are both relevant and subjective (for example, ‘is my data safe?’), 1.4% are subjective but not relevant (for example, ‘are there any in game purchases in the wordscapes app that i should be concerned about?’) and finally 19.71% are neither relevant nor subjective (‘does the app require an account to play?’).
Question Ontology Categories Next we ask our
annotators to indicate the OPP-15 data practice category
User Access, Edit and Deletion User Choice//Control Other International and Specific Audiences No Agreement
We would like to characterize and study the difficulty of the
question-answering task for humans. We formulate the
problem of identifying relevant evidence in the document to
answer the question as a sentence-selection task, where it is
possible to choose not to answer a question by not
identifying any relevant sentences. We evaluate using
sentencelevel F1 rather than IR-style metrics so as to accommodate
models to abstain from answering 8. Similar to
As discussed previously, since most questions are
difficult to answer in a legally-sound way based on the contents
of the privacy policy alone, abstaining from answering is
often going to be a safe action. We would like to emphasize
that this is not a criticism of the annotators or the people
asking the questions, but rather a characteristic of this
domain where privacy policies are often silent or ambiguous
on issues users are likely to inquire about. To quantify the
magnitude of this effect, we demonstrate that a model which
always abstains from answering the question can achieve
reasonable performance (Table 5), yet still leaves a large
gap for improvement. We would further like to understand
what makes the majority of our annotators decide a question
should not be answered. We randomly sample 100
questions that were deemed unanswerable, and annotate them
post-hoc with reasons informed by expert annotations. We
find that for 56% of unanswerable questions, the answer to
the question would typically not be present in most privacy
policies. These would include questions such as ‘how does
the currency within the game work?’ and suggests that users
would benefit from being informed about the scope of
typical privacy policies. However, they also include questions
such as ‘has Viber had data breaches in the past?’ which
8Similar to
36.2 71.1 ideally a privacy assistant would be able to answer, but is not present within a typical privacy policy. In the future, a privacy assistant could draw upon various sources of information such as metadata from the Google Playstore, background legal knowledge, news articles, social media etc. in order to broaden its coverage across questions. For an additional 24% of unanswerable questions, the answers were expected to be found in the privacy policy, but the privacy policy was silent on a possible answer (such as ‘is my app data encrypted?’). Generally when a policy is silent it is not safe to make any assumptions. 6% of questions asked by a user are too vague to understand correctly such as ‘who can contact me through the app?’, such questions would benefit from the assistant engaging in a clarification dialogue. Another 4% are ambiguously phrased, such as ‘any difficulties to occupy the privacy assistant?’. These kind of questions are very hard to interpret correctly. 3% of unanswerable questions are too specific in nature, and it is unlikely the creators of the privacy policy would anticipate that particular question (‘does it have access to financial apps i use?’). Finally, 7% of unanswerable questions are too subjective and our annotators tend to abstain from answering (for example, ‘how do i know this app is legit?’).
We would also like to be able to characterize the disagreement on this task. It is important to note here that all of our annotators are experts with legal training rather than crowdworkers, and their provided answers can generally be assumed to be valid legal opinions about the question. We tease apart the difference from where they abstained to answer to their disagreements by comparing against the No Answer (henceforth known as NA) baseline (Table 5). In Table 5 we observe the human F1 is 70.7%, demonstrating considerable agreement on the right answer. We would still like
NA Model
Human
User Access, Edit and Deletion User Choice/Control Other International & Specific Audiences No Agreement to investigate whether any disagreements are valid, or if they are due to poor definitions or lack of adequate specification in the annotation instructions. We randomly sample 50 samples and annotate them for likely reasons for disagreement 9. We find that they ”agree on 64% of instances and disagree on 36%. We further determine that 92.8% of disagreements were legitimate, valid different interpretations. For 43.75% the question was interpreted differently, in 25% the contents of the privacy policy were interpreted differently and the remaining were due to other sources of error (for example, in the question ‘who is allowed to use the app’, most annotators abstain from answering, but one annotator points out that the policy states that children under the age of 13 are not allowed to use the app.)
We next analyze disagreements based on the type of question that was asked (Table. 6). As observed, the wh-type of the question may give us some information about the intent of the questions. We observe that our expert annotators rarely abstain to answer when a user asks a ’will’ question about a potential privacy harm, taking care to identify relevant sections of the privacy policy. Similarly ’if’ type questions generally are quite specific and require careful reasoning. On the other hand ‘where’ questions are generally about data storage. They are vague, for example ’where is my data stored?’ is probably not asking for the exact location of the company’s datacenters but it is unclear what granularity is meant in the question (e.g., particular country, versus knowing whether the data is stored on a mobile phone or in the cloud).
We also analyze disagreements based on the OPP-115 category of the question (Table. 7). As expected, questions where annotators disagree on the category of the question, have more disagreements than simply abstaining to answer. Similarly for user choice, the policy typically does not answer questions like ‘how do I limit its access to data’ fully, so the annotators tend to abstain from answering. In contrast, questions about first party and third party practices are usually anticipated and often have answers in the privacy policy.
9We do not use F1 to measure disagreement, and instead manually filter samples so we can capture both when the legal experts interpreted the question differently, as well as when they interpret the contents of the privacy policy differently.
What kinds of questions should an automated privacy assistant expect to receive? We explore this question by designing a study that elicits questions from crowdworkers who are asked to think about the data practices of mobile apps they might consider downloading on their smartphones. We qualitatively analyze the types of questions asked by users, and identify a number of challenges associated with generating answers to these questions. While in principle privacy policies should be written to answer questions users are likely to have, in practice, our study shows that questions asked by users often go beyond what is disclosed in the text of privacy policies. Challenges arise in automated question answering, both because policies are often silent or ambiguous on issues that users are likely to inquire about, and also because users are not very good at articulating their privacy questions - and occasionally even ask questions that have nothing to do with privacy. Determining a user’s intent may be a process of discovery for both the user and the assistant, and thus in the future it would be helpful if the assistant was capable of engaging in clarification dialogue. Such a privacy assistant would have to reconcile the need to be helpful to the user and provide answers that are legally accurate with the need to be helpful. It would have to be capable of disambiguating questions by engaging in dialogues with users; it would have to be able to supplement information found (or lacking) in the privacy policy with additional sources of information such as background legal knowledge. Ideally, it would also be able to interpret ambiguity in the policy and also be able interpret silence about different issues. We hope that the identification of these requirements will help inform the design of effective automatic privacy assistants.
This work has been supported by the National Science Foundation under Grant No. CNS 13-30596 and No. CNS CNS 13-30214. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the NSF, or the US Government. The authors would like to thank Lorrie Cranor, Florian Schaub and Shomir Wilson for insightful feedback and discussion related to this work.