Privacy policies are the main instruments for disclosing and describing apps’ or other software’s privacy practices. However, the sheer volume of text an individual user would need to read for the software he or she is using makes privacy policies impractical for meaningfully conveying privacy practices (McDonald and Cranor 2008) . Some research has focused on the structure of privacy policies. For example, the problem of identifying policy sections relating to different topics (Ramanath et al. 2014; Liu et al. 2018) . Sathyendra et al. classified advertising opt outs and similar consumer choice options on websites (Sathyendra et al. 2017) . Other work has focused on building tools for users. Using a simple naive Bayes classifier, Zimmeck and Bellovin provided a browser extension for identifying common privacy practices in policy text (Zimmeck and Bellovin 2014) . Tesfay et al. used a machine learning-based approach to identify text addressing various GDPR provisions (Tesfay et al. 2018) . Harkous et al. developed PriBot, a chatbot for answering questions about privacy policies (Harkous et al. 2018) . Different from those studies, however, our domain consists of app policies instead of website policies.

Various studies analyzed privacy policies in specific domains. Cranor et al. evaluated financial institutions’ privacy notices, which, in the US, nominally adhere to a model privacy form released by federal agencies (Cranor, Leon, and Ur 2016) . They found clusters of institutions sharing consumer data more often than others. They also found institutions that do not follow the law, by disallowing consumers to limit such sharing. Further, Zhuang et al. aimed to help university researchers by automating enforcement of privacy policies of Institutional Review Boards (Zhuang et al. 2018). Auditing the disclosure of third party data collection practices on 200,000 website privacy policies, Libert found that the names of third parties are usually not explicitly disclosed in website privacy policies (Libert 2018) . We focus on classifying first and third party access of contact, location, and unique identifier data in smartphone apps’ privacy policies.

We are extending the emerging domain of verifying privacy practices of mobile apps against privacy requirements, notably privacy policies. The closest related work to ours analyzed the practices of 17,991 Android apps and determined whether those with a privacy policy adhered to it (Zimmeck et al. 2017) . Several other studies have also compared privacy policies to apps’ code (Yu et al. 2016; Slavin et al. 2016) . Going beyond this work, our system is capable of large-scale analyses, which we demonstrate by an analysis of 1,035,853 free apps on the Google Play Store. Additionally, our analysis evaluates compliance issues at a finer granularity. This advance is notable because the access of coarse-grained location data (e.g., city) is far less privacyinvasive than the access of fine-grained data (e.g., latitude and longitude).

We are motivated to study privacy in the Android ecosystem due to numerous findings of potentially non-compliant privacy practices. Story et al. studied the metadata of over a million apps on the Play Store and found that many apps lack privacy policies, even when developers describe their apps as collecting users’ information (Story, Zimmeck, and Sadeh 2018) . Analyzing close to a million Android web apps (i.e., Android apps that use a WebView), Mutchler et al. found that 28% of those have at least one vulnerability, such as data leakage through overridden URL loads (Mutchler et al. 2015) . Differences in how apps are treating sensitive data were used to identify malicious apps (Avdiienko et al. 2015) . More recently, AppCensus revealed that many Android apps collect persistent device identifiers to track users, which is not allowed for advertising purposes according to the Google Play Developer Program Policy (Reyes et al. 2018; Google 2018b) . The observation of 512 popular Android apps over eight years of version history by Ren et al. came to the conclusion that an important factor for higher privacy risks over time is the increased number of third party domains receiving personally identifiable information (Ren et al. 2018) . In line with these observations, it is one of our goals in this study to examine apps’ third party practices in the Android ecosystem.

3

Our system analyzes privacy practices. A privacy practice, or simply practice, describes a behavior of an app that can have privacy implications. Table 3 contains the list of practices we consider in our model.1 We account for the fact that disclosures found in privacy policies can vary in specificity. For instance, for the access of location data our model includes practices that pertain to location in general (i.e., Location) as well as more specific practices that explicitly identify the type of access (i.e., Location Cell Tower, Location GPS, and Location WiFi). Our model distinguishes between first party access, where data is accessed by the code of the app itself, and third party access, where data is accessed by advertising or other third party libraries. Finally, our model also distinguishes between a policy describing the performance of a practice (e.g., “We access your location information.”) and the description that a practice is not performed (e.g., “We do not access your location information.”). When access is neither explicitly described nor explicitly denied, neither modality classifier flags the statement. Note that a given text fragment can refer to multiple practices.

1In preliminary tests we also considered city, ZIP code, postal address, username, password, ad ID, address book, Bluetooth, IP address (identifier and location), age, and gender practices. However, we ultimately decided against further pursuing those as we had insufficient data, unreliable annotations, or difficulty identifying a corresponding API for the app analysis.

We characterize the detection of privacy practice descriptions in privacy policy text as a classification problem. Dataset We used the APP-350 corpus of 350 annotated mobile app privacy policies to train and test our classifiers (Zimmeck et al. 2019) .2 The corpus’s policies were selected from the most popular apps on the Google Play Store. The policies were annotated by legal experts using a set of privacy practice annotation labels. As they were annotating the policies, the experts also identified the policy text fragments corresponding to the practice annotation labels they applied. All policies were comprehensively annotated. Consequently, it is assumed that all unannotated portions of text do not describe any of the practices and can be used as training, validation, and test data to detect the absence of statements on respective practices.

We randomly split the annotated privacy policies into training (n = 188), validation (n = 62), and test (n = 100) sets. We used the training and validation sets to develop our classifiers. The test set was set aside in order to prevent overfitting. We did not calculate performance using the test set until after we finished developing our classifiers. Classification Task The goal of the classification task is to assign annotation labels to policy segments, that is, structurally related parts of policy text that loosely correspond to paragraphs (Wilson et al. 2016; Liu et al. 2018) . We focus on segments instead of entire policies to make effective use of the annotated data and to identify the specific policy text locations that describe a certain practice.

The infrequent occurrence of certain types of statements makes the training of classifiers for some practices more challenging. In particular, statements on third party practices and statements explicitly denying that activities are performed are rare. For example, our training set only includes 7 segments saying that Location Cell Tower information is not accessed by third parties. To address this challenge, we decompose the classification problem into three subproblems, that is, classifying (1) data types (e.g., Location), (2) parties (i.e., 1stParty or 3rdParty)3, and (3) modalities (i.e., whether a practice is explicitly described as being performed or not performed). For example, the Location Cell Tower 3rdParty Not Performed classification will be assigned to a segment if the Location Cell Tower, 3rdParty, and Not Performed classifiers all return a positive result for the segment.

The decomposition of the classification task allows for an economic use of annotated data. If the subproblems were tackled all at once, 68 monolithic classifiers would be needed, most of which would have to be trained on fewer than 100 positive training samples. By dividing the problem, only 22 classifiers are needed (18 “data type”, 2 “party”, 2The dataset is available at https://data.usableprivacy.org. 3Note that the Single Sign On and Single Sign On: Facebook practices do not use a party classifier, as all data is exchanged between the app developer as first party and the SSO provider as third party. and 2 “modality” classifiers). These classifiers have a much higher number of positive samples available for training, as shown in Figure 1.

Preprocessing As classifier performance depends on adequate preprocessing of policy text as well as domain-specific feature engineering, we normalize whitespace and punctuation, remove non-ASCII characters, and lowercase all policy text. Because stemming did not lead to performance improvements, we are omitting it. In order to run our classifiers on the most relevant set of features, we use an optional preprocessing step of sentence filtering. Based on a grid search, in cases where it improves classifier performance, we remove a segment’s sentences from further processing if they do not contain keywords related to the classifier in question (Zimmeck et al. 2017) . For example, the Location classifier is not trained on sentences which only describe cookies.

Vectorizing Prior to training, we generate vector representations of the segments. Specifically, we take the union of a TF-IDF vector and a vector of manually crafted features. Our TF-IDF vector is created using the TfidfVectorizer (scikit-learn developers 2016a) configured with English stopwords (stop words=’english’), unigrams and bigrams (ngram range=(1, 2)), and binary term counts (binary=True). This configuration is similar to what was used in prior work (Liu et al. 2018). Our vector of manually crafted features consists of Boolean values indicating the presence or absence of indicative strings we observed in our training and validation data. For example, we include the string not collect, because we realized that it would be a strong indicator of the negative modality.

Training Using scikit-learn, version 0.18.1 (Pedregosa et al. 2011) we train binary classifiers for each data type, party, and modality. For all but four classifiers we use scikit-learn’s SVC implementation (scikit-learn developers 2016b) . We train those with a linear kernel (kernel=’linear’), balanced class weights (class weight=’balanced’), and a grid search with five-fold cross-validation over the penalty (C=[0.1, 1, 10]) and gamma (gamma=[0.001, 0.01, 0.1]) parameters. We create rule-based classifiers for four data types (Identifier, Identifier IMSI, Identifier SIM Serial, and Identifier SSID BSSID) due to the limited amount of data and their superior performance. Our rule-based classifiers identify the presence or absence of a data type based on indicative text strings.

Table 1 shows the effects of our features and preprocessing steps on the F1 scores of our non-rule-based classifiers. The performance is calculated using our training and validation sets. We made sentence filtering an optional part of preprocessing because of the large detrimental effect it has on some of our classifiers, as highlighted in Table 2. In general, our results suggest that the chosen feature and preprocessing steps improve classifier performance. However, ideally they should be chosen on a per-classifier basis to avoid any negative performance impact.

Performance Analysis Table 3 shows the performance of the classifiers on the privacy policies of the test set. We say a policy describes a practice if at least one segment is flagged by the corresponding data type, party, and positive modality classifiers. Since our definition of potential compliance issues does not depend on the negative modality classifier, we do not include it in the table. Because detecting potential compliance issues is dependent on detecting when practices are not described in policies (Zimmeck et al. 2017) , negative predictive value, specificity, and negative F1 scores are of particular importance.

In the closest related work (Zimmeck et al. 2017) , classifiers for contact, identifier, and location data practices covered multiple specific practices. Thus, a direct performance comparison to our classifiers is not possible. However, with negative F1 scores ranging from 78% to 100%, 23 of our specific classifiers achieve better negative F1 scores than the corresponding course-grained classifiers, and 3 performed equally. These results demonstrate that our approach constitutes an overall improvement over the state of the art. We believe that decomposing the classification task into three subproblems increases performance as it allows for a better exploitation of training data compared to monolithic classifiers.

Our results reveal that generally + support is lower for third party practices; that is, third party practices are often not as extensively described in privacy policies as first party practices. It should be further noted that higher counts of support generally correlate with higher performance scores. Intuitively, it is easier to classify a policy that does not describe a practice, which makes up the majority of - support instances.

We reviewed the errors made by our classifiers and identified several potential areas for improvement. First, approaching the classification task at the level of segments, as suggested by prior work (Wilson et al. 2016; Liu et al. 2018) , can pose difficulties for our subproblem classifiers. For example, if a segment describes a 1stParty performing the Location practice, and a 3rdParty performing Contact, our classifiers cannot distinguish which party should be associated with which practice. Thus, performing classifications at the level of sentences may yield performance improvements. Second, the variety of technical language in privacy policies poses challenges. For example, we observed a false positive when “location” was used in the context of “co-location facility”, and a false negative when “clear gifs” was used to refer to web beacons. Such errors might be prevented by training on more data or using domain-specific word embeddings (Kumar et al. 2019). Finally, a more sophisticated semantic representation might be necessary in certain cases. For example, we observed misclassification of a sentence which said that although the first party does not perform a practice, third parties do perform the practice.

MAPS detects apps’ privacy practices at app store-wide scale. Detecting which practices an app performs relies on static code analysis, a relatively resource-efficient technique Policy Classification Contact 1stParty Contact 3rdParty Contact Email Address 1stParty Contact Email Address 3rdParty Contact Phone Number 1stParty Contact Phone Number 3rdParty Identifier 1stParty Identifier 3rdParty Identifier Cookie 1stParty Identifier Cookie 3rdParty Identifier Device ID 1stParty Identifier Device ID 3rdParty Identifier IMEI 1stParty Identifier IMEI 3rdParty Identifier IMSI 1stParty Identifier IMSI 3rdParty Identifier MAC 1stParty Identifier MAC 3rdParty Identifier Mobile Carrier 1stParty Identifier Mobile Carrier 3rdParty Identifier SIM Serial 1stParty Identifier SIM Serial 3rdParty Identifier SSID BSSID 1stParty Identifier SSID BSSID 3rdParty Location 1stParty Location 3rdParty Location Cell Tower 1stParty Location Cell Tower 3rdParty Location GPS 1stParty Location GPS 3rdParty Location WiFi 1stParty Location WiFi 3rdParty Single Sign On Single Sign On: Facebook compared to dynamic code analysis. Our system operates on four app resources: Android APIs, permissions, strings, and class structure. If a sensitive Android API is called, the app has the required permissions to make the call, and required string parameters (e.g., the GPS PROVIDER string) are passed in, the system will flag the existence of a first or third party practice depending on the package name of the class from which the call originated. We assume a threat model which considers data as compromised from the moment a privacy-sensitive API appears to be called (Neisse et al. 2016).

After downloading an app from the Google Play Store our system decompiles it into Smali bytecode using Apktool.4 It then searches through the bytecode, identifying APIs indicative of a privacy practice being performed. Generally, if a practice occurs in a package corresponding to the app’s package ID, the practice is considered a first party practice; otherwise, it is considered a third party practice. In order to evaluate the performance of our system’s app analysis, we compare its results against ground truth obtained by a man4Apktool, https://ibotpeaches.github.io/Apktool/, accessed: March 18, 2019. ual dynamic analysis.

Our system combines policy and app analysis results to identify potential compliance issues. We define a potential compliance issue to mean that an app is performing a practice (e.g., Location GPS 1stParty) while its associated policy does not disclose it either generally (e.g., “Our app accesses your location data.”) or specifically (e.g., “Our app accesses your GPS data.”). We chose this definition because we observed that policies generally either disclose that a practice is performed or omit discussion of the practice— statements denying practices are rare.

Table 4 shows our system’s identification of potential compliance issues and its performance. For the 26 practices for which positive ground truth instances were present, we observe a mean F1 score of 71%. Many potential compliance issues relate to the access of identifiers. However, the three third party location practices Cell Tower, GPS, and WiFi account for 15, 10, and 12 respective findings as well. Notably, all first party practices exhibit a lower number of potential compliance issues than their third party counterparts. Potential Compliance Issue Contact Email Address 1stParty Contact Email Address 3rdParty Contact Phone Number 1stParty Contact Phone Number 3rdParty Identifier Cookie 1stParty Identifier Cookie 3rdParty Identifier Device ID 1stParty Identifier Device ID 3rdParty Identifier IMEI 1stParty Identifier IMEI 3rdParty Identifier IMSI 1stParty Identifier IMSI 3rdParty Identifier MAC 1stParty Identifier MAC 3rdParty Identifier Mobile Carrier 1stParty Identifier Mobile Carrier 3rdParty Identifier SIM Serial 1stParty Identifier SIM Serial 3rdParty Identifier SSID BSSID 1stParty Identifier SSID BSSID 3rdParty Location Cell Tower 1stParty Location Cell Tower 3rdParty Location GPS 1stParty Location GPS 3rdParty Location WiFi 1stParty Location WiFi 3rdParty Single Sign On: Facebook

Designing and implementing a robust system to identify potential compliance issues for large app populations presents challenges of scale. We address those with a pipeline of distributed tasks implemented in a containerized software stack. We performed our Play Store analysis from April 6 to May 15, 2018. Out of 1,049,790 retrieved free apps, 1,035,853 (98.67%) were analyzed successfully. Of the apps which were not analyzed successfully, 1.03% failed to download, 0.21% failed in the static analysis, 0.08% failed in the policy analysis, and 0.01% failed during our re-analysis.5 35.3% of the apps we analyzed had privacy policies.6 For apps with privacy policies, Figure 2 depicts the occurrence of policy statements relating to the practices we examine. It can be observed that most practices are described only infrequently; that is, a policy does not mention it at least once. Further, the statements that are present typically affirm that a practice is occurring. This finding reveals that users seem to be given little assurance of potentially objectionable practices not being performed (e.g., disclosing users’ phone numbers to third parties). Silence about privacy practices in privacy policies is problematic because there are no clear statutory default rules of what the privacy relationship between a user and a service should be, in the absence of explicit statements in the policy (Marotta-Wurgler 2015) .

Our system detects potential compliance issues by comparing the privacy policy analysis to the static analysis. A potential compliance issue is detected when an app performs a practice that is not described in the app’s privacy policy (if the app even has a privacy policy). Note that when our system finds multiple privacy policies for a given app, it pools 5After the completion of the Play Store analysis we noticed a bug in our static analysis code. As a result, we re-performed the static analyses and re-calculated all statistics. 135 additional analyses failed, yielding a final total of 1,035,853 successfully analyzed apps.

6This only counts English-language privacy policies: our system does not identify policies in other languages. the practice descriptions discovered across all those policies. This pooling has the effect of making our results rather conservative. One policy may disclose a particular practice while another policy discloses another practice, and together they may cover all practices performed by the associated app. Overall, the average number of potential compliance issues per app is 2.89 and the median is 3.

Figure 3 shows the percent of apps that perform various practices and the respective percent of apps with potential compliance issues. The figure demonstrates that in many cases the performance of a practice is strongly associated with the occurrence of a potential compliance issue: if a practice is performed, there is a good chance a potential compliance issue exists as well. This result suggests a broad level of potential non-compliance. Identifier-related potential compliance issues are the most common. Three different types of identifiers make up most potential compliance issues: cookies, device IDs, and mobile carriers. In particular, the use of device IDs may constitute a misuse for purposes of ad tracking (Google 2018b) . In addition, there are also elevated levels of location-related potential compliance issues. 15.3% of apps perform at least one location-related practice, and 12.1% of apps have at least one location-related potential compliance issue.

For all data types, third party practices are more common than first party practices and so are third party-related potential compliance issues. One reason for the prevalence of potential compliance issues for third party practices could be that app developers are unaware of the functionality of the libraries they integrate. Perhaps they also hold the mistaken belief that it is not their responsibility but the responsibility of the library developers to disclose to users the practices the libraries are performing. Some libraries’ terms of services—for example, the Google Analytics Terms of Service (Google 2018a) —obligate the developer integrating it to explicitly disclose the integration in the developer’s privacy policy. However, this type of information transfer from the third party via the developer to the user may be susceptible to omissions and mistakes.

5