Current artificial intelligence and information retrieval systems need to be trained with a large amount of data to achieve satisfying performance. A popular solution to create such datasets is to employ crowdsourcing; however, the content to be annotated may contain private or sensitive information that can be extracted by workers, limiting the applicability of crowdsourcing data annotation techniques in privacysensitive contexts. In this paper, we survey the literature finding that current solutions in crowdsourcing and machine learning do not provide satisfactory solutions as they either hinder the capabilities of workers to annotate the data, increase the overall cost, or lack generalizability. We identify current challenges, propose and elaborate a hybrid human-machine approach to detect private information in images, discuss its features and propose future directions.
Crowdsourcing is currently employed for content annotation in various domains: generating ground-truth data for machine learning models, enrichment of multimedia data for better content retrieval and exploration, opinion mining, etc. Crowdsourcing tasks are typically performed by anonymous workers operating online microwork marketplaces (e.g., Amazon Mechanical Turk), or by contract workers operating on private platforms (e.g., oDesk, freelancer.com). A problem common to both types of platforms concerns potential exposure to workers of private information regarding the requester (i.e., the entity having the need for content annotation) or other people related to the content (e.g. owners of invoices to be digitized).
This is especially true for images; consider for instance a crowdsourcing task where workers are asked to transcribe the text contained in an image, such as in Figure 1. There, workers can access the information required to complete the task (i.e., the text on the banner), while being able to infer a wide range of private information: the identity of the people from their faces; their political alignment and, to some extent, where they live from the sign. A real-world example is the case of the security cameras sold by the Ring startup. It was recently discovered that to train and improve the performance of their machine learning model, the employees had total and unfiltered access to the live feed of the cameras 1.
While current research on privacy preservation in crowdsourcing focuses on worker
privacy [
Automated methods for privacy preservation can help in terms of eficiency; however, their
utility is limited by the quality of their output, by the harmful mistakes they might make in the
privacy context, and by their need for a large amount of training data that forces to trade-of
between cost and generalization abilities. This is particularly true for images, where works
focus on obfuscating only particular instances of private data (e.g., faces) [
Works in the crowdsourcing domain deal with privacy by either obfuscating entirely the
content of the tasks [
In this paper, we first contribute with a survey on the state-of-the-art methods to protect privacy in the content shown in crowdsourcing tasks. We investigate how current work in crowdsourcing deals with the private information contained in the task content, and we explore which automatic methods could be used to protect privacy in crowdsourcing tasks.
We elaborate on the strengths and weaknesses of diferent classes of approaches, and we propose a design of a human-machine pipeline to detect and obfuscate private content in images more efectively and eficiently than existing methods. We envision a combination of pre-trained computer vision models for understanding image content in order to reduce the cost of developing the system (i.e., no need for developing a new training dataset and training a new deep learning model) and capabilities of visual and logical reasoning of humans.
We adopted a Systematic Literature Review approach [12], searching papers through Google Scholar.
We strove for a balance between crowdsourcing and machine learning approaches. On average, we explored 30 result pages for searches including keywords “privacy” and (“preservation” or “preserving”) and “crowdsourcing” and (“guarantees” or “tasks” or “surveys”). Similarly, works related to privacy preservation in machine learning are searched with the keyword combination of “privacy” and “preservation” and “machine learning” and (“surveys” or “images” or “personal information”).
Using a snowball strategy, we explored references contained in the retrieved papers. We disregard works addressing the problem of preserving the privacy of workers in crowdsourcing. In the search of papers on machine learning approaches, we exclude methods focusing on machine-to-machine privacy preservation (i.e., privacy-aware version of existing algorithms) and works that images completely unintelligible.
We classify the resulting papers according to the adopted methods for private information detection, and techniques for privacy preservation.
Crowdsourcing works that deal with privacy preservation attempt to make the crowdsourcing workflow privacy-aware by either obfuscating the content of the task (e.g. by applying filters such as blur and visual noise), or segmenting it in small patches in such a way that workers never see the whole content.
With images, the majority of works do not focus on information detection, but on information preservation, by studying how diferent types of obfuscation methods influence the quality of the responses of the crowd.
Varshney et al., [
In [
Works focusing on the detection of private information, use inputs either from the requester
or the crowd itself. In Zensors++ [
CrowdMask [
In this section, we review automatic and machine learning methods that can be used to detect and preserve privacy in the content used in crowdsourcing tasks.
We can identify two main categories of research: studies that focus on building machine learning models to detect private information and works dedicated to create methods to obfuscate the private information while not disrupting the utility of the image.
An attempt to create a general privacy detection model was done by Orekondi et al. [
Recently, another dataset was released by Gurari et al. [15] The pictures were obtained from their VizWiz real-time visual question answering system [16] , and they were manually annotated by the authors using a 23 categories taxonomy.
Uittenbogaard et al. [17] developed a method to detect, remove and inpaint private visual cues in street-level imagery (e.g., people and vehicles). Assuming that in this kind of image moving objects are the ones at risk of disclosing private information, they develop a method that detects moving objects by exploiting inconsistencies between diferent frames.
Work developing new obfuscation methods focuses on specific instances of private
information (e.g., faces). Generative adversarial networks have been used to anonymize faces by
removing privacy-sensitive information [
Currently, works in crowdsourcing deal with privacy by either obfuscating entirely the content
of the tasks, making it more dificult for the workers to complete their work, or requiring input
from the requester, sufering from scalability problems. Approaches based on segmentation
seem promising since they have a minor impact on the performance of the workers. On the other
hand, they cost more, since workers need to examine patches and not the whole content. Most
of the automatic approaches focus on blocking private information disregarding its detection.
For this reason, they are tied to specific use cases (e.g., obfuscating faces). To the best of
our knowledge, a general-purpose model for privacy preservation is still missing. The model
presented in [
We identify two main challenges toward an efective detection of private information: 1. The lack of a clear understanding of what private information is. While this is obvious for text, connecting the visual cues present in an image with certain classes of private information is more dificult. A face can be straightforwardly connected to a person’s identity, but what about the clothes someone is wearing? Or the items present in a room? As stated in the General Data Protection Regulation (GDPR)2, not only information directly identifying a person should be protected, but also any clue about sensitive or personal information. Defining privacy is a complex problem, as it requires translating the concepts described in policies and laws into entities and relations understandable by machines. 2. The lack of a comprehensive dataset to train a general-purpose machine learning model.
The datasets reviewed in this paper [
Concluding, state-of-the-art shows privacy preservation in the context of crowdsourcing tasks cannot be solved efectively and eficiently by the crowd or machine learning alone. We argue that by combining the flexibility of the crowd with the cost-efectiveness of automatic approaches it is possible to improve the performance with respect to the state of the art. Hybrid human-machines solutions have been successful in many fields [ 19], but, to the best of our knowledge, their application for privacy preservation is still unexplored.
To understand what private information is, and to provide an unbiased definition, we look at definitions present in the General Data Protection Regulation (GDPR)3, a legal act dictating how personal data needs to be protected during its collection and processing. In this policy personal data is defined as follow: (personal data) is any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person
Here we can identify private information (e.g., age, face, address, etc..), sensitive information (e.g, sexual orientation, trade or political memberships), or cues that can be used to infer personal information (e.g, economic, physiological, social factors like lifestyle, etc).
While this is a step forward in framing the concept of private information, it is not precise enough in the case of images. The identity of a person can easily be connected to his/her face, but what about his or her economic, cultural or social status?
To get a better understanding we use the Data Privacy Vocabulary [20] drafted by the Data Privacy Vocabularies and Controls Community Group. The vocabulary provides classes and properties describing instances of private data, the process they undergo, and who manages them. In this work, we consider only the categories of data (i.e., the Personal Data Category class) and we limit the scope to the ones that can reasonably appear in images. We exclude classes referring to audio (e.g., Voice Communication Recording), biological (e.g., DNACode) and device-based tracking data (e.g., IPAddress).
Private data is divided in six main categories: Internal, External, Financial, Social, Tracking and Historical. Internal comprehends all the information about an individual that - usually - cannot be observed because they are either kept secret (i.e., Authenticating data such as passwords or pincode) or they are part of someone set of believes (e.g., Religous Belief, Opinion, etc..).
External includes all the characteristics of a person that can be from his/her physical appearance such as Demographic, PhysicalCharacteristic and Ethnicity. Financial refers to information about monetary transactions and ownership. Social include data about social aspects of an individual like family, public life, or professional networks. Tracking refers to the information that can be used to detect the location of an individual. Finally, Historical refers to the historical events a person witnessed.
In this section, we propose a hybrid human-machine approach to detect and obfuscate private information in images, where machine learning models inform the process of involving human
Image
Object Detection Scene Recogniton
OCR Obfuscation and Segmentations
Image Objects
Scenes
Text Segment
Segment
Segment Privacy Risk
Computation Crowdsourcing
Image Image
Privacy Annotations
Updated Annotations intelligence with the generation of crowdsourcing tasks.
Previous works to detect private information in images have shown that machine learning approaches are costly and non-robust since they require building large training datasets, and the trained models can sufer from non-generalization issues to other datasets due to biases. That is why we choose to leverage pre-trained machine learning models as they do not add any cost to the pipeline, while there are many diverse ones available publicly.
Our intuition is that even out-of-the-box machine learning models can provide useful information about the private content included in an image, even though their outputs do not directly map to private information.
For instance, an object detection model such as Mask R-CNN [21] can provide labels as person - that can be easily mapped to the entities described in the previous section - or laptop that can indicate the presence of Authenticating of Financial information, as they may appear on the screen. Also a scene detection model [22] can provide useful insight. For example, a scene detected as indoor may contain information about the work environment, confidential documents, and computer screens.
In addition, work in the crowdsourcing domain show that approaches based on segmentation seems to be efective in hiding private information from the workers, with the negative aspect, however, of an increased cost. Here our idea is that the annotations provided by the machine learning models can be used to reduce the number of patches the crowd needs to analyze by either find part of the image that does not contain any element, or, on the contrary, flagging sections of the images where there are visual cues of private information (e.g, a face).
For these reasons, we envision our pipeline - shown in Figure 2 composed by these steps. First, an ensemble of pre-trained machine learning models for object detection, scene detection, and optical character recognition is used to detect the diferent visual cues in the image. The output consists of: object labels, object masks/bounding boxes, texts in the image, scene attributes, and scene categories along with the confidence scores of the detected objects and scene annotations. The idea is that the label produced by the machine learning models can guide the segmentation and the crowdsourcing phase by providing a machine-readable description of the scene.
For example, let us consider an image of a workplace that has objects like a computer, and few documents on the table - the associate scene attributes for this image from scene detection will be man-made, paper, working, studying, research, etc. give context to the image based on which we perform segmentation. The annotation is mapped to the private entities described in Section 3.1.
The result of this mapping, together with the confidence score of the machine learning models, is used to compute a privacy leak risk score of the image. The scene detection model contributes to a global score, as - intuitively - photos of some places are more at risk of showing sensitive information than others (e.g, an ofice versus a landscape). If the local score (combined with the global) is above a threshold, the object mask is obfuscated.
Then, the image is segmented in small patches, and only the segments that are not completely
obscured - and have a privacy leak score falling in a given range - are sent to the crowd for
further evaluation. Similar to previous works [
Machine Learning Models Obfuscation and
Segmentations Privacy Concepts Mapping Privacy Risk Computation
Crowdsourcing (2) (4) (1) (3)
To better illustrate our approach, Figure 3 shows a running example of the pipeline. First, the machine learning models4 detect people in the picture and extract the text present in the banner (1). Then, we use the mapping with the privacy concepts to identify where in the image they appear (2). If the algorithm is confident enough, it obfuscates the part of the image showing those concepts, then it segments it in small patches for the crowd for further evaluation. Notice 4To keep it simple we only consider an object detection model and OCR that patches that are fully obscured are not sent (3). Finally, the crowd further analyzes the segments and flags the one containing private information (4).
Each step of the pipeline has hyperparameters on which to experiment in order to analyze the efectiveness and eficiency of the component. Such hyperparameters are the following. Choice of the machine learning models Diferent machine learning models allow us to get diferent information from an image. Object detection algorithms can find relevant visual elements present in the image, scene detection models can give an additional indication on the setting shown in the picture, while OCR can extract possible sensible or confidential information from the text.
ways: it can be manually built considering the machine learning model used - i.e., hard-coded -, or automatically defined using, for instance, a rule-based approach - i.e., the logic combination of terms correspond to a private term -, a probabilistic method - i.e., a term has a probability to refer to a private attribute - or learned - i.e., the probability of the terms are learned in an iteratively way during the process.
Threshold for the privacy risk score This parameter regulates when a part of an image needs to be obfuscated, sent to the crowd, or be considered safe. A high threshold will lead to parts of the image be wrongly assumed as safe or with many segments sent to the crowd for further analysis; while a value too low will make the pipeline preventively obscure the majority of the image.
Design of the crowdsourcing task The design of the crowdsourcing task can greatly influence the outcome of the whole pipeline. The amount incentives, the design of the interface, including the type of task the crowd needs to perform impact the quality of the results and define the type of annotations provided.
In this paper, we argue the task of privacy preservation in images cannot be solved efectively and eficiently by the crowd or machine learning alone. We surveyed the state of the art both in crowdsourcing and machine learning domains and found out the main challenges related to privacy preservation lie in the complexity of understanding what privacy is, and in the dificulty of creating a dataset to train machine learning models.
To address these challenges, we then propose a hybrid human-machine pipeline to detect private information in images. The pipeline utilizes pre-trained machine learning models to detect visual cues of private information in images and uses crowdsourcing to detect private information the machine learning models may have missed. In this way, it harnesses the strength of both approaches by combining the eficiency of automatic methods and the flexibility of human intelligence.
Future works will focus on the implementation and deployment of the before-mentioned pipeline to validate the approach and refine, through detailed experimentation, the various components. We will investigate how diferent machine learning models and crowdsourcing task designs influence the outcome of the pipeline. Finally, we will evaluate the impact of privacy preservation on the performances of artificial intelligence and information retrieval system. [12] D. Gough, S. Oliver, J. Thomas, An Introduction to Systematic Reviews, SAGE Publications, 2017. URL: https://books.google.nl/books?id=41sCDgAAQBAJ. [13] T.-Y. Wang, Y. Han, P. Varshney, P.-N. Chen, Distributed fault-tolerant classification in wireless sensor networks, IEEE Journal on Selected Areas in Communications 23 (2005) 724–734. doi:10.1109/JSAC.2005.843541. [14] A. Alshaibani, S. Carrell, L.-H. Tseng, J. Shin, A. Quinn, Privacy-preserving face redaction using crowdsourcing, Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 8 (2020) 13–22. URL: https://ojs.aaai.org/index.php/HCOMP/article/view/ 7459. [15] D. Gurari, Q. Li, C. Lin, Y. Zhao, A. Guo, A. Stangl, J. P. Bigham, Vizwiz-priv: A dataset for recognizing the presence and purpose of private visual information in images taken by blind people, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 939–948. [16] J. P. Bigham, C. Jayant, H. Ji, G. Little, A. Miller, R. C. Miller, R. Miller, A. Tatarowicz, B. White, S. White, et al., Vizwiz: nearly real-time answers to visual questions, in: Proceedings of the 23nd annual ACM symposium on User interface software and technology, ACM, 2010, pp. 333–342. [17] R. Uittenbogaard, C. Sebastian, J. Vijverberg, B. Boom, D. M. Gavrila, et al., Privacy protection in street-view panoramas using depth and multi-view imagery, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 10581– 10590. [18] Z. Chen, T. Zhu, C. Wang, W. Ren, P. Xiong, Gan-based image privacy preservation: Balancing privacy and utility, in: X. Chen, H. Yan, Q. Yan, X. Zhang (Eds.), Machine Learning for Cyber Security, Springer International Publishing, Cham, 2020, pp. 287–296. [19] J. W. Vaughan, Making better use of the crowd: How crowdsourcing can advance machine learning research., Journal of Machine Learning Research 18 (2017) 193–1. [20] Data Privacy Vocabularies and Controls Community Group, Data privacy vocabulary v0.1, 2019. https://www.w3.org/ns/dpv, Last accessed on 2021-07-01. [21] K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask r-cnn, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961–2969. [22] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, A. Torralba, Places: A 10 million image database for scene recognition, IEEE transactions on pattern analysis and machine intelligence 40 (2017) 1452–1464.