Maintaining perceptual faithfulness of adversarial image examples by leveraging color variance Sam Sweere, Zhuoran Liu1 , Martha Larson1 1 Radboud University, Netherlands samsweere@gmail.com,z.liu@cs.ru.nl,m.larson@cs.ru.nl ABSTRACT With the popularity of social networks, large scale user-generated data is accumulated online. Possible misappropriation of these data may arise severe personal privacy problems. In this paper, we pro- pose an image transformation method that protects images against scene classifier by exploiting the knowledge of adversarial examples. At the same time, our method maintains the perceptual faithfulness Original image Color variations Top1 with L1 of protected images by leveraging color variance. 0.804 -> swimming_pool/outdoor 0.369 -> swimming_pool/indoor 0.094 -> swimming_pool/indoor 0.363 -> swimming_pool/outdoor 1 INTRODUCTION Modern machine learning algorithms are able to extract privacy sensitive information from user-generated multimedia data that is available online, such as geo-location [1, 4]. The objective of the Pixel Privacy Task of MediaEval 2019 [5] is to find solutions that could protect privacy-sensitive information in images against Top1 without L1 Top5 with L1 Top 5 without L1 scene classifier, and at the same time keep or increase the visual 0.295 -> swimming_pool/indoor 0.429 -> carrousel 0.562 -> carrousel 0.259 -> swimming_pool/outdoor 0.276 -> water_park 0.270 -> amusement_park appeal of these adversarial images. The provided test-dataset is a subset of the Places365-Standard dataset [11]. This paper proposes Figure 1: Example of the protected images that meet differ- a method to create adversarial examples by perturbing the pixels ent protective conditions. Top 2 predicted labels and proba- in the image based on color variation, which is better aligned with bilities are listed below each example. human perception. two colors based on human perception than for example the Eu- clidean distance. A Gaussian distribution can be used to weight 2 RELATED WORK the contribution of the surrounding pixels in order to smooth the Neural network-based algorithms have weaknesses that make them color variance between two neighboring pixels. Pixels further away susceptible to adversarial examples [2]. By perturbing specific pixels contribute less to the total color difference. in the image, the resulting outcome of the network can be changed without a substantial change to the image. The knowledge of ad- versarial examples can also be applied to protect privacy sensitive 3 APPROACH information. PIRE [6] is an iterative method to generate adversarial In this section, we describe our perturbation-based protection algo- images, but it does not consider the human noticeability of these rithm in detail. In particular, we discuss the construction of model perturbed pixels. One solution to this problem is to consider the loss, hyper-parameter selection and training details. human perception of these perturbations. [7] suggests that perturb- ing pixels in low variance regions (i.e. white walls or blue skies) is 3.1 Protection by perturbation more noticeable than perturbing pixels in high variance regions (i.e. Pixel values of images can be perturbed in a certain way to influence a brick wall). In [7] this is implemented by calculating the standard the predicted label by the threat model when generating adversarial deviation around a pixel based on its intensity (greyscale value). examples. Given the ground truth label, we could optimize the per- However, in this case, the color-specific information is discarded. turbations iteratively until the predicted class meets our adversarial The CV-PIRE method [9] suggests an approach that takes the hu- condition (i.e. fall out of top 2 or top 5). To minimize the noticeabil- man perception of colors into account, using high and low color ity of the perturbations, we include a threshold that determines the variance regions. maximum perturbation of one pixel. If we set the same threshold for Pixel color variance can be calculated by using CIEDE2000 [8] dif- every pixel in the image we may get very noticeable perturbations ference between a specific pixel and the surrounding pixels. The in low color variance regions, while the perturbations in high color CIEDE2000 algorithm gives a better numerical distance between variance regions are less noticeable. In our method, we follow the threshold method [9] in which CIEDE2000-based color variation is Copyright 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution used to generate threshold map (e.g., second figure of top row in 4.0 International (CC BY 4.0). Figure 1). MediaEval’19, 27-29 October 2019, Sophia Antipolis, France MediaEval’19, 27-29 October 2019, Sophia Antipolis, France S. Sweere et al. Algorithm 1: Color Variance based perturbations Table 1: Evaluation results for five submitted runs. Input: Image imд with ground truth label c 0 ; Color variance Top-1 Top-5 Aesthetics SSIM matrix cV ; Classification model f ; Threshold acc. (%) acc. (%) score. multiplier m; Weight of L1-loss α ; Maximum iterations Original 1.00 1.00 4.64 1.00 T ; Cross-entropy loss function lc e and protection Top1-L1 0.00 0.99 4.63 0.9994 condition; Top1-L1-50 0.00 0.93 4.62 0.9992 Output: Generated adversarial example image; Top1-noL1 0.00 0.52 4.61 0.9975 Set the final per pixel threshold matrix; Top5-L1 0.00 0.00 4.65 0.9614 pT = m · cV ; Top5-noL1 0.00 0.00 4.81 0.8955 Set the initial random perturbations and iteration counter; v = random(−0.01, 0.01) ∗ pT ; possible to meet the condition within the maximum interactions t = 0; we return False. while t < T do loдits = f ((imд + v).clamp(0, 1)); 4 SUBMISSION RESULTS AND ANALYSIS if condition is met then We submitted five runs for the Pixel Privacy task: Top1 with L1- return imд + v ; norm (Top1-L1), Top1 with L1-norm and a 50 percent relative dif- ||v ||1 ference in prediction confidence compared to the top1 label (Top1- loss = −1 · lc e (loдits, c 0 ) + α · ; L1-50), Top1 without the L1-norm (Top1-noL1), Top5 with L1-norm ||pT ||1 (Top5-L1) and Top5 without the L1-norm (Top5-noL1). We noticed | {z } | {z } Cross-entropy loss L1 norm loss two potential flaws in the top1 with L1-norm, first average differ- Update the perturbations with an optimizer; ence in the confidence between the ground-truth label and the new v = arдmin loss; top1 label is often small and the new top1 label is often similar v to the ground-truth label, this can also been seen in Figure 1. To Clip and round to stay within threshold and image counter these potential weaknesses we included the Top1-L1-50 relevance boundaries; r ound (cl ip(v,−pT ,pT )·255) run to increase the distance in confidence and the Top5-L1 and v= 255 ; Top5-noL1 runs, that make sure the ground-truth label is not in the t = t + 1; top5. We included the Top2-noL1 and Top5-noL1 runs since these Failed to generate an adversarial example in T iterations; need less computational resources. return False; Table 1 presents the results of the different conditions. As can be observed, all the adversarial images achieved their goal of top-1 or top-5. Following the official evaluation rule, Top5-noL1 achieves 3.2 Color variance-based approach the best result. We also include the Structural similarity (SSIM) [10] Algorithm 1 demonstrates how the adversarial examples are gen- score, this measures the perceptual difference between the original erated. Here cV is the pixel color variance matrix for the image, f image and its adversarial counterpart, this could be interperted as the classification model, in our case a ResNet-50 [3] classifier, m how faithfull the adversarial image is to the original. threshold multiplier that determines how big the actual threshold is relative to the color variance, T the maximum iterations, α the 5 DISCUSSION AND OUTLOOK weight of the L1-norm loss in the total loss function and condition The perturbations of especially the Top1 are small, as can also been the adversarial condition which will later be discussed. seen in the SSIM score. This could cause the robustness of the ad- Using the color variance matrix cV as in [9], we define the per pixel versarial images under image transformations such as compression threshold that represents the maximum perturbation per pixel. The or filters to be weak. initial perturbations are randomly set based on this pixel threshold. As can been seen in Table 1 there is a clear difference between We update the perturbations until the condition is met or the maxi- the aesthetics score and SSIM. SSIM score could represent how mum amount of iterations is reached. Iteratively calculate the logits noticeable the perturbations are, where in the Top1-L1 the pertur- of the image with perturbations, the clamping is done such that bations are least to barely noticeable and the Top5-noL1 have the image stays within the valid image range. If the condition is met most noticeable perturbations. However, the aesthetics score is the based on the logits then we have a successfully adversarial example highest on Top5-noL1, which could mean that the locations where and we stop the loop. If not, we calculate the loss, this consists the perturbations take place in our method are creating somewhat of the cross-entropy loss lce , where the goal is to minimize loss of an adversarial example to the aesthetics score method. function given the ground truth label, and the L1 norm loss, which Further research could look at possible conditions where all the minimizes the total amount of perturbations. The perturbations top predictions would be of a different scenes that are not closely are updated by back-propagation to minimize this loss. Finally, the related to the ground-truth scene. To increase the usability of this updated perturbations are clipped to make sure they stay within approach in practice, the robustness of protected images should the pixel value range of image and rounded such that the perturba- be improved. Data augmentation of original image and ensemble tions would remain when saved in a uint8 image format. If it is not training against different threat models can be considered in the future. Pixel Privacy MediaEval’19, 27-29 October 2019, Sophia Antipolis, France REFERENCES [1] Jaeyoung Choi, Martha Larson, Xinchao Li, Kevin Li, Gerald Friedland, and Alan Hanjalic. 2017. The Geo-Privacy Bonus of Popular Photo En- hancements. In ACM International Conference on Multimedia Retrieval (ICMR). ACM, 84–92. [2] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explain- ing and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR). [3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 770– 778. [4] Martha Larson, Mohammad Soleymani, Pavel Serdyukov, Stevan Rud- inac, Christian Wartena, Vanessa Murdock, Gerald Friedland, Roeland Ordelman, and Gareth JF Jones. 2011. Automatic tagging and geo- tagging in video collections and communities. In ACM International Conference on Multimedia Retrieval (ICMR). ACM. [5] Zhuoran Liu, Zhengyu Zhao, and Martha Larson. 2019. Pixel Privacy 2019: Protecting Sensitive Scene Information in Images. In Working Notes Proceedings of the MediaEval 2019 Workshop. [6] Zhuoran Liu, Zhengyu Zhao, and Martha Larson. 2019. Who’s Afraid of Adversarial Queries? The Impact of Image Modifications on Content-based Image Retrieval. In ACM International Conference on Multimedia Retrieval (ICMR). ACM. [7] Bo Luo, Yannan Liu, Lingxiao Wei, and Qiang Xu. 2018. Towards imperceptible and robust adversarial example attacks against neural networks. In AAAI Conference on Artificial Intelligence (AAAI). [8] M Ronnier Luo, Guihua Cui, and Bryan Rigg. 2001. The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Research & Application 26, 5 (2001), 340–350. [9] Sam Sweere. 2019. Increasing the Perceptual Image Quality of Ad- versarial Queries for Content-based Image Retrieval. Bachelor The- sis. Radboud University Nijmegen, the Netherlands. Available at: https://github.com/SamSweere/CV-PIRE. [10] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. [11] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million image database for scene recogni- tion. IEEE transactions on pattern analysis and machine intelligence 40, 6 (2017), 1452–1464.