=Paper=
{{Paper
|id=Vol-2670/MediaEval_19_paper_25
|storemode=property
|title=Maintaining Perceptual Faithfulness of Adversarial Image Examples by Leveraging Color
Variance
|pdfUrl=https://ceur-ws.org/Vol-2670/MediaEval_19_paper_25.pdf
|volume=Vol-2670
|authors=Sam Sweere,Zhuoran
Liu,Martha Larson
|dblpUrl=https://dblp.org/rec/conf/mediaeval/SweereLL19
}}
==Maintaining Perceptual Faithfulness of Adversarial Image Examples by Leveraging Color
Variance==
https://ceur-ws.org/Vol-2670/MediaEval_19_paper_25.pdf
Maintaining perceptual faithfulness of adversarial image
examples by leveraging color variance
Sam Sweere, Zhuoran Liu1 , Martha Larson1
1 Radboud University, Netherlands
samsweere@gmail.com,z.liu@cs.ru.nl,m.larson@cs.ru.nl
ABSTRACT
With the popularity of social networks, large scale user-generated
data is accumulated online. Possible misappropriation of these data
may arise severe personal privacy problems. In this paper, we pro-
pose an image transformation method that protects images against
scene classifier by exploiting the knowledge of adversarial examples.
At the same time, our method maintains the perceptual faithfulness
Original image Color variations Top1 with L1
of protected images by leveraging color variance. 0.804 -> swimming_pool/outdoor 0.369 -> swimming_pool/indoor
0.094 -> swimming_pool/indoor 0.363 -> swimming_pool/outdoor
1 INTRODUCTION
Modern machine learning algorithms are able to extract privacy
sensitive information from user-generated multimedia data that
is available online, such as geo-location [1, 4]. The objective of
the Pixel Privacy Task of MediaEval 2019 [5] is to find solutions
that could protect privacy-sensitive information in images against
Top1 without L1 Top5 with L1 Top 5 without L1
scene classifier, and at the same time keep or increase the visual 0.295 -> swimming_pool/indoor 0.429 -> carrousel 0.562 -> carrousel
0.259 -> swimming_pool/outdoor 0.276 -> water_park 0.270 -> amusement_park
appeal of these adversarial images. The provided test-dataset is a
subset of the Places365-Standard dataset [11]. This paper proposes Figure 1: Example of the protected images that meet differ-
a method to create adversarial examples by perturbing the pixels ent protective conditions. Top 2 predicted labels and proba-
in the image based on color variation, which is better aligned with bilities are listed below each example.
human perception.
two colors based on human perception than for example the Eu-
clidean distance. A Gaussian distribution can be used to weight
2 RELATED WORK the contribution of the surrounding pixels in order to smooth the
Neural network-based algorithms have weaknesses that make them color variance between two neighboring pixels. Pixels further away
susceptible to adversarial examples [2]. By perturbing specific pixels contribute less to the total color difference.
in the image, the resulting outcome of the network can be changed
without a substantial change to the image. The knowledge of ad-
versarial examples can also be applied to protect privacy sensitive
3 APPROACH
information. PIRE [6] is an iterative method to generate adversarial In this section, we describe our perturbation-based protection algo-
images, but it does not consider the human noticeability of these rithm in detail. In particular, we discuss the construction of model
perturbed pixels. One solution to this problem is to consider the loss, hyper-parameter selection and training details.
human perception of these perturbations. [7] suggests that perturb-
ing pixels in low variance regions (i.e. white walls or blue skies) is 3.1 Protection by perturbation
more noticeable than perturbing pixels in high variance regions (i.e.
Pixel values of images can be perturbed in a certain way to influence
a brick wall). In [7] this is implemented by calculating the standard
the predicted label by the threat model when generating adversarial
deviation around a pixel based on its intensity (greyscale value).
examples. Given the ground truth label, we could optimize the per-
However, in this case, the color-specific information is discarded.
turbations iteratively until the predicted class meets our adversarial
The CV-PIRE method [9] suggests an approach that takes the hu-
condition (i.e. fall out of top 2 or top 5). To minimize the noticeabil-
man perception of colors into account, using high and low color
ity of the perturbations, we include a threshold that determines the
variance regions.
maximum perturbation of one pixel. If we set the same threshold for
Pixel color variance can be calculated by using CIEDE2000 [8] dif-
every pixel in the image we may get very noticeable perturbations
ference between a specific pixel and the surrounding pixels. The
in low color variance regions, while the perturbations in high color
CIEDE2000 algorithm gives a better numerical distance between
variance regions are less noticeable. In our method, we follow the
threshold method [9] in which CIEDE2000-based color variation is
Copyright 2019 for this paper by its authors. Use
permitted under Creative Commons License Attribution
used to generate threshold map (e.g., second figure of top row in
4.0 International (CC BY 4.0). Figure 1).
MediaEval’19, 27-29 October 2019, Sophia Antipolis, France
MediaEval’19, 27-29 October 2019, Sophia Antipolis, France S. Sweere et al.
Algorithm 1: Color Variance based perturbations Table 1: Evaluation results for five submitted runs.
Input: Image imд with ground truth label c 0 ; Color variance Top-1 Top-5 Aesthetics
SSIM
matrix cV ; Classification model f ; Threshold acc. (%) acc. (%) score.
multiplier m; Weight of L1-loss α ; Maximum iterations Original 1.00 1.00 4.64 1.00
T ; Cross-entropy loss function lc e and protection Top1-L1 0.00 0.99 4.63 0.9994
condition; Top1-L1-50 0.00 0.93 4.62 0.9992
Output: Generated adversarial example image; Top1-noL1 0.00 0.52 4.61 0.9975
Set the final per pixel threshold matrix; Top5-L1 0.00 0.00 4.65 0.9614
pT = m · cV ; Top5-noL1 0.00 0.00 4.81 0.8955
Set the initial random perturbations and iteration counter;
v = random(−0.01, 0.01) ∗ pT ; possible to meet the condition within the maximum interactions
t = 0; we return False.
while t < T do
loдits = f ((imд + v).clamp(0, 1));
4 SUBMISSION RESULTS AND ANALYSIS
if condition is met then We submitted five runs for the Pixel Privacy task: Top1 with L1-
return imд + v ; norm (Top1-L1), Top1 with L1-norm and a 50 percent relative dif-
||v ||1 ference in prediction confidence compared to the top1 label (Top1-
loss = −1 · lc e (loдits, c 0 ) + α · ; L1-50), Top1 without the L1-norm (Top1-noL1), Top5 with L1-norm
||pT ||1
(Top5-L1) and Top5 without the L1-norm (Top5-noL1). We noticed
| {z }
| {z }
Cross-entropy loss
L1 norm loss two potential flaws in the top1 with L1-norm, first average differ-
Update the perturbations with an optimizer; ence in the confidence between the ground-truth label and the new
v = arдmin loss; top1 label is often small and the new top1 label is often similar
v to the ground-truth label, this can also been seen in Figure 1. To
Clip and round to stay within threshold and image
counter these potential weaknesses we included the Top1-L1-50
relevance boundaries;
r ound (cl ip(v,−pT ,pT )·255)
run to increase the distance in confidence and the Top5-L1 and
v= 255 ; Top5-noL1 runs, that make sure the ground-truth label is not in the
t = t + 1; top5. We included the Top2-noL1 and Top5-noL1 runs since these
Failed to generate an adversarial example in T iterations; need less computational resources.
return False; Table 1 presents the results of the different conditions. As can be
observed, all the adversarial images achieved their goal of top-1 or
top-5. Following the official evaluation rule, Top5-noL1 achieves
3.2 Color variance-based approach
the best result. We also include the Structural similarity (SSIM) [10]
Algorithm 1 demonstrates how the adversarial examples are gen- score, this measures the perceptual difference between the original
erated. Here cV is the pixel color variance matrix for the image, f image and its adversarial counterpart, this could be interperted as
the classification model, in our case a ResNet-50 [3] classifier, m how faithfull the adversarial image is to the original.
threshold multiplier that determines how big the actual threshold
is relative to the color variance, T the maximum iterations, α the 5 DISCUSSION AND OUTLOOK
weight of the L1-norm loss in the total loss function and condition
The perturbations of especially the Top1 are small, as can also been
the adversarial condition which will later be discussed.
seen in the SSIM score. This could cause the robustness of the ad-
Using the color variance matrix cV as in [9], we define the per pixel
versarial images under image transformations such as compression
threshold that represents the maximum perturbation per pixel. The
or filters to be weak.
initial perturbations are randomly set based on this pixel threshold.
As can been seen in Table 1 there is a clear difference between
We update the perturbations until the condition is met or the maxi-
the aesthetics score and SSIM. SSIM score could represent how
mum amount of iterations is reached. Iteratively calculate the logits
noticeable the perturbations are, where in the Top1-L1 the pertur-
of the image with perturbations, the clamping is done such that
bations are least to barely noticeable and the Top5-noL1 have the
image stays within the valid image range. If the condition is met
most noticeable perturbations. However, the aesthetics score is the
based on the logits then we have a successfully adversarial example
highest on Top5-noL1, which could mean that the locations where
and we stop the loop. If not, we calculate the loss, this consists
the perturbations take place in our method are creating somewhat
of the cross-entropy loss lce , where the goal is to minimize loss
of an adversarial example to the aesthetics score method.
function given the ground truth label, and the L1 norm loss, which
Further research could look at possible conditions where all the
minimizes the total amount of perturbations. The perturbations
top predictions would be of a different scenes that are not closely
are updated by back-propagation to minimize this loss. Finally, the
related to the ground-truth scene. To increase the usability of this
updated perturbations are clipped to make sure they stay within
approach in practice, the robustness of protected images should
the pixel value range of image and rounded such that the perturba-
be improved. Data augmentation of original image and ensemble
tions would remain when saved in a uint8 image format. If it is not
training against different threat models can be considered in the
future.
Pixel Privacy MediaEval’19, 27-29 October 2019, Sophia Antipolis, France
REFERENCES
[1] Jaeyoung Choi, Martha Larson, Xinchao Li, Kevin Li, Gerald Friedland,
and Alan Hanjalic. 2017. The Geo-Privacy Bonus of Popular Photo En-
hancements. In ACM International Conference on Multimedia Retrieval
(ICMR). ACM, 84–92.
[2] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explain-
ing and Harnessing Adversarial Examples. In International Conference
on Learning Representations (ICLR).
[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep
residual learning for image recognition. In Proceedings of the IEEE
conference on computer vision and pattern recognition (CVPR). 770–
778.
[4] Martha Larson, Mohammad Soleymani, Pavel Serdyukov, Stevan Rud-
inac, Christian Wartena, Vanessa Murdock, Gerald Friedland, Roeland
Ordelman, and Gareth JF Jones. 2011. Automatic tagging and geo-
tagging in video collections and communities. In ACM International
Conference on Multimedia Retrieval (ICMR). ACM.
[5] Zhuoran Liu, Zhengyu Zhao, and Martha Larson. 2019. Pixel Privacy
2019: Protecting Sensitive Scene Information in Images. In Working
Notes Proceedings of the MediaEval 2019 Workshop.
[6] Zhuoran Liu, Zhengyu Zhao, and Martha Larson. 2019. Who’s
Afraid of Adversarial Queries? The Impact of Image Modifications on
Content-based Image Retrieval. In ACM International Conference on
Multimedia Retrieval (ICMR). ACM.
[7] Bo Luo, Yannan Liu, Lingxiao Wei, and Qiang Xu. 2018. Towards
imperceptible and robust adversarial example attacks against neural
networks. In AAAI Conference on Artificial Intelligence (AAAI).
[8] M Ronnier Luo, Guihua Cui, and Bryan Rigg. 2001. The development
of the CIE 2000 colour-difference formula: CIEDE2000. Color Research
& Application 26, 5 (2001), 340–350.
[9] Sam Sweere. 2019. Increasing the Perceptual Image Quality of Ad-
versarial Queries for Content-based Image Retrieval. Bachelor The-
sis. Radboud University Nijmegen, the Netherlands. Available at:
https://github.com/SamSweere/CV-PIRE.
[10] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.
2004. Image Quality Assessment: From Error Visibility to Structural
Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.
[11] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio
Torralba. 2017. Places: A 10 million image database for scene recogni-
tion. IEEE transactions on pattern analysis and machine intelligence 40,
6 (2017), 1452–1464.