INTRODUCTION

The internet is flooded with images. This is especially true with the growth of social networks over the last decade. All this data is used to perform analysis to bring out new trends or to train predictive models. When it comes to images, deep neural networks vastly lead the landscape of machine learning. These deep neural networks are especially known to thrive on big datasets. This leads to the idea that more data leads to better models. While there certainly is truth to that affirmation, better learning mostly comes out of better data. Good data is data that both fits the task (e.g. people, places, objects detection) and whose quality is good. Due to the amount of available data, a human could not perform this cherry-picking of good data. Automated classifiers like BIQA [4] have been trained to assess the quality of an image. This classifier was trained on images whose quality was labeled based on the perceived quality of the media (e.g. resolution, compression artifacts). To protect one's data, images can be manipulated and slightly modified to defeat the automatic quality assessment [6]. We chose an adversarial attack approach to achieve this goal.

APPROACH 2.1 Adversarial Examples

Adversarial examples were first introduced by Szegedy et al. [8] in early 2014. They are usually studied in the case of image classification: An attack effectively crafts a perturbation of an image to a small extent but enough to fool even the best classifiers.

In this setup, an original image 𝑥 0 is given as an input to the trained neural network to estimate the probabilities ( p𝑘 (𝑥 0 )) 𝑘 of being from class 𝑘 ∈ {1, . . . , 𝐾 }. The predicted class is given by: ĉ (𝑥 0 ) = arg max 𝑘 p𝑘 (𝑥 0 ).

(

)1

The classification is correct if ĉ (𝑥 0 ) = 𝑐 (𝑥 0 ) the ground truth class for 𝑥 0 . The goal of an attack is to craft an imperceptible perturbation 𝑝 such that the adversarial sample 𝑥 𝑎 = 𝑥 0 + 𝑝 verifies ideally:

Copyright 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). MediaEval'20, December 14-15 2020, Online

𝑥 ★ 𝑎 = arg min 𝑥: ĉ (𝑥)≠𝑐 (𝑥 0 ) ∥𝑥 − 𝑥 𝑜 ∥,(2)

Where ∥ • ∥ is a measure of distortion, in most cases the Euclidean distance. A small distortion makes it less likely for human to perceive that the image was manipulated.

BIQA is a deep neural network and as such is vulnerable to adversarial attacks. However BIQA is not a classifier returning a class prediction but a regressor giving a quality score 𝐵𝐼𝑄𝐴(𝑥) ∈ [0, 100]. The notion of adversarial sample thus needs to be redefined. In our case, we set a target score 𝑠 𝑎 ∈ [0, 100]. Regardless of the original score 𝐵𝐼𝑄𝐴(𝑥 𝑜 ), our adversarial sample now ideally verifies:

𝑥 ★ 𝑎 = arg min 𝑥:𝐵𝐼𝑄𝐴(𝑥) <𝑠 𝑎 ∥𝑥 − 𝑥 𝑜 ∥,(3)

Quantization

An original image 𝑥 0 in the spatial domain (e.g. PNG format) is a 3-dimensional discrete tensor: 𝑥 0 ∈ {0, 1, . . . , 255} 𝑛 (with 𝑛 = 3 × 𝑅 × 𝐶, 3 color channels, 𝑅 rows and 𝐶 columns of pixels). The main objective of this task is to craft images: 𝑥 𝑎 ∈ {0, 1, . . . , 255} 𝑛 . This additional constraint to the attack is yet not easy to enforce. In a deep neural network, this input image is first preprocessed onto a range domain that usually reduces variance of the data. Its purpose is to ease the learning phase and thus to increase the performance of a deep neural network. This preprocessing is defined by design before the training stage and cannot be modified at testing. In the case of BIQA, the range domain is [−0.5, 0.5] 𝑛 .

Most attacks of the literature are performed in this domain without consideration of the transformation it represents. This leads to an adversarial sample 𝑥 𝑎 ∈ [0, 255] 𝑛 after reverting the preprocessing. To save this adversarial sample 𝑥 𝑎 as an image, the first step is then to round it which will erase most of the perturbation in the case of a low-distortion attack. Rounding is therefore likely to remove the adversarial property of the sample.

Paper [1] addresses this problem presenting a post-processing added on top of any attack to efficiently quantize a perturbation: It keeps the adversarial property while lowering the added distortion. The method is based on a classification loss to ensure adversariality defined as follows:

𝐿(𝑥) = log( p𝑐 (𝑥 0 ) (𝑥)) − log( p ĉ (𝑥) (𝑥)). (4)

To adapt this method to the context of BIQA, we only need to redefine it to:

𝐿(𝑥) = 𝐵𝐼𝑄𝐴(𝑥) − 𝑠 𝑎 . (5)

For a given x, 𝐿(𝑥) < 0 ensures x scores under the target 𝑠 𝑎 .

In this task, we know the classifier (BIQA) and its parameters. We are therefore in a white-box setup. Most modern attacks are developed in this scenario, from the most basic FGSM [3] and IFGSM [5] to the most advanced PGD [7], C&W [2], BP [10]. FGSM is a noniterative attack bringing a fast solution of the problem. Our work used this attack in the early stages as a proof of concept bringing a quick further understanding of the problem. Artifacts were visible. Instead all the results reported here are crafted using more the advanced PGD attack [7] in its 𝐿 2 optimization version. One input parameter is the distortion budget. We run the attack over 7 iterations with different distortion budgets (whose maximum value is set to 2000). A binary search quickly finds an adversarial sample with the lowest distortion.

JPEG compression

The final images will be evaluated on their JPEG [9] counterpart. This compression is done with a quality factor of 90. However there are many image compression sofwares providing different results. We used the command line $ convert to simulate this compression. Tables 1 and 2 show for different methods both 𝑃 𝑃 𝑁 𝐺 and 𝑃 𝐽 𝑃𝐸𝐺 respectively the percentage of images successfully beating the target score in the PNG domain and the JPEG domain. Additionally Table 2 shows results of the jury as well.

Quantization

Spatial domain.

The work [1] serves as a baseline for quantization. We only slightly adapt it as stated in Sect. 2.2. Table 1 reports our results for two target scores: 𝑠 𝑎 = 30 and 𝑠 𝑎 = 50. It appears that the perturbation crafted in the pixel domain is fragile when facing a JPEG compression.

DCT domain.

The final image being evaluated after a JPEG compression, we explore a method adapting the quantization [1] to the DCT domain. Using the same notations [1]: Let 𝑋 𝑜 denote the image in the DCT domain, 𝑋 𝑎 = 𝑋 𝑜 + 𝑃 is the result of an initial attack like PGD, and 𝑋 𝑞 = 𝑋 𝑜 + 𝑃 + 𝑄 the final quantized transformed coeffcients. We solve a Lagrangian formulation:

𝑋 𝑞 = 𝑋 𝑜 + 𝑃 + arg min 𝑄 𝐷 (𝑄) + 𝜆𝐿(𝑄),(6)

where 𝜆 is the Lagrangian multiplier controlling the tradeoff between the distortion 𝐷 (𝑄) and 𝐿(𝑄) defined in (5). The distortion 𝐷 (𝑄) is defined as the squared 𝐿 2 norm of added perturbation:

𝐷 (𝑄) = ∥Δ × (𝑃 + 𝑄)∥ 2 .

The quantization noise 𝑄 is s.t. 𝑋 𝑜 + 𝑃 + 𝑄 ∈ ΔZ 𝑛 , where Δ ∈ N 𝑛 is the quantization step matrix for JPEG QF=90. If we use a first order approximation of 𝐿(𝑄), we can develop (6) in a second-degree polynomial function. For any coefficient 𝑗, this function is locally minimized by:

𝑄 ★ ( 𝑗) = −𝑃 ( 𝑗) − 𝜆 𝐺 ( 𝑗) 2Δ( 𝑗) ,(7)

where 𝐺 = ∇𝐿(𝑄)| 𝑄=0 the gradient computed at 𝑄 = 0. This minimum however does not enforce (𝑃 + 𝑄 ★ ) ∈ Z 𝑛 . A simple rounding of (𝑃 + 𝑄) will then finalize the quantization. Finally we need to control a maximum allowed distortion. If 𝜆 gets big, 𝑄 ( 𝑗) become a very high value which is not desirable. The final value

RESULTS AND ANALYSIS

Tables 1 and 2 show the importance of considering the JPEG compression. When the image is quantized by the 𝐿 2 optimization in the spatial domain, most images will successfully be adversarial images. However, very few of them remain adversarial after the JPEG compression. The BIQA score on most images increases up to 10 points. If the quantization is done in the DCT domain, most of them remain adversarial and the task is successful. It is however obviously more difficult to beat a lower target score 𝑠 𝑎 . An interesting property of the DCT quantization is that it creates typical JPEG artifacts as seen on Figure 1. This is especially true in low frequency images since it is harder to remain undetectable in a such situation.

DISCUSSION AND OUTLOOK

The MediaEval task was a good opportunity to extend our previous work [1] to 1) a regressor BIQA, and 2) in the DCT domain. Saving the DCT coefficients directly into a JPEG image is more consistent as it offers a better control on adversariality. Another difficulty of this task was the lack of knowledge about the compression algorithm. We therefore worked in a 'gray' box setup. The results showed that JPEG compression have a big effect on the BIQA score of, at least, adversarial images (and probably any other quality estimator). Hopefully our JPEG compression is close to the one used in the contest which allowed transferability of our adversarial images.