This paper presents our work on the 2020 MediaEval task: “Pixel Privacy: Quality Camouflage for Social Images". Blind Image Quality Assessment (BIQA) is an algorithm predicting a quality score for any given image. Our task is to modify an image to decrease its BIQA score while maintaining a good perceived quality. Since BIQA is a deep neural network, we worked on an adversarial attack approach of the problem.
The internet is flooded with images. This is especially true with the
growth of social networks over the last decade. All this data is used
to perform analysis to bring out new trends or to train predictive
models. When it comes to images, deep neural networks vastly lead
the landscape of machine learning. These deep neural networks
are especially known to thrive on big datasets. This leads to the
idea that more data leads to better models. While there certainly is
truth to that afirmation, better learning mostly comes out of better
data. Good data is data that both fits the task ( e.g. people, places,
objects detection) and whose quality is good. Due to the amount of
available data, a human could not perform this cherry-picking of
good data. Automated classifiers like BIQA [
Adversarial examples were first introduced by Szegedy et al. [
In this setup, an original image 0 is given as an input to the trained neural network to estimate the probabilities (ˆ (0)) of being from class ∈ {1, . . . , }. The predicted class is given by: ˆ(0) = arg max ˆ (0).
(1) The classification is correct if ˆ(0) = (0) the ground truth class for 0. The goal of an attack is to craft an imperceptible perturbation such that the adversarial sample = 0 + verifies ideally: ★ = arg
min :ˆ()≠ (0) ∥ − ∥,
Where ∥ · ∥ is a measure of distortion, in most cases the Euclidean distance. A small distortion makes it less likely for human to perceive that the image was manipulated.
BIQA is a deep neural network and as such is vulnerable to adversarial attacks. However BIQA is not a classifier returning a class prediction but a regressor giving a quality score ( ) ∈ [0, 100]. The notion of adversarial sample thus needs to be redefined. In our case, we set a target score ∈ [0, 100]. Regardless of the original score ( ), our adversarial sample now ideally verifies: ★ = arg :mi(n)< ∥ − ∥, 2.2 An original image 0 in the spatial domain (e.g. PNG format) is a 3-dimensional discrete tensor: 0 ∈ {0, 1, . . . , 255} (with = 3 × × , 3 color channels, rows and columns of pixels). The main objective of this task is to craft images: ∈ {0, 1, . . . , 255} . This additional constraint to the attack is yet not easy to enforce.
In a deep neural network, this input image is first preprocessed onto a range domain that usually reduces variance of the data. Its purpose is to ease the learning phase and thus to increase the performance of a deep neural network. This preprocessing is defined by design before the training stage and cannot be modified at testing. In the case of BIQA, the range domain is [−0.5, 0.5] .
Most attacks of the literature are performed in this domain without consideration of the transformation it represents. This leads to an adversarial sample ∈ [0, 255] after reverting the preprocessing. To save this adversarial sample as an image, the first step is then to round it which will erase most of the perturbation in the case of a low-distortion attack. Rounding is therefore likely to remove the adversarial property of the sample.
Paper [
( ) = log(ˆ (0) ( )) − log(ˆˆ() ( )).
To adapt this method to the context of BIQA, we only need to redefine it to:
( ) = ( ) − .
For a given x, ( ) < 0 ensures x scores under the target . (2) (3) (4) (5) 3
In this task, we know the classifier (BIQA) and its parameters. We
are therefore in a white-box setup. Most modern attacks are
developed in this scenario, from the most basic FGSM [
The final images will be evaluated on their JPEG [
Tables 1 and 2 show for diferent methods both and respectively the percentage of images successfully beating the target score in the PNG domain and the JPEG domain. Additionally Table 2 shows results of the jury as well. 3.2
Quantization
3.2.1 Spatial domain. The work [
3.2.2 DCT domain. The final image being evaluated after a JPEG
compression, we explore a method adapting the quantization [
The quantization noise is s.t. + + ∈ ΔZ , where Δ ∈ N is the quantization step matrix for JPEG QF=90. If we use a first order approximation of (), we can develop (6) in a second-degree polynomial function. For any coeficient , this function is locally minimized by: ★( ) = − ( ) − ( ) , (7) 2Δ( ) where = ∇ () |=0 the gradient computed at = 0. This minimum however does not enforce ( + ★) ∈ Z . A simple rounding of ( + ) will then finalize the quantization. Finally we need to control a maximum allowed distortion. If gets big, ( ) become a very high value which is not desirable. The final value for the quantized perturbation in the DCT domain is thus bounded by [− Δ , Δ1 ]. These images were submitted to the jury.
1 4
Tables 1 and 2 show the importance of considering the JPEG compression. When the image is quantized by the 2 optimization in the spatial domain, most images will successfully be adversarial images. However, very few of them remain adversarial after the JPEG compression. The BIQA score on most images increases up to 10 points. If the quantization is done in the DCT domain, most of them remain adversarial and the task is successful. It is however obviously more dificult to beat a lower target score . An interesting property of the DCT quantization is that it creates typical JPEG artifacts as seen on Figure 1. This is especially true in low frequency images since it is harder to remain undetectable in a such situation. 5
The MediaEval task was a good opportunity to extend our previous
work [