Deep learning models are vulnerable to adversarial attacks, which could generate adversarial perturbation to make deep learning classifiers fail in classification tasks. In this paper, we reviewed a variety of defensive methods against adversarial attacks and proposed our solution. We present a modification of Generative Adversarial Network (GAN) architecture to train a classifier for the purpose of defending against adversarial attacks. We trained multiple deep learning classifiers with different generator formulations to compare and evaluate their robustness against adversarial attacks. We show how the GAN architecture could contribute to the adversarial machine learning problem and how the capacity of the generator affects the training performance. Our results show that GAN architectures can improve the adversarial robustness of a deep learning classifier with more efficiency training time. The CIFAR 10 classifier accuracy remains around 45% under 8/255 L infinity norm adversarial distortion.
In recent years, there has been an increasing
attention to the security of deep learning networks
regarding the area of adversarial machine
learning. Deep learning models are prone to the
threats caused by well-known attack algorithms
that could generate malicious data samples (e.g.,
fast gradient sign method [
There are several proposals for mitigating the
problems of adversarial samples. Adversarial
training is one of the well-established defense
methods that augment the training data with the
attack algorithms to make the classifiers
generalize on adversarial samples [
The limitations of adversarial training include
the challenge of generalization, the clean and
robustness trade-offs, and the high training
complexity. Usually, a classifier needs to be
trained against a multi-iteration adversarial attack
to achieve a good robustness against strong
adversarial attacks [
Wang et al. [
The rest of the paper is organized as follows: Section 2 discusses the related work regarding adversarial attacks and defenses and introduces the defensive methods related to GANs. Section 3 introduces the methodologies and architectures of our frameworks. Section 4 presents our experimental setups and results and provides discussion regarding the results. Finally, Section 5 concludes the paper and provides a discussion about the future work.
In this section, we discuss recent works related to adversarial attacks, adversarial training and GANs’ contributions to the adversarial machine learning community. The paper focuses on reviewing the gradient-based adversarial attacks since they have a closer relationship with GANs training process.
A simpler formulation of gradient-based attack
algorithm is Fast Gradient Sign methods (FGSM)
[
Projected Gradient descent (PGD) attack [
+1 = ∏ + ( ’ + Ɛ ( ( , ))), (2) where the gradient sign sign(∇x L(x, y)) is calculated multiple times and iteratively adds on the original input X. X’ in this formula represents the perturbed input from the last step of the calculation, and Xt+1 is the output of the current iterative. We think these gradient-based algorithms had a similar property to GANs formulation, that the generator of GANs could also be used to capture the loss gradient of a classifier.
Adversarial training refers to the training
schema that incorporates an adversarial attack
algorithm to augment the training data [
min ∑ max ( ( + ), ), (3)
where the classifier is optimized to minimizes the
cross-entropy loss L on adversarial sample x+δ,
and the adversarial sample x+δ maximizes the
loss of the classifier. Later, Madry et al. [
GANs are popular deep learning techniques
for data synthesis. Recently, some high-quality
synthetic images could be generated by using
some state-of-the-art GANs [
On the other side, multiple GANs frameworks
were proposed to defend the adversarial attack.
Shen et al. [
∑ ( ( ), ) + ∑ ( ( + ( )), ), (4) The key difference between this framework and adversarial training is that it replaced the attack algorithm as a generative network to synthesize the adversarial samples. The generator outputs a vector of perturbation G(X), and the perturbation G(X) adds to the original sample X with a scaler ϵ to construct the final adversarial samples. The scaler ϵ is a variable to constrain the L infinity norm of the perturbation vector from the generator. The optimization goals were to minimize the classification loss L(D(Xi),yi) of discriminator D regarding both original sample X and perturbed sample Xi+ϵG(Xi), and generator G is optimized to maximize the loss of D. The results showed that this framework could improve the robustness of the classifier (i.e., discriminator) and the classifier could generalize on the generated samples easier than traditional adversarial attack samples. Our work makes inspirations from this work. We propose multiple modifications to this framework and aim to improve the stability and overall performance of the training.
In this paper, we applied GAN architecture to adversarial training formulation. Different from the previous works, we implemented four frameworks with four different generators’ formulations. The four formulations estimate the adversarial noises as four function (5) (6) (7) (8) (9) transformations from different inputs.
The training performance of the GAN frameworks involved different generator formulations was compared and analyzed.
The general formulation of our network is a
modification to equation (4) from Wang et al. [
), ) + ∑ ( ( + with a slight difference in considering a set of different inputs for the generator. The different inputs are represented by I in the formula and it affects the formulation of the generator for adversarial noise generation.
The following describes the four different formulations we considered for the generator: (∇), ), The four types of generator formulation are represented as:
G1: formulates adversarial noise N as function transformation of only target image x,
G2: formulates adversarial noise N as function transformation of the target image x and a latent noise z,
G3: formulates adversarial noise N as function transformation of sign of the loss gradient sign(∇) of the current classifier state regarding the target input x,
and G4: formulates adversarial noise N as function transformation of the target image x and the sign of the loss gradient sign(∇) of the current classifier state regarding the target input x.
The loss gradient of the classifier is calculated each time before the input is fed into the generator for the formulations (8) and (9). For simplicity, we will refer to the formula (8) and (9) as G(∇) and
The optimization goal is to minimize the discriminator (classifier) loss on both clean and synthesized data from the generator and maximize the synthesis image's loss discriminator’s classification regarding output.
the The generator in our framework is responsible for constructing perturbation vectors instead of images. The perturbation vectors are added to the original images to produce adversarial samples that feed into the classifier for training. The perturbation from generator output is soft-clipped by "tanh" activation and constrained by a scaler value ϵ that is defined manually. This scaler value is the control variable that defines the maximum L infinity norm of the perturbation. The details of the value settings
will be discussed in the experiment section. We also refer to this scaler as the strength of the perturbation or attack strength in the later section.
The
GAN
architecture also involves dual
generators during the training. Both generators
provide perturbation solutions for the classifier to
train. Im et al. [
Inputs Dataset Image
Generator
+ Generator +
Classifier
Output minimize loss Output maximize loss Figure 1 Framework architecture. The dotted lines represent gradient propagation.
Additionally, we used a standard categorical cross-entropy loss as the loss function for the discriminator.
As for the generator, the optimization should maximize the cross-entropy loss of the discriminator. Hence, the loss function is defined as a negative cross-entropy loss in contrast to the discriminator loss.
We considered the Canadian Institute For Advanced Research (CIFAR) 10 dataset for the experiments. We chose the CIFAR 10 dataset as it is a more challenging problem to optimize a robust classifier for colorful images and scalability of CIFAR 10 is within our scope of the project.
In our experiment, we used a generator similar
to the cycle GAN generator [
C o n Input v 7 x 7 C o n v 3 x 3 C o n v 3 x 3
Residual
Blocks Residual Block C C o o xv3n xv3n + 3 3
C o n v 3 x 3
C o n v 3 x 3
C o n v 7 x 7 Ɛ X
Output 0.9 Beta2 for discriminators. The learning rate for the generators is higher because, during the training dynamic, we expect the generator to capture the loss gradient of the discriminator's current state. Hence, we reduced the learning process of the discriminator to make the learning process easier for generators.
We used a scaler (ϵ) of 16/255 as perturbation strength to constrain the generator perturbation vectors.
All the images’ pixels were normalized within
the range of [
A baseline models was also trained without GANs framework. We used a conventional training schema to train the baseline models. The structure of baseline model is consistent with the shallow VGG model. 4.1.
This section lists and compares the training
results from different formulations. In this
section, we used “×1” generator filter number for
experiments, shown in Table 1. The same filter
parameters were used for all formulations from
section 3. The baseline model was also tested for
comparison. To evaluate the robustness of the
models, we implemented FGSM and PGD as two
standard testing attack algorithms. It is suggested
that only the robustness evaluated under the
strong multi-iteration attack is valid [
From the plot, all the models trained with GANs framework significantly improved in terms of robustness against different levels of adversarial noise. The discriminators trained against the generators that included target image x as an input option (G(x) and G(x, z)) generally gained similar robustness against FGSM and PGD. The accuracy against FGSM was slightly higher compared to against PGD under the highest noise norm setting (16/255). This result means that the discriminators (classifiers) trained against G(x) and G(x, z) formulations had improved adversarial robustness but also had a more obvious loss gradient compared to other models. One step of gradient descent is sufficient to discover adversarial samples similar to those computed from multi-step gradient descents. We also observe that the formulations, including gradient inputs, generally gain better robustness than other formulations. Formulations G(∇) and G(∇,x) generally had a similar performance under all attack settings. However, the performance under PGD attacks downgrades when noise norm increases compared to FGSM attacks, but the accuracies stay higher than other formulations. Furthermore, the clean data accuracy of the baseline model is higher than all the formulations. The generator that considered both gradient and image inputs had the lowest clean accuracy.
Different Generator Input Comparison G (x) FGSM G (x) PGD G (x,z) FGSM G (x,z) PGD G (▽) FGSM G (▽) PGD G (▽,x) FGSM G (▽,x) PGD Baseline FGSM Baseline PGD 90.00% 80.00% s70.00% k c ta60.00% t A re50.00% d nU40.00% y ca30.00% rcc A20.00% 10.00% 0.00% We observed overfitting effects with all GANs frameworks’ discriminators, with the training set data accuracy could reach over 90% after GANs training, but testing set accuracy is only around 70%. With the same discriminator architecture, the conventional training accuracy is around 81% for both training and test set data. We tried adding L2 regularization for each layer or using an Adam optimizer with weight decay (AdamW). However, the regularization helped little to the results, and the performance downgraded when the weight decay was above 0.0001. Currently, we have no idea why the overfit is happening. We suspect the synthetic data from the generator has a different data distribution that shifts the decision boundaries of the discriminator. 4.2.
In this section, we evaluate the discriminators’ accuracy performance with different training epochs. The architecture selected for this evaluation is the G(∇) formulation generator with “×2” filter numbers and the same discriminator, shown in Table 1. The other setting for the experiment is consistent with the previous section. Figure 4 shows the results of the experiment.
Train Epochs Comparison 0/255 4/255 8/255 16/255
Attack L Infinity Norm Constraint training iterations, and the discriminator learned to classify the images with this perturbation direction; however, the gradient was saturated for the generator to learn a new adversarial direction. 4.3.
In previous sections, we used PGD with 100 iterations to evaluate our models. However, the results obtained by 100 iterations may not describe the overall robustness. Therefore, in this section, the accuracy results from 1000 iterations are collected and shown in Table 2. All parameters and settings are the same to the last section 4.2. The model trained with 100 training epochs was selected for this experiment. From the table, we did not observe a significant accuracy difference when iteration increases. With all norm constraints, the accuracy of the model stays relatively identical. This result shows that the robustness evaluation for this model is sufficient by using the PGD 100 iterations. Therefore, we suggest that the evaluations from previous sections are valid and sufficient to conclude the overall robustness of the model.
Table 2 The model accuracies under PGD 100 and 1000 L infinity 4/255 8/255 16/255 norm size Accuracy 57.41% 45.6% 23.71% PGD 100 Accuracy 57.33% 45.28% 23.31% PGD 1000
of
In this section, we visualize the perturbation generated by the generator from our GANs frameworks. We chose the generator with G(∇) formulation and “×2” filter numbers (Table 1). The perturbations were generated with the framework trained after 300 epochs. We chose the maximum epochs from our experiments since we want to investigate the maximum effect of training with the framework. The FGSM and PGD noises were generated based on the classifier’s (discriminator’s) state after the 300 epochs of training. The visualization is shown in Figure 5.
The target image (a) is the top left image with a correct label of “ship”. The noise images under the target image are the perturbation vectors generated by FGSM (b), PGD (c), and our generator (d), respectively. The right-hand side plots (e, f, g, h) show the classifier’s logits outputs trending regarding the different norm size by adding the respective noise vector onto the target image. The top-right plot (e) shows the logits output trending when the classifier is perturbed by a random Gaussian noise vector with zero mean and one standard deviation. We do not show the Gaussian noise since the plot indicates the model’s behavior in standard non-adversarial condition and there is no significant information within the Gaussian noise vector. Furthermore, the negative norm size in the x axis represents the opposite perturbation direction by the noise vector, which means subtracting the noise vector from the target image.
When the target image is perturbed by random noise (e), the logit distribution shows a relative symmetry shape whether the noise norm increases in either direction. However, with adversarial noise perturbation (f, g, h), the logit output regarding the correct class label decreases more dramatically with increasing norm size in a positive direction. We labeled the cross points when another class label output value started surpassing the correct class label’s output. The surpassed point for FGSM is labeled with the blue line, and the surpassed point for PGD is labeled with the yellow line. The orange line indicates the position where the noise norm is zero, which means no perturbation is added to the image. As expected, the surpassed point of PGD is closer to the zero-norm point compared to the FGSM’s surpassed point since PGD could find an adversarial sample with smaller perturbation with its iterative gradient descent. The surpass point for the noise of the generator stays closer to the zeronorm point compared to the FGSM noise and stay further compared to the PGD noise. These results suggest that the adversarial noise from the generator is stronger than the noise from a singlestep adversarial attack but weaker than the noise from an iterative adversarial attack. We also observed that the noise from the generator is more uniform compared to the noisier noise from the gradient descent attacks. This more uniform noise may relate to the convolution generator’s properties and capacities. 4.5.
With the experiments illustrated in this paper,
we showed that the framework of GANs could be
used to improve the adversarial robustness of the
deep learning classifier. The discriminator could
be trained with increasing robustness against the
gradient-based adversarial attack algorithms.
However, the trade-off still presents between
clean data accuracy and robustness of the model,
especially with a high dimensional dataset. In the
current state, we need more experiments to
analyze the training dynamics of GANs and make
further improvements. We also found that the
structure of the generator could affect its
performance in perturbation generation. This
finding is different from the previous work [
This work intends to verify the possibility of GANs contributing to adversarial attack defenses. We suggest that the properties of GANs could be utilized further for defending against adversarial attacks. We might be able to improve the formulation of the framework to make it more applicable to real-life implementation scenarios. There are a lot of hidden properties to be discovered from GANs and the phenomenon of adversarial samples. We believe the synthetic adversarial sample could explain more about the properties of deep learning models. In the next step, we hope to extend this framework to other types of deep learning models, such as attentionbased models, and evaluating how they perform under the GANs training. Beside image models, GANs can also train deep learning models used in other application domains, including text recognition models, malware detection models, and reinforcement learning models. We are also interested in exploring the different application domains with GAN robustness training architecture to address the problem of adversarial samples.