Evaluating defensive solutions against adversarial evasion attacks means quantifying the defense's capability to detect or tolerate attacks. Ideally, a defense should be tested against all the possible attacks: however, this is not achievable, and it is necessary to identify a representative attack set for the evaluation. Unfortunately, how to select such an attack set is an open question. Arguably, the selected attacks should apply diverse effects on the original image, in terms of dimension and distribution of the perturbation. We propose to quantify the perturbation through Image Quality metrics in addition to L-norms, such that adversarial attacks can be grouped (and only one representative of the group can be selected to test the defense) if they i) similarly perturb the attacked image, and ii) have similar success rate and detectability rate. Disappointingly, the analysis reveals that attacks with similar image perturbation cannot be related. Substantial evidence discourages grouping attacks and suggests that any reduction of the attack set impacts the validity of the defense evaluation. Evasion attacks, adversarial attacks, attack categories, image quality metrics, distance metrics. 1 0000-0002-0297-2108 (T. Puccetti); 0000-0001-9820-6047 (T. Zoppi); 0000-0002-2291-2428 (A. Ceccarelli)
Deep Neural Networks (DNNs) that perform image classification are vulnerable to
adversarial samples, which are deliberately crafted by perturbing legitimate input (e.g., images,
texts, tabular data) to mislead the target model [
Ideally, the attack set used for the evaluation should include all the possible attacks against
the target classifier, and the defense should be evaluated against each of these attacks. This is
unfeasible because of both computational effort and the potential occurrence of unknown
attacks. On top of this, without a rigorous methodology for the evaluation, many of the attacks
and defenses in the literature have been evaluated with custom approaches that are difficult to
compare [
The overarching challenge in evaluating defenses lies in the choice of the attack set to be
considered: the attack set should deliver a realistic representation of the attack space that allows
defining precise boundaries of the defense and, consequently, details the defense’s coverage
against different attacks. The state of the art agrees that i) there is no individual attack that
sufficiently covers the entire attack space, and ii) there is no evidence that a defense able to
tolerate certain attacks (i.e., a defined portion of the attack space) will also be able to tolerate
others [
To summarize, different attacks may have similar effects on the target classifier, and similar values of L-norms, but the defense may decide differently. The same applies to attacks that produce different image perturbations. Additionally, grouping attacks based on their mathematical formulation is not effective: small differences in the formulation may have a relevant impact on the perturbation applied to images and on the success rate.
Position statement and approach. We believe that the number of attacks to be used when testing a defense can be reduced only by finding similarities in the perturbation introduced in the adversarial images, together with evidence that such similarities will have similar effects on the classifier and the defense. This paper investigates whether quantifying the perturbation through distance metrics in addition to L-norms allows partitioning adversarial attacks into classes, which could then be used to craft r epresentative attack sets. In other words, we investigate whether measuring perturbations using distance metrics allows for the identification of characteristics of the attacks that permit to discriminate between attack families.
We identify alternative distance metrics to extrapolate different information from the
adversarial image. In particular, the Image Quality Assessment (IQA) domain particularly fits
our necessities: IQA aims at quantitatively evaluating the quality of images modified by a
variety of distortions (e.g., processing, compression, etc.) to exploit the Human Visual System
(HVS) model for low-level perception [
We identify groups through an experimental evaluation. First, we generate 222 attack sets
from 12 different attacks targeting two state-of-the-art models, namely ConvNet12 [33] and
ResNet50 [
The results indicate that employing distance metrics to quantify perturbations exhibits some discriminatory capabilities, as it effectively clusters a small number of the selected attacks with highly similar mathematical formulations. However, these results, while insightful, do not demonstrate a high level of generalizability. Some attacks with shared mathematical formulations are correctly grouped, while others with similar formulations are not consistently recognized as a distinct group. Our experiments reveal that numerous attacks remain ungrouped, lacking observable signatures or defining characteristics in their perturbations. Consequently, we argue that reducing the number of attacks for testing a defense compromises the validity of the results.
Adversarial attacks consist of deliberately manipulating the input to a DNN to cause wrong predictions [27]. In this paper, we focus on evasion attacks against image classifiers, because i) they have large applications in the real world [30], and ii) image classification is the typical application domain for such attacks [28, 29]. Evasion attacks can be broadly classified based on the knowledge of the attacker about the target classifier: white box attacks imply some knowledge of the classifier architecture, while black box attacks approximate such information. Evasion attacks can be grouped into finer categories, based on the specific implementation details needed to run the attack: i) gradient-based, ii) score-based, and iii) decision-based. Gradient-based attack methods exploit the gradient of the loss of the target classifier during the input processing. The score-based attacks solely on the output score of the target classifier. Decision-based attacks are the most indicated for a black box setting as they need only prediction labels [31]. We select white box and black box attacks from the three categories, to create a heterogeneous set of 12 attacks.
Gradient-Based Attacks. One of the most effective gradient-based attacks is Carlini and Wagner (CW2, [37]). It is formulated as the following optimization problem: ′ = {|| − 0||2 + ( )} (1)
The first term enforces perturbation on the original image, the second term is the loss function of the model, and g(x) is g(x)= { ( ) 0 − ≠ 0 ( ) , 0} where 0 is the ( ) is the score returned predicting x with label i. ground truth label of the input 0, is the score computed on the input x for the 0 label, and
Instead, Elastic Net (ELA, [38, 39]) formulates the generation of adversarial attacks as an elastic-net regularized optimization problem: ′ = | − 0| 2 2 + | − 0| 1 + ( ) (2) where g(x) is the same as the CW2 attack.
Projected Gradient Descent (PGD, [41]) solves the optimization problem using the projected gradient descent. Practically, it finds adversarial images by adding or subtracting a small error to each dimension of the input based on the gradient sign. Fast Gradient Sign Method (FGM, [40]) can be viewed as a one -step PGD attack, while the Basic Iterative Method (BIM, [42]) is the iterative form of the attack. Deep Fool (DEEP, [43]), performs an iterative linearization of the classifier to calculate the closest decision boundary for a genuine image x. Jacobian-based Saliency Map (JSMA, [44]), generates an adversarial saliency map, which helps identify the input features to be included in the perturbation. Newton Fool Attack (NEW, [45]) operates by executing a gradient descent, which reduces the likelihood of the initial class.
Score-Based Attacks. We select the Zeroth Order Optimization-Based Attack (ZOO, [46]). The attack is a black box, and it uses a finite difference method to estimate the gradient sign of the loss function w.r.t the input image. The gradient estimation is used to run the CW2 attack and craft the final adversarial image.
Decision-Based Attack. We select from this category the Boundary Attack (BOUND, [47])
based on the random walk on the decision boundary of the input image. The method starts with
a sample categorized in the target class and searches for a minimum amount of perturbation to
keep the image adversarial. Also, we select the HopSkipJump Attack (HOP, [
Modifying the template — including but not limited to adjusting margins, typeface sizes, line spacing, paragraph and list definitions — is not allowed. At the state of the art, the amount of perturbation introduced by an attacker is quantified using the pixel Lp norms, for any p>0: |v|p =
∑ =1 | |
1
(3)
where v = ||x–x'|| is the perturbation introduced in an image x to obtain its adversarial
counterpart x', and vi=||xi–xi'|| is the difference pixel by pixel of the two images. Each L-norm
has a different mathematical meaning and, therefore, captures different characteristics of the
perturbation [
We also use distance metrics from the image quality domain. L-norms work very well to quantify the wideness and intensity of the perturbation but may not be sufficient to capture the finer characteristics of the perturbation. We select the following metrics.
Mean Squared Error (MSE) calculates the cumulative squared error between the original image and the distorted image:
MSE = 1 −1 −1 =0 [ ( , )]2
(4)
=0
where x and y provide the pixel position, M and N are the image width and height, and E(x,
y) = Io(x, y) – Ip(x, y) is the difference pixel by pixel of the original image Io and the perturbed
image Ip [
Block Sensitive - Peak Signal-to-Noise Ratio (PSNR-B, [
Universal Quality Image Index (UQI, [15]) calculates the amount of transformation of relevant data from the reference image into the perturbed image. UQI is defined as:
UQI= ( 2 + 42 ′ ̅ ̅′ (5)
′ )[( ̅)2 + ( ̅′)2 ]
where ̅ and ̅ ′ are respectively the mean values of the original and perturbed images, 2 and
2′ are the variances and ′ is the covariance. The range of this metric is -1 to 1, where 1
indicates that the reference and perturbed images are similar [
Erreur Relative Globale Adimensionnelle Synthèse (ERGAS, [16]) measures the global radiometric distortion between two images; it calculates the average error of each band of the perturbed image for the reference one. As reported in [17], high values of ERGAS indicate low quality of the perturbed image, while lower values indicate good quality. The ERGAS is given as:
ERGAS = 100 ℎ 1 ( )2 ∑ =1( ( )2 ) (6) where h and w are the height and width of the image, the RMSE(Bk) is the Root Mean Squared Error for k-band computed between the original and altered image, and mean(k)2 denotes the mean k-band of the original image. Relative Average Spectral Error (RASE, [21]) determines the difference in spectral information between each band of the merged image and the original image. Given M the mean radiance of the N spectral bands Bi of the original image, and the root mean square error RMSE, RASE is computed as: SAM =
〈 , ′〉 || ||2∙|| ′||2 SCC = ∑ ∑( − )
2 x' − x' ∑( − )2 ∑ x' − x' 2 2
Spectral Angle Mapper (SAM, [22]) computes the spectral angle between the pixel vector of the reference image and of the perturbed image. It is performed on a pixel-by-pixel basis. A value of SAM equal to zero denotes the absence of spectral distortion. In the following expression, x is the original image, and x' is the perturbed image:
Spatial Correlation Coefficient (SCC, [19]) represents the correlation between two visual signals of images in a cortical visual space. The SCC is expressed as: (7) (8) (9) ′ is the mean of the modified image. The value of SCC ranges from -1 to 1 [20]. where x' is the modified image, x is the original image, is the mean of the original image, and
Visual Information Fidelity (VIF, [18]) quantifies the Shannon information present in a
processed image. As summarized in [
We select two state-of-the-art attack detectors as adversarial defenses, namely MagNet [23] and Feature Squeezing [24]. MagNet [23] exercises an autoencoder trained on the normal images to reconstruct an input image before it is fed to the classifier. Once the image is reconstructed, the detector computes the reconstruction error between the input x and a reformed input x', and applies a threshold learned during the training to target a specific false positive rate (FPR). If the image is normal, the reconstruction error lies below the target threshold; otherwise, the input is marked as an adversarial attack. Feature Squeezing (Squeezer, [24]) compares the prediction of the classifier on the inputs with the predictions obtained using pre-processed inputs. The Squeezer detector computes a score that is the maximum distance among these predictions and then applies a threshold that is learned in an unsupervised fashion to target a specific FPR on the training set.
We set up an experimental campaign to group adversarial attacks. Our experimental methodology is in Section 3.1, and details on attacks, image classifiers, and detectors are in Section 3.2. We execute the experiments on a Dell Precision 5820 Tower with an Intel I9- 9920X, GPU NVIDIA Quadro RTX6000 with 24GB VRAM, 128GB RAM, Ubuntu 18.04 with kernel 5.4.0, and runtime CUDA 11.0.
We first generate adversarial images targeting two state-of-the-art models. We apply the 12 attacks in Section 2.1 with multiple configurations on the first 100 images of the CIFAR-10 dataset [25]. Out of the existing benchmark datasets for image classification, we choose CIFAR10 [25] because it is composed of RGB images that have a reasonable size to make experiments feasible. We detail the selected image classifiers and the attack configurations in Section 3.2. Then, we compute distance metrics (both L-norms and image quality metrics) between each of the adversarial and original (clean) images. We feed the adversarial images to both detectors MagNet [23] and Feature Squeezing [24], logging their answer: 0 if the attack was not detected, 1 otherwise. These actions allow building a tabular dataset as in Table 1, which contains the following data for each adversarial image: i) the values of the 12 distance metrics, ii) the success outcome (i.e., does the attack trigger a misclassification?), iii) binary flags that indicate if either MagNet [23] or Feature Squeezing [24] can detect that the image was counterfeited, and iv) the attack configuration (not in the paper for brevity). This data will allow conducting the following three analyses detailed below
Distance metric clusters. We analyze the values of distance metrics to investigate if grouping adversarial images according to the perturbation they apply is meaningful. Intuitively, distance metrics quantify different perturbation aspects: we want to investigate if they capture differences between the way attack images are constructed. For this purpose, we run several clustering algorithms and check if images created using (the same or) different attacks are grouped.
Correlation between distance metrics and the success rate of the target classifier. We investigate if distance metrics are informative enough to predict the success of an attack against the target classifier. This allows discovering if distance metrics relate to the success of a given attack.
We selected two trained state-of-the-art classifiers, namely ConvNet12 from [33] and
ResNet50 [
We configure MagNet [23] and Feature Squeezing [24] to defend our models. We train MagNet using 5 000 normal images from the CIFAR-10 training set, with a False Positive Rate (FPR) of ≈ 0.02. We set the threshold for Feature Squeezing manually to have the same FPR on the test set.
We craft attacks using the ART Toolbox [35]. In Table 2 in the appendix A, we report the number of configurations applied for each attack and the configuration values.
We run the clustering algorithms K-Means, DBSCAN, and variants of the
ExpectationMaximization and Self-Organizing Maps that suit clustering analyses and are available in the
WEKA toolkit [
We look for a correlation between the success rate of the attack and the values of the selected metrics, to the extent that we can predict the success rate with minimal error. We measure our prediction capability by computing the cosine distance CD between the actual and the predicted success rate of the adversarial images on the target classifier. More specifically, we train an XGBoost regressor with the distance metrics of the adversarial images and the attack outcome to predict the success of each adversarial image. Given the distance metrics computed from an adversarial image as input, the trained regressor outputs a good estimation of the success probability, i.e., the CD is generally low. We repeat the study by training the regressor multiple times using adversarial images generated from the same attack and testing on adversarial images generated using the other attacks. This shows the mutual predictability of the success rate between attacks, which we can use to understand if two attacks are similar. According to the distance metric cluster analysis in Section 4.1, both models grouped BIM and FGM. However, unlike the cluster analysis, PGD was excluded from this grouping, indicating a weaker similarity compared to the other two attacks. We report in correlation between distance metrics and the behavior of the target classifier and Table 4 in the appendix the detailed discussion and the related data, respectively.
We quantify the correlation between the detectability of an adversarial image and the value of the distance metrics. In this case, we train an XGBoost classifier by using distance metrics as input features, to predict the binary output (detected/undetected) of a detector, either Squeezer or MagNet. Similarly to the previous experiment, the attacks in the columns of Table 5.a and Table 5.b are used for training, and the attacks in the rows are used for testing. Each cell of the table reports the mean Matthew Correlation Coefficient (MCC) obtained by XGBoost when predicting the outputs of MagNet and Squeezer detectors. Notably, BOUND and HOP exhibited strong mutual predictability, with MCC scores indicating their ability to predict each other's detectability. Additionally, CW attacks showed some predictive capability for BOUND and HOP. However, PGD, UNI, and ZOO attacks showed no mutual predictability with any other attacks.
On the ResNet50 model, PGD, UNI, and ZOO attacks displayed poor predictability with other attacks, with MCC scores near zero. Conversely, BIM, BOUND, DEEP, ELA, FGM, HOP, JSMA, and NEW attacks showed some degree of predictability with each other, suggesting relationships between these attacks. Notably, HOP and BOUND exhibited the strongest mutual predictability, consistent with previous findings. Furthermore, while BIM, FGM, and DEEP attacks demonstrated predictive capability for each other on ResNet50, this relationship was not observed in the ConvNet12 results.
We aggregate together the results of each of the three analyses for both models, respectively, in Figure 1.a and 1.b. The x-axis refers to the clustering analysis in Section 4.1 for clusters k=2, the y-axis to the CD scores in Section 4.2, and the z-axis to the MCC scores in Section 4.3.
From the ConvNet12 analysis in Figure 1.a, two groups can be identified easily. BOUND, HOP, and ELA are very close (BOUND and HOP are almost overlapping), while PGD is a bit far from BIM and FGM: since PGD has success rate 1, its scores in the analysis in Section 4.2 are not meaningful and are represented as 0 in the plot. Interestingly, CW2 is overall close to BOUND, HOP, and ELA, even if not so evidently as those three attacks. Other attacks are far from anyone else such as NEW or JSMA. DEEP, UNI and ZOO. Note that they are visually close in Figure 1.a, but according to the clustering analysis, they belong to neither group when k=2, and the other two analyses report contradicting results. This may suggest that those three attacks could represent a new group. However, the results of the other analyses do not confirm this result.
a) Aggregated results on ConvNet12.
From the ResNet50 results in Figure 1.b, we can identify one group composed of BOUND, HOP, and NEW. BOUND and HOP are very close to each other, validating the outcome of the ConvNet12 analysis. Differently, ELA is not included in the group as the detection axis has opposite scores (BOUND and HOP have a score near -0.5 while ELA is 0.5). In Figure 1.b, ZOO and JSMA are very close suggesting a possible group, but this is just due to the view angle of the plot: the two attacks show different scores on each of the 3 dimensions. Differently, JSMA and ZOO are closer to each other but with different scores from the detectability analysis: JSMA is likely part of a cluster, while UNI does not belong to any of them. DEEP is the farthest from all the others, forming a group by itself. According to the results of detectability and classification analyses, BIM and FGM are likely to be in the same group. However, they are very far from each other as it results from the clustering analysis. This contrasts with what we observed in Figure 1.a, in which they are grouped strongly.
We proposed an experimental methodology to identify the most suitable attack set when testing defenses against evasion attacks. The overall idea is to apply distance metrics to group attacks that introduce a similar perturbation on the image and have a similar effect on the target classifier. While the analysis performed on the two models shows some discriminative power, it does not provide enough evidence to drive guidelines that can reduce the number or types of attacks to be used when testing a defense. Results on ConvNet12 and ResNet50 agree in grouping BOUND and HOP attacks, which, share the same approach to craft attacks, according to the mathematical formulation. In weaker form, BIM and FGM are also grouped, with experiments on ConvNet12 giving a solid grouping and experiments in ResNet50 partially confirming the result. This is somehow expected as BIM is the iterative form of the FGM attack. However, this result alone applies only to those specific attacks, and no relevant relations are found in all the other cases.
In conclusion, our preliminary assessment suggests that measuring the adversarial perturbations does not provide a sensitive proxy to select diverse attack classes, meaning that different attack methods, with distinct mathematical formulations, do not introduce distinguishable (measurable) perturbations in the image. However, the methodology should be further enriched to provide additional verification of these results. This can be achieved by applying a broader range of models and attack methods, as well as identifying different metrics. However, the preliminary results we obtained are quite worrying. When evaluating a defense, it is risky to reduce the diversity of attacks, because the exhaustiveness of the evaluation can be unexpectedly compromised by essentially any reduction attempt.
This paper was partially supported by the MUR PRIN 2022 project FLEGREA - Federated Learning for Generative Emulation of Advanced Persistent Threats and the PRIN PNRR 2022 project BREADCRUMBS- Building up Robust and Efficient Routing Algorithms for Drones by integrating Connectivity and Risk awareness in an Urban air Mobility Bvlos Scenario. [14] Yim, Changhoon, and Alan Conrad Bovik. "Quality assessment of deblocked images." IEEE
Transactions on Image Processing 20.1 (2010): 88-98. [15] Wang, Zhou, and Alan C. Bovik. "A universal image quality index." IEEE signal processing letters 9.3 (2002): 81-84. [16] Wald, Lucien. "Quality of high-resolution synthesised images: Is there a simple criterion?"
SEE/URISCA, 2000. [17] Renza, Diego, Estibaliz Martinez, and Agueda Arquero. "A new approach to change detection in multispectral images by means of ERGAS index." IEEE Geoscience and Remote Sensing Letters 10.1 (2012): 76-80. [18] Sheikh, Hamid R., and Alan C. Bovik. "Image information and visual quality." IEEE
Transactions on image processing 15.2 (2006). [19] Zhou, Jie, Daniel L. Civco, and J. A. Silander. "A wavelet trans-form method to merge Landsat TM and SPOT panchromatic data." International journal of remote sensing 19.4 (1998). [20] Pushparaj, Jagalingam, and Arkal Vittal Hegde. "Evaluation of pan-sharpening methods for spatial and spectral quality." Applied Geomatics 9 (2017): 1-12. [21] González-Audícana, María, et al. "Fusion of multispectral and pan-chromatic images using improved IHS and PCA mergers based on wavelet decomposition." IEEE Transactions on Geoscience and Remote sensing 42.6 (2004): 1291-1299. [22] Yuhas, Roberta H., Alexander FH Goetz, and Joe W. Boardman. "Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm." Summaries of the 3rd Annual JPL Airborne Geoscience Workshop. Volume 1: AVIRIS Workshop. 1992. [23] D. Meng, and H. Chen, "Magnet: a two -pronged defense against adversarial examples," Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2017. [24] W. Xu, D. Evans, and Y. Qi, "Feature Squeezing: Detecting Adversarial Examples in Deep
Neural Networks," Network and Distributed System Security Symposium, 2018. [25] A. Krizhevsky, and G. Hinton, "Learning multiple layers of features from tiny images,"
Technical Report, 2009. [26] D. Warde-Farley, and I. Goodfellow, "Adversarial perturbations of deep neural networks," Perturbation, Optimization, and Statistics (editors: T. Hazan, G. Papandreou, D. Tarlow), 2016. [27] M. Xue, et al., "Machine learning security: Threats, countermeasures, and evaluations,"
IEEE Access 8: 74720-74742, 2020. [28] S. Bulusu et al., "Anomalous instance detection in deep learning a survey, " In: IEEE
Symposium on Security and Privacy, 2020. [29] D. J. Miller, Z. Xiang, and G. Kesidis, "Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks" Proceedings of the IEEE 108.3 (2020): 402-433. [30] Y. Deng, et al., "An analysis of adversarial attacks and defenses on autonomous driving models," IEEE Int. Conf. on Pervasive Computing and Communications (PerCom), 2020. [31] Li, Yao, et al. "A review of adversarial attack and defense for classification methods." The
American Statistician 76.4 (2022): 329-345. [32] N. Carlini, and D. Wagner, "Towards evaluating the robustness of neural networks," IEEE symposium on security and privacy (SP), pp. 39-57, IEEE, 2017. [33] Ma, Xingjun, et al. "Characterizing adversarial subspaces using lo-cal intrinsic dimensionality." arXiv:1801.02613 (2018). [34] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778. [35] M. I. Nicolae, et al., "Adversarial Robustness Toolbox v1.0.0," arXiv:1807.01069v4, 2019. [36] Xu, Han, et al. "Adversarial attacks and defenses in images, graphs and text: A review."
International Journal of Automation and Computing 17 (2020): 151-178. [37] Carlini, N., and Wagner, D., (2017a), “Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 3–14.G. Eason, B. Noble, and I. N. Sneddon, “On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,” Phil. Trans. Roy.
Soc. London, vol. A247, pp. 529–551, April 1955. (references) [38] Zou, H., and Hastie, T. (2005), “Regularization and Variable Selection via the Elastic Net,” Journal of the Royal Statistical Society: Series B, 67, 301–320. DOI: 10.1111/j.14679868.2005.00503.x. [39] Chen, P.-Y., Sharma, Y., Zhang, H., Yi, J., and Hsieh, C.-J. (2018) “EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples,” in Thirty-Second AAAI Conference on Artificial Intelligence. [40] Goodfellow, I., Shlens, J., and Szegedy, C. (2015), “Explaining and Harnessing Adversarial
Examples,” in International Conference on Learning Representations. [41] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018), “Towards Deep Learning Models Resistant to Adversarial Attacks,” in International Conference on Learning Representations. [42] Kurakin, A., Goodfellow, I., and Bengio, S. (2016), “Adversarial Examples in the Physical
World.” [43] Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. (2016), “Deepfool: A Simple and Accurate Method to Fool Deep Neural Networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2574–2582. [44] Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., and Swami, A. (2016), “The Limitations of Deep Learning in Adversarial Settings,” in Security and Privacy (EuroS&P), 2016 IEEE European Symposium on, 372–387. [45] Jang, Uyeong, Xi Wu, and Somesh Jha. "Objective metrics and gradient descent algorithms for adversarial examples in machine learning." Proceedings of the 33rd Annual Computer Security Applications Conference. 2017. [46] Chen, P.-Y., Zhang, H., Sharma, Y., Yi, J., and Hsieh, C.-J. (2017), “Zoo: Zeroth Order Optimization Based Black-Box Attacks to Deep Neural Networks Without Training Substitute Models,” in Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security [47] Brendel, W., Rauber, J., and Bethge, M. (2018), “Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models,” International Conference on Learning Representations.
Attack configuration parameters. In Table 2 we report the configuration parameters used to craft
the attacks in the experimental campaign. How these values are combined, and all configuration
details can be found at [
Distance metrics cluster analysis. First, we observe in Table 3.a that cluster 5 and cluster 8 group all the attacks except, again, BIM and PGD. On the contrary, some clusters indicate small groups of attacks. Cluster 2 and 11 strengthen the BIM, FGM, and PGD grouping. We also identify a group of attacks composed of HOP, BOUND, and ELA, in clusters 3, 5, and 8. Another cluster group is composed of ZOO and DEEP, which mostly fall in clus ters 5, 8, 9, and 12. Interestingly, the ZOO attack does not share its mathematical formulation with DEEP but generates a perturbation that distance metrics quantify very similar to the perturbation created by DEEP. Lastly, we observe that the JSMA attack is different from any other attack since it is
a) Results with k=12 clusters, using the ConvNet12 model b) Results with k=12 clusters, using the ResNet50 model 1 r e t s u l c the only attack that gets assigned to clusters 4 and 7. The analysis shows similar results concerning the ResNet50 model (Table 3.b). The BIM, FGM, and PGD group seems confirmed since the three attacks are the only ones with high scores in clusters 4 and 5. This is somehow expected as the Basic Iterative Method (BIM) and Projected Gradient Descent Attack (PGD) can be considered iterative versions of FGM [31]. As observed in ConvNet12 results, JSMA is isolated in clusters 1, 2, and 12 suggesting no similarities for the other attacks. Differently, cluster 9 evidences a big group of attacks. However, BOUND and HOP show a score (respectively, 1 and 0.998) that is significantly higher than the others, suggesting a stronger similarity. Further, ELA and NEW provide scores that are high (respectively, 0.768 and 0.747) but separated from the rest of the scores. This trend agrees partially with the analysis on ConvNet12 that groups together BOUND, HOP, and ELA, while NEW is excluded. The similarity between HOP and BOUND with the ELA and NEW cannot be explained or confirmed by checking their mathematical formulation, and it is a relation observable solely thanks to the distance metrics. Differently from the ConvNet12 case, DEEP and ZOO are not similar and form isolated groups (clusters 7 and 8 of Table 3.b). Lastly, UNI is different from any other attack according to the analysis of both models.
Correlation between distance metrics and the behavior of the target classifier. We show the results of, respectively, the ConvNet12 and the ResNet50 models in Table 4.b and Table 4.b. The columns report the attacks used to train the regressors, while the rows enlist the attacks used to test the regressors. Each cell i,j of the tables reports the CD when predicting the success rate
Trained using distance metrics from the adversarial images of attacks:
Trained using distance metrics from the adversarial images of attacks: Tested on:
BIM
BOUND
DEEP
FGM
HOP
JSMA
NEW
PGD of the attack in the i-th row using as a training set the attack in the j-th column. The CD is low (i.e., there is mutual predictability) in most cases: therefore, it is more interesting to observe the combinations for which the CD is high. Considering ConvNet12 and Table 4.a, it is the case of FGM against all attacks but BIM and PGD; also, BIM shows low CD only when paired with FGM. UNI shows good predictability when used to train the regressor, but rather poor predictability if it is used as a test set: therefore, we cannot get actionable information about this attack. Similar results can be observed in the ResNet50 analysis (Table 4.b). BIM can predict with CD=0 the success rate of FGM, PGD, and UNI, with a very low CD in detecting JSMA (see the 1st column in Table 4.b). We obtain similar results using FGM as training, showing a CD=0 in predicting BIM, PGD, and UNI. Using both UNI and PGD, the CD is 0 except for BIM and FGM. This seems to define a group composed of BIM and FGM, excluding UNI and PGD.
Correlation between distance metrics and the detector outputs. In Table 5.a, we observe that BOUND and HOP show mutual predictability. BOUND can predict the detectability of HOP with MCC=0.698, whereas, using HOP to predict the detectability of BOUND, we obtain MCC=0.642. CW attacks seem grouped with BOUND and HOP: if used as the training set, it can predict the detectability of HOP and BOUND attacks with an MCC of roughly 0.42. The results are similar if using HOP to predict CW2 (MCC = 0.365) but are lower using BOUND attacks (MCC = 0.234). Using HOP, BOUND, and CW2 as training sets shows some capabilities in detecting the BIM attack, however, the MCC scores are very low (below 0.3). The PGD attacks are not considered because it has a detection rate of 1.0 and as such cannot be efficiently used to train a classifier. Training with JSMA shows MCC of 0.360, 0.350, and 0.381 when tested with BOUND, CW2, and HOP respectively. Despite there being no mutual predictability between JSMA and these attacks (using CW2, BOUND, and HOP in the training, and JSMA as a test shows a low MCC), training on JSMA offers some capability to predict the others. The UNI, ZOO, NEW, and DEEP attacks are the ones with the lowest average MCC if used as training.
We repeat the analysis on the ResNet50 model, considering all the attacks except for CW2 which is computationally too complex to generate with our hardware. We show the results in Table 5.b. PGD, UNI, and ZOO do not have mutual predictability with any other attacks. Each of them shows an MCC score near 0 in predicting the detectability of each other, with a few exceptions showing a low MCC score. PGD, UNI, and ZOO are predicted poorly by other attacks. BIM, BOUND, DEEP, ELA, FGM, HOP, JSMA, and NEW attacks appear somehow related: attacks can be predicted by or can predict others. These results can be easily seen by looking at the rows of Table 5.b: except for PGD, UNI, and ZOO, there is no attack with the majority of MCC score to zero. The only exception is the BOUND attack (in the 2nd column) that shows MCC greater than 0 only in predicting BIM, HOP, and NEW. Also, the MCC score obtained using NEW when training suggests that NEW can be related to BIM, DEEP, ELA, FGM, HOP, and JSMA. The strongest similarity between attacks is observed for HOP and BOUND, with HOP being the only attack that shows MCC substantially greater than 0 in detecting BOUND. This result is consistent with the BOUND and HOP group evidenced by the ConvNet12 results. Differently from ConvNet12 results, BIM can predict FGM and DEEP with, respectively, MCC=0.60 and MCC=0.48. We obtain analogous results using FGM or DEEP in the training. FGM achieves MCC=0.50 in predicting BIM and MCC=0.62 in predicting DEEP. Similarly DEEP can predict BIM and FGM with MCC=0.39 and MCC=0.47. 0.234 0.043 0.262 0.071 0.698 0.143 0.149 0.018 0.074 0.117
Trained using distance metrics from the adversarial images of attacks:
BOUND CW2 ZOO
Trained using distance metrics from the adversarial images of attacks: BIM BOUND DEEP ELA FGM HOP JSMA NEW PGD UNI