Although the attention to machine learning robustness and its security implications became suddenly of interest, the idea of robust of machine learning (ML) is not new.
Pioneering articles back in 2004 raised awareness that the community naively assumed that the data generation process in data mining is independent of the system responses [1]. A few years later, [2] provided the first consolidated perspective of adversarial ML and the need for a change in how ML is thought, conceived, and applied. Standard, non-robust ML has three limits in the adversarial setting: (1) the first limitation is more proper 1 y7TIyr2jK9oHUJpNkEY/Zw5mQCz l<atexish1_b64="DuXqRfn+3dPM8OvGgW0>ABcVLSF code y to solve a generic task (i.e. classification). When we invert the latent code y and we go back in the input space, we have a full set of solutions. Adversarial examples are points not in that set and also very close to x yet mapping to another els operate under the assumption that the testing data is sampled from the same unknown generation process that have large variations of the output. Adversarial training fixes these points, making the model more “aware” of the distribucreated the data–i.i.d. assumption, independent and iden- tion of the input data. tically distributed. This is a too strict assumption when actual input data can be sampled from another generation process–o.o.d., out of distribution; (2) more importantly, the generation process is assumed to be disconnected classification, we ofer a new view on adversarial perturbations connecting them with generative modeling.
Following [6, 7], we analyze a robust model under the from the system capabilities of taking decisions (3) mini- lens of Energy-based Models (EBM), and we decompose mization of the empirical risk does not guarantee to be resilient to an adversary. To address those issues, the formalization of minimizing the maximum risk [3] by A. Wald of 1944 came at hand to cast the empirical risk the cross-entropy loss of (|x) to show that it contains the notion of the joint distribution of the data and the labels (x, ) along with the distribution of the data itself (x). Although adversarial perturbations are known minimization into adversarial risk minimization [4, 5], as input points that switch the decision boundary—thus following a proper definition.
In this paper, following the interpretation in [6] of a discriminative classifier (|x) implemented by a Convolutional Deep Neural Network (DNN) for multi-class afecting (|x)—we show that there is a strong dependency also with (x). In the scenario of untargeted attacks, we show that an increase in the attack strength implies a significant increase of (x). The increase is
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License so substantial that the attacks’ energy surpasses the mag- criminative and generative. They also show that hybrid nitude of the energy of natural data. Under the lens of training somehow slightly improves robustness to advergenerative modeling via EBM, we hope to shed some light sarial attacks. JEM was extended in [9] to stabilize and on a few phenomena that emerge from robust classifiers, speed up the training. Later on, [7] established the first such as: 1) robust models are naturally better calibrated connection on how adversarial training (AT) and EBM than standard classifiers, 2) robust models ofer gradi- bend the energy function in a diferent way yet using ents of the loss wrt to the input with more structure, and a similar contrastive methodology. Very recently [10] better “explain” the data 3) the trade-of between robust moved in the same direction of demystifying the generand natural accuracy. Fig. 1 shows the idea of training ative capabilities of robust models developing a unified a robust model: in the standard case, when we train a probabilistic framework, and working also in unsuperdiscriminative classifier, we model (|x), and thus we vised settings. Finally, [11] showed that AT improves seek for a function that maps the fixed input point x Joint Energy-Based Generative Modelling, while incorpoto a latent code y that is useful for the task, for instance, rating into the modeling a sharpness-aware minimization minimize the cross-entropy loss. A discriminative model (SAM) procedure [12]. ignores the input data distribution, thereby adjusting its Synthesis with a robust model. At the best of our parameters to separate the data. Furthermore, the input knowledge [13] is the first paper to employ a robust classpace is usually larger than the embedding space, and the sifier for synthesis since it is shown that robust models model has to build invariance to nuances of the training attain input gradients that better correlated with “human data to achieve classification. What we do not enforce, perception” [14, 15]. The community argued in [16] that instead, is that the inverse of has to fall within a bound- AT renders the discriminative model invertible, while ary of the input data. In other words, when we train the [17] believes that AT yields gradients closer to the image discriminative model, the function is not bijective, and manifold, which is in line with the connection between inverting the function gives as output a set. Adversarial robust and generative model. The generative capabilities samples are points in the input space very close to x yet of robust models have been used by [18] for solving inmap to a diferent latent code far from y. Adversarial verse problems and for controllable image synthesis [8]. Training (AT) [4, 5] restores the “right” connection.
In this paper we make the following contributions: ◇ Viewing a discriminative model under the lens of an 3. Preliminaries EBM, we show that the more we increase the strength of an untargeted attack, the more (x) increases, even In Section 3.1, we will briefly review the settings of advermore than the points naturally present in the dataset. sarial attacks in a white box scenario, while Section 3.2 ◇ We show that the energy of an input point x can be we examine data density modeling of the input data using used as a strong detector for adversarial data. The more an Energy-based Model (EBM). the attack increases its strength, the easier to detect. We do so by performing no training of the detector except 3.1. Adversarial Perturbations choosing a threshold.
◇ We ofer a way to bypass the aforementioned detector demonstrating that is possible to design an adversarial perturbation that crosses the decision boundary yet its energy is kept similar to those of natural data.
◇ Finally, we emphasize that a robust or quasi-robust model can be efectively employed to enhance image synthesis in generative AI, supported by a recent paper by [8], where image synthesis is performed by inverting a quasi-robust model.
A DNN : R → R is a universal approximator for generic functions [19] that maps an input data x into another meaningful code y using a loss function ℒ. In machine vision, usually is high dimensional, with ≫ , and ℒ guides in solving the task at hand.
The definition of adversary specifies that, though may perform well on the task yet is brittle since is possible to obtain a slightly perturbed instance of x⋆ that fools on the same task such that x⋆ ≈ x. The gist of all the attacks follows an inverse optimization process in which the attacker optimizes an adversarial point x⋆ ∈ R in the input space so that the model ascends the loss in the output space (i.e. untargeted attack) or makes a confident response of an erroneous label, decided a priori by the attacker (i.e. targeted attack). The attacker often yields a constraint on how much information needs to be changed in the input. In most of the current attacks, the bound on the information is implemented with pixelwise norm as =. x⋆ − x and || || ≤ . All the
Robust and Generative Models. We believe the first connection between robust and generative models has been done in the paper by Grathwoh et al. [6], where for the first time, a discriminative DNN ending with the softmax classifier is interpreted as an EBM. [ 6] proposes JEM to train the DNN in a hybrid way—to be both disattacks follow the above procedure with some diferences Nevertheless, the latter is not efortless and several on how many steps are taken to ascend the loss: one for sampling methods have been designed to approximate it Fast-Gradient Sign Method (FGSM) with its variants [4, eficiently [ 26]. Specifically, it is known that the derivative of Eq. (4) w.r.t is: ∇ ℒ( ) = Ex+∼ [︁ ∇ (x+)]︁ − Ex− ∼ [︁ ∇ (x− )]︁ with the number of classes. Notice that, for any classiifer , the energy of a data point x is: (x) = − log ∑︁ exp (︀ (x)[])︀ =1 Indeed a data point with a high probability of occurrence— high (x)—is equivalent to having low energy. 4.
Analyzing the energy (x) gives us the opportunity to (4) examine and study the generative model inside classifiers. 20], multi-steps with projections for Projected Gradient Descent (PGD) [5] or in how the loss is designed, e.g. Carlini & Wagner (CW) attack.
Attacks can be crafted in targeted or untargeted settings even in the physical world [21] and can transfer between diferent models. Whilst attack literature is solid, defenses have several limitations [22]: despite being computationally expensive, the only defense to withstand is Adversarial Training (AT) [4, 5]. Though in this paper we deal mainly with the image domain, adversarial perturbations are known in other domains (Natural Language Processing and tabular data) as counterfactual explanations [23]. Below we briefly review the two main methods to generate perturbations of the input given a DNN. Fast Gradient Sign Method. FGSM [4] is a ∞-norm adversarial attack and it is described in the following formula below: x * = x + sign (︁
∇x︀[ ℒ( (x), )]︀ )︁ where sign returns the sign of its argument.
Projected Gradient Descent. PGD can be seen as an iterative generalization of the FGSM algorithm [5]. The iterative procedure is shown below: x
distribution as: EBMs [25] are based on the idea that any probability density (x) for x ∈ R can be expressed via a Boltzmann (x) = exp (− (x))
( ) where (x) is called energy, modeled as a neural network, that maps each input x to a scalar. ( ) is the normalizing constant, known as the partition function, such that (x) is a valid probability density function; the challenge for training EBMs is approximating the constant ( ). Training an EBM is performed via maximum likelihood estimation by minimizing the negative log-likelihood of the data: ℒ( ) = Ex∼ − log (x) step size. This algorithm is able to produce strong adver- (x)[] is the k-th index of logits (x), and (x, ) = =1 (|x) = (x, ) (x) = exp (︀ − (x, ))︀ ∑︀
=1 exp (︀ (x)[])︀ (x) = ∑︁ (x, = ) = ∑︀ =1 exp ( (x)[])
( )
Unfortunately, for EBMs we can not sample from , therefore several MCMC methods have been proposed to sample from that density function [26]. For instance, in [6] Stochastic Gradient Langevin Dynamics (SGLD) has been used to train EBMs using gradient knowledge.
SGLD sample with the following procedure: x0 ∼ 0(x) , x+1 = x − (x) 2
+ ∼ where 0(x) is typically a Uniform distribution and
(0, 1). Thus, we can summarize that the training of EBMs consists of two steps: generating approximate data samples from using MCMC methods and optimizing the model parameters to increase the energy of these samples and decreasing the energy of the real samples via gradient descent.
DNN as an EBM. Recently, Grathwohl et al. [6] made a connection between DNN classifiers and EBMs. They show that an EBM is hidden inside a standard classifier and we can re-interpret the logits of a model to build an EBM to compute (x, ) and (x): (x, ) = exp ( (x) [])
( ) where ( ) is the unknown normalizing constant, compute ( |x) and (x) as: − (x)[] is the energy function. Following [6] we can (1) (2) (3) (5) (6) (7) (8) 700 600 0% 10
20 # steps (b) 0% 0% example detection. Our strategy detects PGD adversarial perturbations at inference time in a DNN equipped with a softmax classifier, without any additional computation overhead and any additional parameters. Firstly, the approach estimates a threshold in the domain of (x) using a validation set that balances the true positive rate (TPR) and false negative rate (FNR) based on a chosen metric. Subsequently, it uses this threshold to perform detection: ( , x) = {︃1 if − log ∑︀
=1 exp (︀ (x)[])︀ ≤ 0 otherwise To better understand the characteristics of adversarial examples, we performed an energy analysis of DNNs for both adversarial examples generated using the untargeted PGD attack and natural data. Our results indicate a substantial diference in energy between adversarial and natural data, with adversarial points having much lower energy than natural points. These findings are exhibited in Fig. 2, which displays the energy for two diferent models on diverse datasets. In Fig. 2(a), the #steps (iterations) used is 40 in PGD and is always 8/255. Indeed, our analysis reveals an intriguing dependency between the energy function (x) and the “strength” of an untargeted adversarial attack. It appears that the energy of adversarial data decreases as the strength of the attack increases. In our case, we define the attack strength as the discrete value of the number of steps of a PGD attack.
Considering these results and reinterpreting them using yond the classic idea that adversarial attacks cross the decision boundary, we show that a DNN tends to “believe” that adversarial data are highly likely under the hidden generative model (x). More surprisingly, they are even more likely than the natural data itself.
Based on the previous observations we propose the construction of a very simple detection algorithm for catching adversarial data using the energy function as a 1D discriminant. This idea has already been applied in [27] to detect out-of-distribution (OOD) data; however, there the EBM framework, it can be inferred the following. Be- is a hyper-parameter that controls the strength of the is a key diference from [ 27] and us: while the OOD data In this section, we describe our experiments and confirm tend to move to higher values of (x), in our case, we the eficacy of the energy function as a detection method edge, this technique was not yet applied to adversarial between robust and generative models.
The development of strong adversarial detectors is a significant research area, as such detectors typically depend on specific types of attacks. Although the energy approach is efective for identifying adversarial data with low energy, such as those generated via PGD, a crucial question is whether it is possible to generate perturbations that can mislead the aforementioned method. That means whether it is possible to produce adversarial examples that own energy comparable to natural data. High-Energy PGD. In order to generate adversarial examples with similar energy as natural data, we propose to introduce a regularization term during the optimization step to force the creation of perturbations that lead to adversarial data with higher energy. The regularizer will remove the energy gap between natural and adversarial data. Specifically, we define the following optimization [︁ℒ(︀ (x + ), )︀ + (x + ) ]︁ (10) problem: arg max (9) (11) where ℒ is the loss function, like cross-entropy and regularization. Eq. (11) describes the High-Energy PGD attack (HE-PGD) implemented as: x
We use the imagenette [28], CIFAR-10 [29], and CIFAR100 [29] datasets as benchmarks. All attacks except HEPGD are implemented using torchattacks [30]. HE-PGD is implemented directly in PyTorch with a parameter of 1.2.
Evaluation Metrics. We measure the following metrics: 1) detection rate (DR); 2) false positive rate (FPR). We utilize the G-mean metric to estimate the threshold; G-mean is defined as the squared root of the product of sensitivity and specificity. Table 1 shows the detector’s performance on diverse attack settings. Energy as a detector achieves larger DR for PGD-based perturbations on CIFAR-10, also scale to imagenette [28] where the input space is much larger than CIFAR, and works for unseen attacks such as APGD [24].
Energy (ResNet10) Energy (ResNet10) KD+BU[31] LID [32]
PGD (8) PGD (16) PGD (8) APGD (8) PGD (8) PGD (8)
DR 98.24 99.6 98.38 85.45 92.27 94.39 ℓ2, =0.01 ℓ2, =0.05 ℓ∞, = 205.55 ℓ∞, = 215.05 Figure 4: A regular discriminative model shows gradients of the loss respect to the input with sparse activations compared to robust models while these latter are more meaningful which better motivate them for synthesis and connect them to generative models.
model that becomes generative by estimating gradients of the data distribution as a score-matching model [34].
There is also a strong parallelism between adversarial 5.2. HE-PGD Results attacks and sampling from score-matching using SGLD— Our experiments in Fig. 3 show that HE-PGD is at least Eq. (6). Recently [8] showed that is possible to invert a as efective as conventional PGD while it keeps the en- quasi-robust model to perform convincing image syntheergy of the adversarial points similar to the natural data. sis. A quasi-robust model is a model which is non-robust Indeed in Fig. 3(a) the two distributions now largely over- from the point of view of security yet it inherits generalap since the adversarial samples generated by HE-PGD tive capabilities and keeps its well-performing accuracy. have higher energy than those produced by PGD and are To generate the samples, [8] takes steps in the direction better aligned with the energy of the natural data. Con- of the gradients that better carries information of class sequently, when the value of is chosen appropriately, conditional data distribution (x|). HE-PGD can bypass the energy-based detector, presented in Section 4.2. The stepsize of all attacks is fixed at 1/255. 6. Conclusions
From Fig. 3(b), we note that, unlike before, (x) is approximately invariant to the number of steps (#steps) in projected gradient descent. At the same time, the classifier accuracy still drops to zero for higher iterations yet it does so slightly more gradually than before.
In this paper, by viewing a robust discriminative model as an EBM, we were able to detect adversarial example efectively and demonstrate that there is the possibility of bypassing the detector introducing a new attacked called High-Energy PGD. We believe that exploring the connection between robust and generative modeling may shed some light on interesting properties of robust classifiers.
The evidence that robust models have input gradients aligned with semantic properties of the object, as shown Acknowledgment. The authors thank Dr. Mozhdeh in Fig. 4, can be explained under the lens of generative Rouhsedaghat for providing the figure of the gradient of modeling. Aggregating gradients of the loss over the the standard and quasi-robust models. This project is parinput can be seen as a way to sample points from the tially supported by Sapienza research project “Prebunkgenerative model hidden inside the discriminative clas- ing: predicting and mitigating coordinated inauthentic sifier. What AT may be doing is rendering it a hybrid behaviors in social media” and PNRR PE01 FAIR project. bustness and gradient interpretability, in: ICLR Workshops, 2019. [1] N. Dalvi, P. Domingos, S. Sanghai, D. Verma, Ad- [18] R. A. Rojas-Gomez, R. A. Yeh, M. N. Do, A. Nguyen, versarial classification, in: Proceedings of the tenth Inverting adversarially robust networks for image ACM SIGKDD international conference on Knowl- synthesis, arXiv:2106.06927 (2021). edge discovery and data mining, 2004, pp. 99–108. [19] B. C. Csáji, et al., Approximation with artificial [2] M. Barreno, B. Nelson, A. D. Joseph, J. D. Tygar, The neural networks, Faculty of Sciences, Etvs Lornd security of machine learning, Machine Learning 81 University, Hungary 24 (2001) 7.
(2010) 121–148. [20] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, [3] A. Wald, Statistical decision functions which min- J. Li, Boosting adversarial attacks with momentum, imize the maximum risk, Annals of Mathematics arXiv preprint (2018).
(1945) 265–280. [21] A. Kurakin, I. Goodfellow, S. Bengio, Adversarial [4] I. Goodfellow, J. Shlens, C. Szegedy, Explaining and machine learning at scale, in: ICLR, 2017.
harnessing adversarial examples, in: ICLR, 2015. [22] F. Tramer, N. Carlini, W. Brendel, A. Madry, On [5] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, adaptive attacks to adversarial example defenses, A. Vladu, Towards deep learning models resistant arXiv:2002.08347 (2020).
to adversarial attacks, in: ICLR, 2018. [23] N. Madaan, I. Padhi, N. Panwar, D. Saha, Generate [6] W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Du- your counterfactuals: Towards controlled countervenaud, M. Norouzi, K. Swersky, Your classifier factual generation for text, in: AAAI Conference on is secretly an energy based model and you should Artificial Intelligence, volume 35, 2021, pp. 13516– treat it like one, in: ICLR, 2020. 13524. [7] Y. Zhu, J. Ma, J. Sun, Z. Chen, R. Jiang, Y. Chen, Z. Li, [24] F. Croce, M. Hein, Reliable evaluation of adversarial Towards understanding the generative capability robustness with an ensemble of diverse parameterof adversarially robust classifiers, in: ICCV, 2021, free attacks, in: International conference on mapp. 7728–7737. chine learning, PMLR, 2020, pp. 2206–2216. [8] M. Rouhsedaghat, M. Monajatipoor, C.-C. J. Kuo, [25] Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, I. Masi, MAGIC: Mask-guided image synthesis by F. Huang, A tutorial on energy-based learning, inverting a quasi-robust classifier, in: AAAI Con- Predicting structured data 1 (2006). ference on Artificial Intelligence, 2023. [26] Y. Song, D. P. Kingma, How to train your energy[9] X. Yang, S. Ji, Jem++: Improved techniques for based models, arXiv:2101.03288 (2021).
training jem, in: ICCV, 2021, pp. 6494–6503. [27] W. Liu, X. Wang, J. Owens, Y. Li, Energy-based [10] Y. Wang, Y. Wang, J. Yang, Z. Lin, A unified con- out-of-distribution detection, Advances in neural trastive energy-based model for understanding the information processing systems 33 (2020) 21464– generative ability of adversarial training, in: Inter- 21475. national Conference on Learning Representations, [28] fastai, imagenette, 2023. URL: https://github.com/ 2022. fastai/imagenette. [11] R. Korst, A. Asadulaev, Adversarial training im- [29] A. Krizhevsky, G. Hinton, et al., Learning multiproves joint energy-based generative modelling, ple layers of features from tiny images, Technical arXiv:2207.08950 (2022). Report, Citeseer, 2009. [12] P. Foret, A. Kleiner, H. Mobahi, B. Neyshabur, [30] H. Kim, Torchattacks: A pytorch repository for Sharpness-aware minimization for eficiently im- adversarial attacks, arXiv:2010.01950 (2020). proving generalization, in: ICLR, 2021. [31] R. Feinman, R. R. Curtin, S. Shintre, A. B. Gard[13] S. Santurkar, D. Tsipras, B. Tran, A. Ilyas, L. En- ner, Detecting adversarial samples from artifacts, gstrom, A. Madry, Image synthesis with a single arXiv:1703.00410 (2017).
(robust) classifier, in: NeurIPS, 2019. [32] X. Ma, B. Li, Y. Wang, S. M. Erfani, S. Wijewickrema, [14] G. Aggarwal, A. Sinha, N. Kumari, M. Singh, On G. Schoenebeck, D. Song, M. E. Houle, J. Bailey, the benefits of models with perceptually-aligned Characterizing adversarial subspaces using local gradients, in: ICLR, 2020. intrinsic dimensionality, arXiv:1801.02613 (2018). [15] S. Kaur, J. Cohen, Z. C. Lipton, Are perceptually- [33] A. Aldahdooh, W. Hamidouche, S. A. Fezza, O. Déaligned gradients a general property of robust clas- forges, Adversarial example detection for dnn modsifiers?, arXiv:1910.08640 (2019). els: A review and experimental comparison, Artifi[16] M. Terzi, A. Achille, M. Maggipinto, G. A. Susto, cial Intelligence Review 55 (2022) 4403–4462.
Adversarial training reduces information and im- [34] Y. Song, S. Ermon, Generative modeling by estimatproves transferability, arXiv:2007.11259 (2020). ing gradients of the data distribution, in: NeurIPS, [17] B. Kim, J. Seo, T. Jeon, Bridging adversarial ro- 2019.