As the use of machine learning continues to expand, the importance of ensuring its safety cannot be overstated. A key concern in this regard is the ability to identify whether a given sample is from the training distribution, or is an "Out-Of-Distribution" (OOD) sample. In addition, adversaries can manipulate OOD samples in ways that lead a classifier to make a confident prediction. In this study, we present a novel approach for certifying the robustness of OOD detection within a ℓ2-norm around the input, regardless of network architecture and without the need for specific components or additional training. Further, we improve current techniques for detecting adversarial attacks on OOD samples, while providing high levels of certified and adversarial robustness on in-distribution samples. The average of all OOD detection metrics on CIFAR10/100 shows an increase of ∼ 13%/5% relative to previous approaches. Code: https://github.com/FraunhoferIKS/distro
to diferentiate between in-distribution (ID) and
out-ofdistribution (OOD) samples. Recent approaches include
Although recent advances in Machine Learning (ML) Virtual Outlier Synthesis (VOS) [
OOD Detection. With Maximum Softmax Probability terwolf et al. [13], Meinke et al. [14] bring forth ℓ∞-norm
(MSP) [
Methods
- Standard
OE [
Clean ✓ ✓ ✓ (✓) (✓) ✓ ✓ In-Distribution (ID) Accuracy Out-Of-Distribution (OOD) Detection Adversarial ℓ∞
Certified ℓ2
Clean
Adversarial ℓ∞
Certified ℓ∞ ℓ2
Asymptotic underconfidence ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ (✓) (✓) ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 2. Background anteed upper bound on the classifier’s confidence within a defined region surrounding the input. Unlike before, certified robust OOD detection can now be computed for standard OOD detection approaches. Additionally, we incorporate a difusion denoiser [ 16, 17], which recovers the perturbed images and returns high quality denoised inputs. This leads to better levels of both adversarial and certified robustness for ID and OOD data. This work and previous methods are compared in Table 1.
In summary, our contributions are: created during training, it can still be susceptible to unseen new attacks. To overcome this problem, certified defenses formally guarantee the stability of the prediction in a neighbourhood of the input. In other words, a neural network is certifiably robust for the input ∈ R, if the prediction for all perturbed versions ˜ remains unchanged such that ‖˜ − ‖ ≤ , where ‖·‖ is the ℓ-norm around of size > 0.
tion method [
SmooThing for Robust OOD detection. This method incorporates a difusion denoiser model to improve the detection of adversarial and certified OOD samples, while providing high adversarial and certified accuracy for ID data.
where > 0 represents the standard deviation. As
previously, we define the hard version of () as () =
arg max∈ (). Cohen et al. [
We define a hard classifier as a function : R → sLiefiermma d2e.fin1e.d[Yaasnign eEt qaul.at[i1o8n]] 1G,isvuecnhathsmatoothe(d)cla=swhich maps input samples ∈ R to output ∈ , (()1, . . . , () ) is a vector of probabilities that wAdhdeirteiona=lly{,1w, e..i.n,tro}duiscethae sdoifstcrveetressioent of : cRlass→es. oanssiignnpsuttoe,aacnhdctlhaessp1r,ob..a.b,ility. Sisupp=osepr∈edicts(cl)ass> P() of , where P() is the set of probability distri- 1/2, then continues to predict class when is perturbed butions over . It is possible to convert any soft clas- by any with: sifier into a hard classifier by mapping () = arg max∈ (). Additionally, we define as (0, 1) ‖ ‖2 < Φ − 1(). the standard Gaussian distribution, as Φ( ) the Gaussian CDF and as Φ − 1() its inverse (or quantile). One should consider as the probability that the
Robustness Certificates. Even though an smoothed classifier will assign to the predicted class adversarially-trained network is resilient to attacks rather than any other. As a consequence, if > 1/2, Difusion Denoised Smoothing. In a nutshell, for- i.e. ℎ() ≤ max‖˜− ‖≤ ℎ(˜) ≤ ℎ¯(). The lower ness for plain models. Following this trend Carlini et al. then, we have that:
Due to the intractable nature of the maximization
problem, we can compute upper or lower bounds only,
bound ℎ() is typically calculated using projected
gradient methods [
∞-norm, (lower bound of WCAUC), is computed using IBP in Bitterwolf et al. [13] and Meinke et al. [14].
bound of any classifier without the need for special train(2) ing or modifications. Thus, the main theorem for an ℓ2-norm robustly certified upper bound is stated.
Theorem 3.1. Let : R
→ P() be any soft classiifer and be its associated smooth classifier as defined in Equation 1, with > 0. If = max∈ () > 1/2, it will continue to do so even if the input is perturbed by Gaussian noise of magnitude smaller than the radius Then is diferentiable, and moreover: be measurable, and let () = E ∼ (0, 2)[ℎ( + )]. 0, let ℎ : R → R ∇() = 1 2 ∼ (0, 2)
E [ · ℎ( + )] .
The smoothed function is also known as the Weierstrass transform of ℎ, and a classical property of the Weierstrass transform is its induced smoothness. ward difusion involves adding Gaussian noise to an image until it produces an isotropic Gaussian distribution with a large variance. Denoising difusion probabilistic samples a noisy image as follows: models work by learning how to reverse this process. In formal terms, given an input sample ∈ R, a difusion model selects a predetermined timestep ∈ N + and d=ef √ · + √ 1 − · (0, ), where the amount of noise to be added to the image is determined by a constant called derived from .
As Salman et al. [21] suggested, denoising Gaussian pertubed images leads to out-of-the-box certified robust[17] make use of a difusion model as one-shot denoiser achieving state-of-the-art performances. The minor prosion ˜ = + is scaled by √ posed adjustment held in the estimation of , computed such that 1− = 2. Additionally, the perturbed ver , to match the noise model of Equation 2. ples within a ℓ2-sphere around the input.
Preliminaries. To determine how well a classifier distinguishes between ID and OOD samples, it is common to threshold the confidence level and to calculate the area under the receiver operating characteristic curve (AUROC or AUC). Formally, let us consider a function1
tion [22]. ℎ ∈ R
→ R, the AUC is defined as: AUCℎ(, ) = ︀[ 1ℎ()>ℎ()︀] , ∼ ∼
E , where , are ID and OOD data sets, respectively, and 1 returns 1 if the argument is true and 0 otherwise.
the worst-case AUC (WCAUC), which is defined as the lowest AUC attainable when every OOD sample is perturbed so that the highest level of confidence is achieved within a specific threat model. Specifically, the WCAUC is defined as: WCAUCℎ(, ) = ∼ ∼
E , ︂[ 1ℎ()>
max ‖˜− ‖≤
︂] ℎ(˜) .
This section explains how using local Lipschitz conti- () = E ∼ nuity, achieved through smoothing the classifier with Gaussian noise, can guarantee the detection of OOD sam- transform of (which coincide with the smooth version ∈ √︂ 2 Φ − 1() + ,
(3) for every ‖ ‖2 < Φ − 1().
Proof. As a prerequisite to proving the theorem, we need to know the analytic form of the gradient of a smoothed function given in Lemma 2.2. Let us consider the soft classifier () : R
(0, 2)[ ( + )], with >
0. Since → P(), and its smooth version its a measurable function, we consider the Weierstrauss of ):
E ∼ (0, 2)
[ ( + )] = (︀ * (0, 2))︀ (), is diferentiable and from Lemma 2.2 we have: where * denotes the convolution operator. Thus, () ∇() = 1 2 ∼ (0, 2)
E [ · ℎ( + )] .
2 2
⃒ 2 √︂ 2 ⃒ 2
, Since : R
→ [
we have that ⃒ ⃒ ≤ ⃒⃒ (2 1 ⃒⃒ , − ⟩ ⃒⃒⃒ (︁ − 212 ‖− ‖22)︁ , (2 1 = 2 ∼ (0, 2)
E
[||] = where we make use of the triangle inequality and know sians are Gaussians and from the classical integration of the Gaussian density, we obtain: 1 2)/2 ∫︁ ⃒ ⟨
R ⃒
⃒⃒ , − ⟩ ⃒⃒⃒ (︁ − 212 ‖− ‖22)︁ , where we consider the supremum over all unit vectors . Since, we know that () is
√︁ Formally: it is possible to use the Lipschitz constant to bound the diference between
( + ) and () for any value of , with ‖ ‖2 < Φ − 1(), where = max∈ (). |( + )| ≤ 2 ‖ ‖2 + |()|, where we make use of the reverse triangle inequality. √︂ 2 √︂ 2 Φ − 1() + max ().
∈ Since () : R (), and moreover:
∈
→ [
Another limitation is that the guarantees provided by
this theorem are only probabilistic in practice. Therefore,
a hypothesis test [24] should be used to avoid making
predictions with low confidence. As with randomized
smoothing [
Despite these limitations, the theorem provides a novel way of calculating the upper bound of any classifier, without the need for special training or modification. Addivious approaches [13, 14], as they used IBP. This can be useful for evaluating the certified robustness of a broader category of standard OOD detection methods as well as larger models, where IBP bounds explode in size and make them unusable [25].
combines three techniques: (i) a difusion denoiser, (ii) a standard OOD detector, and (iii) a certified binary discriminator. Each component of this method is designed to overcome a specific problem of ordinary classifiers, as they are not robust to adversarial attacks, either ID or
Denoiser denoiseonce( + ; ) ˜
Classifier
ℎ(˜) Discriminator ()
P(|, ) P(|)
P(|) that is bounded by 1. Given that projections of Gaus- tionally, we provide a tighter certificate compared to pre
most likely class more than half the time, it is locally
bounded within a radius smaller than =
√︁ 2 Φ − 1().
Discussion
While this theorem provides some advantages, it is
important to note a couple of its limitations. One of the
main limitations is that the upper bound of the smooth
classifier only applies to and not to the original
classifier . As a result, the guarantee only applies to , and
its robustness at a given input point cannot be precisely
evaluated or certified. To overcome this, Monte Carlo
algorithms can be used to approximate these evaluations
with high probability [
Secondly, numerous post-hoc OOD detection methods
exist. The most straightforward being MSP [
104 nator is combined with the OOD detection method from Asymptotic Underconfidence. Here, we show that (ii) which is necessary to have the property of asymptotic by coupling a classifier trained to be OOD aware with a underconfidence for far-OOD inputs. difusion denoiser and running a certified discriminator
Configuration. This method does not require any in parallel, we can guarantee asymptotic underconfidence
new technical knowledge. We begin by making the as- for data far enough from the training distribution.
sumption that OOD samples are unrelated and thus max- To obtain asymptotic underconfidence of the joint
clasimally un-informative to the ID data. Thus, for every sifier, we consider P(|, ) ≤ 1 and rewrite Equation 4
class ∈ , the conditional distribution on the input as follows:
is given as:
where P(|) is the conditional distribution representing
the probability that is part of the ID, while P(|, ) is
the conditional distribution representing the ID. Similarly
to Meinke et al. [14], we assign independent models to
each distribution:
• P(|, ) = ℎ(denoiseonce( + ; )), where
ℎ : R → [
As can be seen, the denoiser is the main addition. The one-step denoiser denoiseonce estimates the fully denoised image from the current timestep . Then it computes the average between the denoised image and the noisy image from the previous timestep. As discussed in Carlini et al. [17], multiple applications of the denoiser will only destroy information about . Denoising with iterative steps essentially transfers the classification task to the denoiser, which can determine how the image should be filled. For these reason, we apply only a single step of denoising.
P(|) ≤ − 1
P(|) + 1 .
(5) Since the right term only depends on P(|), we just need to assure that lim →∞ P(| ) → 0. If we employ a certified binary discriminator, trained with IBP on OOD data, as descibed in Meinke et al. [14], to compute P(|), we achieve asymptotic underconfidence independently of the main classifier. Readers are referred to Meinke et al. [14] for a more detailed explanation.
pirical evaluation of the asymptotic confidence for standard and robust OOD detection methods2. In this test, we consider a single ID sample and multiply by a scalar .
Figure 2b we plot the Energy [22] for increasing values of > 0. In the context of MSP, we observe that standard
dent, after a small drop, whereas certified methods such as GOOD [13], ProoD [14] and DISTRO converge to 1/.
whereas GOOD [13], ProoD (Meinke at al., 2022), and
As a result, underconfidence can be easily obtained when using an energy score instead of MSP, regardless of whether it is on a plain or OOD aware model. However, asymptotic underconfidence does not necessarily imply that the model will perform better in detecting OOD
samples since all inputs are usually normalized to some
range (e.g. [
set of CIFAR10 to estimate the clean accuracy and report the average. Further, we ran AutoAttack in both rand and standard modes, and considered the lowest results for DISTRO and DDS.
In this section, DISTRO is evaluated for a variety of
robust ID and OOD tests and is compared to previous
approaches. As baseline, we consider the pre-trained
(mPoladienls)3afnrdomouMtlieeirnekxepeotsuarle. [(1O4E].) [T7h]emnoodremlsaslhtarraeinthede Method Clean =A2d/v2e5r5saria l =(ℓ∞8/)255 = C0.e1r2tified (ℓ=2)0.25
same ResNet18 [26] architecture and hyperparameters Plain* 95.01 2.16 0.00 28.14 14.17
as ProoD [14]. GOOD [13] uses a ’XL’ convolutional VOOE*S† 9954..5632 12..9274 00..0000 3113..4183 1100..8082
neural network. Additionally, we evaluate the pretrained LogitNorm‡ 94.48 2.65 0.00 12.53 10.25
DenseNet101 [27] models for ATOM [12] and ACET [10]; ATOM* 92.33 0.00 0.00 0.00 0.00
and the standard OOD detection methods: VOS4 [
We evaluate all methods on the standard datasets demonstrates better results than ProoD for adversarial CIFAR10/100 [29] as ID. For the OOD detection attacks and worse in terms of certified accuracy. Suprisevaluation we consider the following set of datasets: ingly, ACET reveals strong adversarial and certified acCIFAR100/10, SVHN [30], LSUN [31] cropped curacy despite of its reduced clean accuracy. Meanwhile, (LSUN_CR) and resized (LSUN_RS), TinyImageNet [32] ATOM results in zero for all tests since any slight percropped (TinyImageNet_CR), Textures [33] and turbation of the input triggers the last neuron used for synthetic (Gaussian and Uniform) noise distributions. OOD detection.
5.1. In-Distribution Results Here, we compare clean, adversarial, and certified accuracy for ID samples. Adversarial accuracy is evaluated with AutoAttack [23] for ℓ∞-norm attacks of budget ∈ {2/255, 8/255}. We ran the standard version of AutoAttack without additional hyper-parameters. Certified accuracy is evaluated for ℓ2-norm robustness of deviation ∈ {0.12, 0.25}. To this end, random smoothing is performed on 10’000 Gaussian distributed samples around the input with a failure probability of 0.001. All > 0 are considered for the certified accuracy. In the context of
and certified robustness while maintaining high clean accuracy. As difusion introduces variance into gradient estimators, standard attacks become much less efective. Nevertheless, robustness accuracy of difusion models varies over diferent runs for the same input, so it should be defined diferently from deterministic accuracy, e.g. as expectation. Luckily, one-shot difusion introduces such a tiny variance that throughout a few of runs, our results were similar. 5.2. Evaluation Metrics
the confidence of the classifier, i.e. MSP [
and GFPR) are computed for ℓ2 and ℓ∞ norms robustness certificates. Similarly to Meinke et al. [14], the ℓ∞-norm is obtained with IBP only on OOD data. On the other hand, the ℓ2-norm is computed with Theorem 3.1 on both ID and OOD data. Similarly to subsection 5.1, we sampled 10’000 Gaussian data points around the input with a deviation = 0.12. Since, the certified bound is only probabilistic in practice, we ran a binomial proportion confidence test [ 34] with failure probability of 0.001. We have assigned a score of 0 to all samples that fail to be certified, i.e. with ≤ 1/2. The Lipschitz continuity does not hold in the case of non-certified samples, therefore we are unable to bound the score. To ensure a fair comparison, we decided to compute the ℓ2-norm GAUC on both ID and OOD datasets.
In Figure 3, we plot the normalized frequency of occurrences of the certified upper bound ( √︀2/ · Φ − 1() + ) for ID versus OOD data of DISTRO. We observe that OOD data tend to peak close to zero, while ID data are spread out with larger values. This suggests that a large radius is more likely to be associated with ID data versus OOD samples. As a result, robustly certifying the detection of
5.3. Out-Of-Distribution Results
viously, we grayed-out models with an accuracy drop greater than 3% with respect to the model with highest accuracy. The objective of this choice is to prioritize
demonstrates the po
√ · . - Standard Plain* OE* VOS† LogitNorm‡ - Adversarial ACET* ATOM* - Guaranteed GOOD*80 ProoD* Δ = 3 DISTRO (our) ID: CIFAR100 Acc.
methods using a ResNet18 [26] architecture for CIFAR10 It is evident that the ℓ2-norm GAUC (and GAUPR)
diand CIFAR100 respectively. For methods that require verge from zero when standard OOD detection models
an additional OOD dataset for training, such as OE [
For Plain, OE and LogitNorm we run the implemen- appears to be slightly better, even though the denoistation8 from Yang et al. [36] and leave the hyperpa- ing process should produce images that are as similar rameters unchanged. Similarly for ACET and ATOM, as possible to those considered during training. This is we only change the model architecture and normaliza- because a single shot of denoising does not compromise tion and run both implementations from ATOM9. Lastly, the OOD sample or generate an allucinated one. Addiwe train ProoD10 from Meinke et al. [14] using their tionally, one-shot denoising introduces so little variance training configuration files, where the discriminator is that in this benchmark, the results were similar across trained for 1000 epochs and the bias shift (∆ ) is 3/1 for multiple runs.
Plain LogitNorm OE ACET ATOM ProoD DISTRO (our) Plain LogitNorm OE ACET ProoD DISTRO (our)
Acc.
We propose an alternative using randomized
smoothing [
ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, [24] K. Hung, W. Fithian, Rank verification for expoOpenReview.net, 2020. nential families, The Annals of Statistics 47 (2019) [12] J. Chen, Y. Li, X. Wu, Y. Liang, S. Jha, Atom: Robus- 758–782.
tifying out-of-distribution detection using outlier [25] N. Jovanovic, M. Balunovic, M. Baader, M. T. Vechev, mining, in: Joint European Conference on Machine Certified defenses: Why tighter relaxations may Learning and Knowledge Discovery in Databases, hurt training?, CoRR abs/2102.06700 (2021).
Springer, 2021, pp. 430–445. [26] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learn[13] J. Bitterwolf, A. Meinke, M. Hein, Certifiably adver- ing for image recognition, in: Proceedings of the sarially robust detection of out-of-distribution data, IEEE conference on computer vision and pattern in: Advances in Neural Information Processing Sys- recognition, 2016, pp. 770–778. tems 33: Annual Conference on Neural Information [27] G. Huang, Z. Liu, L. Van Der Maaten, K. Q. WeinProcessing Systems 2020, NeurIPS 2020, December berger, Densely connected convolutional networks, 6-12, 2020, virtual, 2020. in: Proceedings of the IEEE conference on computer [14] A. Meinke, J. Bitterwolf, M. Hein, Provably robust vision and pattern recognition, 2017, pp. 4700–4708. detection of out-of-distribution data (almost) for [28] S. Zagoruyko, N. Komodakis, Wide residual netfree, in: NeurIPS, 2022. works, in: British Machine Vision Conference 2016, [15] M. Mirman, T. Gehr, M. Vechev, Diferentiable ab- British Machine Vision Association, 2016. stract interpretation for provably robust neural net- [29] A. Krizhevsky, V. Nair, G. Hinton, Cifar-10 and works, in: ICML, PMLR, 2018, pp. 3578–3586. cifar-100 datasets, URl: https://www. cs. toronto. [16] A. Q. Nichol, P. Dhariwal, Improved denoising edu/kriz/cifar. html 6 (2009) 1. difusion probabilistic models, in: International [30] P. Sermanet, S. Chintala, Y. LeCun, ConvoluConference on Machine Learning, PMLR, 2021, pp. tional neural networks applied to house numbers 8162–8171. digit classification, in: Proceedings of the 21st [17] N. Carlini, F. Tramer, K. D. Dvijotham, L. Rice, international conference on pattern recognition M. Sun, J. Z. Kolter, (certified!!) adversarial ro- (ICPR2012), 2012, pp. 3288–3291. bustness for free!, in: The Eleventh International [31] F. Yu, A. Sef, Y. Zhang, S. Song, T. Funkhouser, Conference on Learning Representations, 2023. J. Xiao, Lsun: Construction of a large-scale image [18] G. Yang, T. Duan, J. E. Hu, H. Salman, I. Razen- dataset using deep learning with humans in the shteyn, J. Li, Randomized smoothing of all shapes loop, arXiv preprint arXiv:1506.03365 (2015). and sizes, in: ICML, PMLR, 2020, pp. 10693–10705. [32] Y. Le, X. Yang, Tiny imagenet visual recognition [19] H. Salman, J. Li, I. Razenshteyn, P. Zhang, H. Zhang, challenge, CS 231N 7 (2015) 3.
S. Bubeck, G. Yang, Provably robust deep learn- [33] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , ing via adversarially trained smoothed classifiers, A. Vedaldi, Describing textures in the wild, in: in: H. Wallach, H. Larochelle, A. Beygelzimer, Proceedings of the IEEE Conf. on Computer Vision F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances and Pattern Recognition (CVPR), 2014. in Neural Information Processing Systems, vol- [34] L. D. Brown, T. T. Cai, A. DasGupta, Interval Estimaume 32, Curran Associates, Inc., 2019. tion for a Binomial Proportion, Statistical Science [20] C. M. Stein, Estimation of the mean of a multivariate 16 (2001) 101 – 133.
normal distribution, The annals of Statistics (1981) [35] A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, 1135–1151. I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Mal[21] H. Salman, M. Sun, G. Yang, A. Kapoor, J. Z. Kolter, loci, A. Kolesnikov, et al., The open images dataset Denoised smoothing: A provable defense for pre- v4: Unified image classification, object detection, trained classifiers, Advances in Neural Information and visual relationship detection at scale, InternaProcessing Systems 33 (2020) 21945–21957. tional Journal of Computer Vision 128 (2020) 1956– [22] W. Liu, X. Wang, J. Owens, Y. Li, Energy-based 1981.
out-of-distribution detection, Advances in Neural [36] J. Yang, P. Wang, D. Zou, Z. Zhou, K. Ding, W. PENG, Information Processing Systems 33 (2020) 21464– H. Wang, G. Chen, B. Li, Y. Sun, X. Du, K. Zhou, 21475. W. Zhang, D. Hendrycks, Y. Li, Z. Liu, OpenOOD: [23] F. Croce, M. Hein, Reliable evaluation of adversarial Benchmarking generalized out-of-distribution derobustness with an ensemble of diverse parameter- tection, in: Thirty-sixth Conference on Neural Infree attacks, in: H. D. III, A. Singh (Eds.), Pro- formation Processing Systems Datasets and Benchceedings of the 37th International Conference on marks Track, 2022.
Machine Learning Research, PMLR, 2020, pp. 2206–