A measure of robustness against naturally occurring distortions is key to the safety, success, and trustworthiness of machine learning models on deployment. We investigate an adversarial black-box attack that adds minimum Gaussian noise distortions to input images to make deep learning models misclassify. We used a Reinforcement Learning (RL) agent as a smart hacker to explore the input images to add minimum distortions to the most sensitive regions to induce misclassification. The agent employs a smart policy also to remove noises introduced earlier, which has less impact on the trained model at a given state. This novel approach is equivalent to doing a deep tree search to add noises without an exhaustive search, leading to faster and optimal convergence. Also, this adversarial attack method efectively measures the robustness of image classification models with the misclassification inducing minimum 2 distortion of Gaussian noise similar to many naturally occurring distortions. Furthermore, the proposed black-box 2 adversarial attack tool beats state-of-the-art competitors in terms of the average number of queries by a significant margin with a 100% success rate while maintaining a very competitive 2 score, despite limiting distortions to Gaussian noise. For the ImageNet dataset, the average number of queries achieved by the proposed method for ResNet-50, Inception-V3, and VGG-16 models are 42%, 32%, and 31% better than the state-of-the-art "Square-Attack" approach while maintaining a competitive 2. Demo: https://tinyurl.com/2p8pnjn6 Figure 1: An example of adversarial perturbations driven by the learnt policy of RLAB agent. The image "" classified as Panda, an adversarial sample generated with RLAB (ours) " + " has been classified as dolphin where represents the noise added to the image.
Deep learning models have yielded impressive results
in numerous applications, but research on adversarial
attacks has shown that these models sufer from a
vulnerability where small distortions could lead to wrong
predictions. Specifically, naturally occurring distortions that
afect the inputs are of greater concern in safety-critical
applications such as self-driving cars, facial recognition,
and image-based authorization [
Literature has provided us with two major paths to identify the sensitivity of the deep learning models, White box attacks [3][4] and Black box attacks [5][6]. Even though recent works have introduced eficient white-box approaches targeting a specific region or very minimum distortion to fool the Convolutional Neural Network (CNN) models, it requires complete visibility of the network architecture and the parameters. In general, and with a 100% success rate while maintaining other metrics like distortion at a minimum. The motivation for using RL is to learn an optimum policy that incrementally adds noise to deceive a model, unlike the hand-crafted heuristics that are used in State-of-the-art adversarial attacks. Our method includes a dual action RL agent, which makes parallel addition and removal of distortions to image regions, based on the image region sensitivity at the current state and the history of progression of added distortion as shown in Figure 1. The goal is to cause a misclassification with minimum number of queries In an extensive evaluation of un-targeted attacks with ImageNet and CIFAR-10 datasets on CNN architectures such as ResNet-50, Inception-V3, and VGG-16, RLAB outperforms the state-of-the-art methods for 2 threat model on the number of queries while achieving competitive 2 norm as shown in figure 2. The main contribution of the work can be summarized as follows.
Some of the established metrics to evaluate the performance of a machine learning model include accuracy, precision, recall, and F1 score. With the recent advances in adversarial attacks, the models that showed excellent performance on static test sets with the above metrics were easily misclassified with adversarial examples. For example, work done by Szegedy et al. [3] was one of the ifrst works to introduce adversarial attacks. White-box attacks showed great results with one of the initial works from Goodfellow et al. in their work [4] introducing Fast Gradient Sign Method (FGSM) based attack where a small vector whose elements are equal to the sign of the elements of the gradient of the cost function with respect to the input changed the classification outcomes. Following this work, there were other incremental works based on gradients-based distortion that could flip the model [7][8][9]. Papernot et al. [10] generated an indication map representing the right area on the input that can be attacked. Similarly, DeepFool by Moosavi et al. [11] proposed a simple yet efective approach to add perturbations to the input to fool the machine learning models.
visibility into the model. In a partially visible black-box attack, information about the loss function, the prediction probabilities, or top-K sorted labels could be available based on which the attack is executed in a query access approach. Work done by Michel et al. [12] and Chakraborty et al. [13] provides a detailed survey on the current trends in adversarial attacks on neural networks.
Further, Ilyas et al. [14] in their early work approached 1. A novel Reinforcement Learning agent, that beats this problem with multiple level of restrictions including the state-of-the-art un-targeted black-box 2 at- limited visibility, limited query access and so on. Some tack models in terms of an average number of of the most popular black-box attack in recent times queries by a wide margin with a 100% success that has been acknowledged by the research community rate while keeping the 2-norm minimum. include Square attack [5], SimBA [15], and LeBA [16], 2. This RL approach learns a policy to form an op- which achieved significant results in breaking Convolutimum adversarial attack agent that can outper- tional Neural Network based models. Guo et al. [15] in form the engineered heuristic approach of the their work proposed a simple approach where they iteraprevailing SOTA adversarial attacks by the above tively and randomly sample a vector from a predefined metrics. orthonormal basis such that it can be added or subtracted 3. A high-performance adversarial attack agent that from the target image. Similarly, Andriushckenko et al. limits the distortions to Gaussian noise, which [5] proposed an approach where square-shaped updates is one of the naturally occurring real-life non- are added at random positions such that at each iteration, malicious distortions, unlike most adversarial at- the total budget constraint is still preserved. Furthermore, tacks. some of the most recent works in the black-box attack include EigenBA [17], Pixle [18], Querynet [19], advFlow [20], and CG attack [21] producing state-of-the-art results.
Reward ΔProb Dilution for GT
/ ΔL2-Distance
Altered vs Previous image Adds Distortion Removes Distortion
CNN Classifier *Image Cleanup
Adversarial
Sample Misclassified * Takes out distortions, without flipping classification back to Ground Truth
Distorted Image Probability of classes, L2 Distortion - Last N
Reinforcement Learning has solved problems that clas- patches and sensitivity of the ground truth probability
sic machine learning struggles in various domains and , to addition and removal of distortion, is computed
applications such as healthcare, energy [22, 23], medical for each patch. Based on the sensitivity information,
imaging, etc. Their unique ability to learn a policy for the RL decides the patches to which Gaussian noise is
action is a key attribute of their success. Reinforcement added or removed at every step. This process is done
learning for adversarial attacks has not been explored iteratively until the model misclassifies the image. To
furmuch. Sun et al. [24] in their work use reinforcement ther reduce 2, we perform an iterative image cleanup
learning to target graph neural networks via node in- as a post-processing step while maintaining the
misclasjections. Similarly, work done by Yang et al. [16](Patch sification. The overall flow of the proposed method is
Attack) applies reinforcement learning to attack CNN represented in the figure 3.
models by superimposing textured patches on the input
image. Unlike the previous approach, our RL agent uses 3.3. Image Sensitivity Analysis
a comprehensive state representation that captures the
model’s sensitivity to various image regions and imple- In our proposed approach, we limit all distortions to
ments a patch-based process with natural distortions. Gaussian noise, as it is a commonly encountered and
natThis enables our approach to significantly outperform urally occurring distortion. During the image sensitivity
state-of-the-art adversarial attacks, including RL-based analysis, we generate a fixed number of noise masks of
methods in terms of minimum distortion measured by same noise level, of size × sampled from a normal
L2-norm, query eficiency, and success rate. distribution as represented in the equation 1.
(× ) = (0, _)
(1)
3. Proposed Method At every step during the training and validation, one
mask is randomly chosen from the generated noise masks
3.1. Reinforcement Learning/problem and applied across all image patches to evaluate the drift
formulation in the ground truth classification probability . A
The Deep Neural Network (DNN) model under test/e- lower noise level is chosen as it helps more granular
valuation can be represented as = (; ), where addition of noise in successive steps to specific regions
denotes the input image, represents the prediction and that create maximum drift with the , while keeping
represents the model parameters. The motivation is to 2 minimum. The noise mask is generated such that
generate a perturbation such that, ̸= ( + ; ). The they have the same efect on change in 2 distance. The
objective is to minimize which represents a measure of perturbations ˆ − are constrained to the values [
Generating adversarial examples for image classification through multiple steps is similar to board games. For board games, the most efective moves or actions are figured out through a Deep Tree Search (DTS) of multiple layers to determine the efectiveness of an action taken at the current step on a longer time horizon as the game evolves. DTS is computationally expensive, even with approximations like Monte Carlo Tree Search (MCTS). But unlike a board game, in this problem, there is a possibility to reset the earlier moves when we realize that we have made a less optimized move a few steps back. In RLAB this is done by removing distortions from some patches and adding distortions to some other patches, considering the state of the modified image at any given step (equivalent to position on the board). This is equivalent to replaying all the moves in one step while keeping the sensitivity analysis restricted to the current state of the image without a tree search.
Our method reduces the complexity from ( ) to ( ) where N represents the computation complexity of one level of evaluation and corresponds to the image size, and d represents the depth of the tree search, which translates to how many queries and actions we would like to look ahead if we were doing a tree search. d=[1, max_steps].
RL Test Agent Agent for Image Classifier Test (Dueling DQN)
States Sensitivity of classification probability to Gaussian noise for various image tiles Probability of various classes L2 distance for last N steps
Action Distortion and Restoration mask for adding and / or removing noise to image tiles
Reward Calssification probability change for Ground Truth normalized by change in L2 distance of the image Environment Image Classifer Model Analytics
Image Transformations
we define RL action as the number , where RLAB adds distortion to the top ( + 1) patches from the in the state and removes distortion from the top patches from as represented in Figure 4. ∈ [1, ] where 3.5. Reinforcement Learning is a hyperparameter and is set to 8 for ImageNet The decision of which patches to choose for adding or re- (224 × 224) image size with 2 × 2 patch size), to balmoving distortion has multiple dependencies and needs ance efectiveness and computation. Note that the net to be adaptive for the most eficient generation of adver- diference is one square patch where distortion is added, sarial examples. Mapping this adversarial sample genera- keeping the change in the 2 distance approximately tion as a Reinforcement Learning (RL) problem requires bound to what we would have got if we had added disdefining the states, actions, and rewards. The state-space tortion to just one patch. However, there is a possibility is constructed such that the environment becomes ob- that the patch where we are removing the distortion may servable in a way it enables the RL agent to learn the have distortion added to it multiple times, which will optimum policy to take actions while maximizing the only lower the net increase of 2 distance. reward. We used the Dueling DQN Reinforcement Learn- 3.5.3. RL Reward ing (RL) based agent in RLAB. Algorithm 1 represents the overall flow of the proposed method. Figure 6 represents the steps involved in adding and removing distortion by the RL agent. 3.5.1. RL States We define a probability dilution (PD) metric, which measures the extent to which the classification probability shifts from the ground truth to the other classes. The diference between the PD of the altered image and the original image as a result of an action (∆ PD) is a measure of the efectiveness of the action. Moreover, the change in 2-distance (∆ 2) as a measure of the distortion added is the cost for an action. The reward is defined by the normalized PD as represented in equation 2.
We designed a state space that gives required observability to the RL agent but is simple enough and of lower dimension such that the agent could be trained eficiently as shown in Figure 5. The image sensitivity analysis acts as a feature extractor where the the top ordered square patch locations are ordered both based on the change in for adding and removing distortion in the state vector. Also included are the classification probabilities and 2 distance progression. = ∆ = − ∆ /∆ 2
(2)
= − ∆ 2/∆ (3)
We developed a Dueling DQN algorithm-based Reinforce- Most state-of-the-art competitive solutions use unnatural ment Learning (RL) agent for RLAB as an adversarial modifications as shown in Figure 7. The only other RL attack agent [6, 7], which also evaluates the CNN image method used for a similar adversarial attack, Patch Atclassification models for robustness, as shown in Figure 8. tack, has completely unnatural squared patches placed The Dueling DQN algorithm splits the Q-values into two on the images. In contrast, our proposed method preparts: the value function V(s) and the advantage function serves the true nature of the image with barely percepA(s, a). As shown in Figure 8, the same neural network tible Gaussian noise. Moreover, Patch Attack’s distorsplits its last layer into two parts, one of them to estimate tion measured in 2-norm of 191 is significantly higher the state value function for states (V(s)) and the other one than our RLAB’s 2-norm of 4.03 for ImageNet. Also, to estimate state-dependent action advantage (A (s, a)). It as shown in Figure 7, the state-of-the-art high-eficiency then combines both parts into a single output, estimating Square Attack has unnatural colors of red and green all the Q-values. This change is helpful because sometimes over the cougar, unlike our RLAB method. In RLAB, any it is unnecessary to know the exact value of each action. distortion is barely perceptible because of low Gaussian So just learning the state-vlue function can be enough noise. in some cases. The main benefit is generalizing learning across actions without imposing specific changes to 4. Experiments the underlying reinforcement learning algorithm. The Dueling DQN model fits well with the discrete action space of a limited number of and has the suitable complexity to predict efectively with a reasonably bounded training.
In this section, we discuss the efectiveness of our proposed method with the same experimental setup as our competitors. We evaluate on two popular image classification datasets ILSVRC2012 [ 31] and CIFAR-10. 80 percent of the validation set was used to train our RL 3.6. Post-processing noise removal agents, and 20 percent of the validation set was used for evaluation. We performed our attacks on three maOnce an adversarial sample is generated using RL fol- jor Convolution-based Neural Network architectures: lowing the initial misclassification we perform a final ResNet, Inception-V3, and VGG-16. We used three metnoise removal process. This iterative process maintains rics to evaluate the performance of our approach. 2 the misclassification while attempting to remove noise distance which is a measure of distortion, the average Original Image
Queries : 22500 L2-Norm: 191 Square Attack
Square Attack
Difference
Queries : 180 L2-Norm: 4.87
RLAB (ours)
RLAB (ours)
Difference number of queries to make a model miss-classify a correctly classified sample, and the average success rate.
For validation, we had an overall average 2 of 4.03 with the values of pixels ranging between 0 and 1 and setting a maximum query budget of 3500 evaluated over 1000 samples from imagenet dataset on ResNet-50 architecture. A failure case is when the proposed method could not fool the victim model within the given budget, and failure cases were not included in any of the metrics calculated except for the success rate. All experiments were performed for a patch size of 2 × 2 and with the noise level of 0.005 as we got the best results for this configuration. pipeline.
The computation for the complete pipeline is GPU- It is worth mentioning that the proposed robustdependent and is eficiently batched, and scaled on GPUs. ness measure in Deep Fool [4] involves minimizing Caching techniques were used for pre-computed infor- the amount of distortion needed for misclassification, mation such as the noise masks for improved eficiency. which is defined by ∆( ; ˆ) := min ‖‖2 subject to Apollo servers with 8 × 100 32 GB GPUs were used ˆ( + ) ̸= ˆ(), where min ‖‖2 = min 2 and for training and validation. We processed 16(images per ∆( ; ˆ) is the robustness of classifier ˆ for input x. As GPU) x 8(GPUs) = 128 images in a batch for the complete we can see, this is consistent with our goal, which is Attack Q-Fool [26] NES (2018) [14] (2018) [27] HopSkipJumpAttack [28] Subspace(2019) [29] P-RGF (2019) [30] LeBA (2020) [16] Square (2020) [5] SimBA-DCT (2021) [15] querynet (2021) [19] AdvFlow (2021) [20] EigenBA (2022) [17] Pixle (2022) [18] CG-Attack (2022)[21] Patch Attack (2022) [16] RLAB (ours) 4.1. Evaluation on Imagenet Both approaches were evaluated on the ResNet-50 model. It can be observed that the success rate of our proposed Table 1 aggregates the proposed method’s results com- method is the same as the competitors which is 100 pared to other state-of-the-art black-box algorithms on percent while the average queries of the proposed apImagenet dataset for ResNet-50 architecture. The com- proach outperform every state-of-the-art technique. Expetitors’ results were generated with the best parame- cept EigenBA [17] and CG-Attack[21] which are close ters described in their papers. The average Success Rate to our results, our approach beats the competitors by a (ASR) and Average Query (AVG.Q) were calculated for large margin. each victim model while the average L2 for most of the competitors were presented in their paper. It can be observed that our proposed approach beats state-of-the-art 4.3. Ablation study on diferent noise algorithms for average queries and success rate by a sig- intensities nificant margin while maintaining competitive 2. It is also worth mentioning that the proposed approach was One of the key hyperparameters for RLAB is the quantity able to achieve 100% success rate for a maximum query of Gaussian noise that it adds for distortion. Experiments set to 3500 while the competitors have experiments per- showed that higher noise levels increased the final 2 of formed with a maximum query set to 10000. Similarly, the adversarial sample, while too less of noise impacted from table 2 our proposed approach outperforms com- the average number of queries. We performed an evalpetitors for Inception-v3 for average number of queries uation of diferent noise levels and their impact on the while maintaining competitive queries for VGG-16. Fur- metrics as represented in table 4. We applied the same thermore, we have achieved a 100 % success rate for both noise level for evaluation on both datasets (ImageNet, Inception-v3 and VGG-16 models. Figure 9 shows the CIFAR-10) and all three victim models. We observed that comparison of RLAB with the competition RL method the chosen noise level gave the best results across all and Square attack. datasets and victim models.
Queries : 1500 L2-Norm : 119
Queries : 44 L2-Norm: 5
Queries : 111 L2-Norm: 2.94 and remove (N) at each step. This decision is based on a learned policy (Dynamic) or based on a tuned hyperparameter baseline. From the table 5, it can be observed that the results improved when the RL learned policy made the decision than the baseline for the number of patches to which noise was added and removed
In our proposed work, we generated all our results with the patch of size 2 × 2 for best results. It can be observed from table 6 that as the patch size increases, the number of queries decreases while the 2 increases. Furthermore, we observed similar pattern with other models such as VGG-16 and Inception-V3. This could be due to that fact that the computation for sensitivity analysis primarily depends on the size of the image and can be accelerated, scaled, and batched on the GPU, there is a smaller variation based on the number of patches due to the post-processing overhead which is typically around 20% of the total computation for 224x224 images with 2 × 2 patch sizes. Depending on the use case, our approach allows using diferent patch sizes at diferent levels of performance which are represented in table 6.
tacks in query eficiency by a significant margin and achieves a highly competitive 2-norm indicative of Patch Size 2x2 4x4 8x8 16x16 32x32 very low distortion with 100% success rate for missclassification. But as RLAB only uses Gaussian noise, the distortions are similar to real-life deployment. This makes it valuable for a more appropriate test for nonmalicious distortions and an efective measure of robustness, which is a key attribute of trustworthiness with a positive social impact.
Also, Reinforcement Learning proved to be very efective in learning the optimum policy to make the complex decision of choosing the square patches for changing distortion and making RLAB adaptive, as compared to hand-crafted heuristics. This is by far the best RL implementation of this type of Black-Box adversarial attack considering both the results achieved and the flexible nature of the optimization approach. This RL design will be extended to include other types of distortions as part of future work. Also, this RL approach is generic enough to extend to a wide variety of adversarial attack agents beyond image classifiers.
The adversarial samples generated by RLAB can be used to augment the train data set to retrain the model and enhance its robustness. itations for securing facial authentication, in: Pro- [15] C. Guo, J. Gardner, Y. You, A. G. Wilson, K. Weinceedings of the 2022 ACM on International Work- berger, Simple black-box adversarial attacks, in: shop on Security and Privacy Analytics, 2022, pp. International Conference on Machine Learning, 2–12. PMLR, 2019, pp. 2484–2493. [2] M. Ozdag, S. Raj, S. L. Fernandes, A. Velasquez, [16] J. Yang, Y. Jiang, X. Huang, B. Ni, C. Zhao, Learning L. Pullum, S. K. Jha, On the susceptibility of black-box attackers with transferable priors and deep neural networks to natural perturbations, in: query feedback, Advances in Neural Information AISafety@ IJCAI, 2019. Processing Systems 33 (2020) 12288–12299. [3] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Er- [17] L. Zhou, P. Cui, X. Zhang, Y. Jiang, S. Yang, Adverhan, I. Goodfellow, R. Fergus, Intriguing properties sarial eigen attack on black-box models, in: Proof neural networks, arXiv preprint arXiv:1312.6199 ceedings of the IEEE/CVF Conference on Computer (2013). Vision and Pattern Recognition, 2022, pp. 15254– [4] I. J. Goodfellow, J. Shlens, C. Szegedy, Explain- 15262.
ing and harnessing adversarial examples, arXiv [18] J. Pomponi, S. Scardapane, A. Uncini, Pixle: a fast preprint arXiv:1412.6572 (2014). and efective black-box attack based on rearranging [5] M. Andriushchenko, F. Croce, N. Flammarion, pixels, arXiv preprint arXiv:2202.02236 (2022).
M. Hein, Square attack: a query-eficient black-box [19] S. Chen, Z. Huang, Q. Tao, X. Huang, Querynet: adversarial attack via random search, in: European Attack by multi-identity surrogates, arXiv e-prints Conference on Computer Vision, Springer, 2020, pp. (2021) arXiv–2105.
484–501. [20] H. Mohaghegh Dolatabadi, S. Erfani, C. Leckie, Ad[6] J. Su, D. V. Vargas, K. Sakurai, One pixel attack for vflow: Inconspicuous black-box adversarial attacks fooling deep neural networks, IEEE Transactions using normalizing flows, Advances in Neural Inforon Evolutionary Computation 23 (2019) 828–841. mation Processing Systems 33 (2020) 15871–15884. [7] A. Kurakin, I. Goodfellow, S. Bengio, et al., Adver- [21] Y. Feng, B. Wu, Y. Fan, L. Liu, Z. Li, S.-T. Xia, Boostsarial examples in the physical world, 2016. ing black-box attack with partially transferred con[8] A. Kurakin, I. Goodfellow, S. Bengio, Adversar- ditional adversarial distribution, in: Proceedings of ial machine learning at scale, arXiv preprint the IEEE/CVF Conference on Computer Vision and arXiv:1611.01236 (2016). Pattern Recognition, 2022, pp. 15095–15104. [9] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, J. Li, [22] S. Sarkar, V. Gundecha, S. Ghorbanpour, Boosting adversarial attacks with momentum, in: A. Shmakov, A. R. Babu, A. Pichard, M. CoProceedings of the IEEE conference on computer cho, Skip training for multi-agent reinforcement vision and pattern recognition, 2018, pp. 9185–9193. learning controller for industrial wave energy [10] N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. converters, in: 2022 IEEE 18th International ConCelik, A. Swami, The limitations of deep learning in ference on Automation Science and Engineering adversarial settings, in: 2016 IEEE European sym- (CASE), IEEE, 2022, pp. 212–219. posium on security and privacy (EuroS&P), IEEE, [23] S. Sarkar, V. Gundecha, A. Shmakov, S. Ghor2016, pp. 372–387. banpour, A. R. Babu, P. Faraboschi, M. Cocho, [11] S.-M. Moosavi-Dezfooli, A. Fawzi, P. Frossard, A. Pichard, J. Fievez, Multi-agent reinforcement Deepfool: a simple and accurate method to fool learning controller to maximize energy eficiency deep neural networks, in: Proceedings of the IEEE for multi-generator industrial wave energy conconference on computer vision and pattern recog- verter, in: Proceedings of the AAAI Conference on nition, 2016, pp. 2574–2582. Artificial Intelligence, volume 36, 2022, pp. 12135– [12] A. Michel, S. K. Jha, R. Ewetz, A survey on the 12144.
vulnerability of deep neural networks against ad- [24] Y. Sun, S. Wang, X. Tang, T.-Y. Hsieh, V. Honavar, versarial attacks, Progress in Artificial Intelligence Adversarial attacks on graph neural networks via (2022) 1–11. node injections: A hierarchical reinforcement learn[13] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, ing approach, in: Proceedings of the Web ConferD. Mukhopadhyay, Adversarial attacks and de- ence 2020, 2020, pp. 673–683. fences: A survey, arXiv preprint arXiv:1810.00069 [25] M. Sewak, Deep q network (dqn), double dqn, and (2018). dueling dqn, in: Deep Reinforcement Learning, [14] A. Ilyas, L. Engstrom, A. Athalye, J. Lin, Black-box Springer, 2019, pp. 95–108.
adversarial attacks with limited queries and infor- [26] J. Chen, M. I. Jordan, M. J. Wainwright, Hopmation, in: International Conference on Machine skipjumpattack: A query-eficient decision-based Learning, PMLR, 2018, pp. 2137–2146. attack, in: 2020 ieee symposium on security and privacy (sp), IEEE, 2020, pp. 1277–1294. [27] A. Ilyas, L. Engstrom, A. Madry, Prior convictions: systems 32 (2019).
Black-box adversarial attacks with bandits and pri- [31] O. Russakovsky, J. Deng, H. Su, J. Krause, ors, arXiv preprint arXiv:1807.07978 (2018). S. Satheesh, S. Ma, Z. Huang, A. Karpathy, [28] A. Rahmati, S.-M. Moosavi-Dezfooli, P. Frossard, A. Khosla, M. Bernstein, et al., Imagenet large scale H. Dai, Geoda: a geometric framework for black- visual recognition challenge, International journal box adversarial attacks, in: Proceedings of the of computer vision 115 (2015) 211–252. IEEE/CVF Conference on Computer Vision and Pat- [32] Y. Dong, T. Pang, H. Su, J. Zhu, Evading defenses tern Recognition, 2020, pp. 8446–8455. to transferable adversarial examples by translation[29] Y. Guo, Z. Yan, C. Zhang, Subspace attack: Ex- invariant attacks, in: Proceedings of the IEEE/CVF ploiting promising subspaces for query-eficient Conference on Computer Vision and Pattern Recogblack-box attacks, Advances in Neural Information nition, 2019, pp. 4312–4321.
Processing Systems 32 (2019). [33] J. Du, H. Zhang, J. T. Zhou, Y. Yang, J. Feng, Query[30] S. Cheng, Y. Dong, T. Pang, H. Su, J. Zhu, Improving eficient meta attack to deep neural networks, arXiv black-box adversarial attacks with a transfer-based preprint arXiv:1906.02398 (2019). prior, Advances in neural information processing