=Paper=
{{Paper
|id=Vol-2881/paper8
|storemode=property
|title=Sparse Adversarial Attack to Object Detection
|pdfUrl=https://ceur-ws.org/Vol-2881/paper8.pdf
|volume=Vol-2881
|authors=Jiayu Bao
}}
==Sparse Adversarial Attack to Object Detection==
https://ceur-ws.org/Vol-2881/paper8.pdf
Sparse Adversarial Attack to Object Detection
Jiayu Bao
Department of Electronic Engineering, Tsinghua University
bjy19@mails.tsinghua.edu.cn
ABSTRACT
Adversarial examples have gained tons of attention in recent years.
Many adversarial attacks have been proposed to attack image clas-
sifiers, but few work shift attention to object detectors. In this
paper, we propose Sparse Adversarial Attack (SAA) which enables
adversaries to perform effective evasion attack on detectors with
bounded l0 norm perturbation. We select the fragile position of the
image and designed evasion loss function for the task. Experiment
results on YOLOv4 and FasterRCNN reveal the effectiveness of our
method. In addition, our SAA shows great transferability across dif-
ferent detectors in the black-box attack setting. Codes are available Figure 1: Framework of sparse adversarial attack.
at https://github.com/THUrssq/Tianchi04.
Experiment results on two state-of-the-art white-box detectors
KEYWORDS and two unknown black-box detectors show that our SAA is a pixel
adversarial example; detector; evasion attack; sparse patch efficient adversarial attack with good transferability. Our method
outperforms many other methods in the same task and get 4th in
1 INTRODUCTION AIC Phase IVCIKM-2020: Adversarial Challenge on Object Detection
Deep neural networks have achieved remarkable success in com- competition. The contributions of this work are as follows:
puter vision tasks like image classification, object detection and β’ We propose sparse adversarial attack (SAA) method to per-
instance segmentation. Deep object detectors can be divided into form evasion attack on object detectors.
one-stage detectors and two-stage detectors. One-stage detectors β’ We design powerful evasion loss functions which can per-
like YOLOv4 [1] and SSD [7] take classification and regression as a form pixel efficient adversarial attacks on both one-stage
single step. Hence they are usually faster than two-stage detectors. and two-stage object detectors.
While two-stage detectors with region proposal process often have β’ Our method can be easily extended to multi-models. We en-
better performance on accuracy than one-stage detectors. semble two off-the-shelf detectors and achieve considerable
However, the finding that deep models are vulnerable to ad- transferability in black-box attack settings.
versarial examples [11] poses great concerns for the security of
deep models. In computer vision field, adversarial examples are 2 METHOD
perturbed images which are designed purposely to fool the deep In this section, we explain our SAA method, which applying sparse
neural models. Many adversarial attack methods have been pro- adversarial patch on images to blind the detectors. The figure1 is an
posed to study the robustness of deep models but most of them [9] illustration of our SAA framework which ensemble multiple detec-
[3] [2] focus on image classifiers rather than the more widely used tors to perform the evasion attack. To perform powerful adversarial
object detectors. We believe adversarial attacks play a significant attacks using limited number of pixels, we purposely design the
role in improving the robustness of deep neural models. Existing position, shape and size of adversarial patch. For one-stage detector
adversarial attacks usually change the category classification of an (YOLOv4 [1]) and two-stage detector (FasterRCNN [10]), we design
image or an object [12] [5], which is reasonable for classifiers but evasion loss function respectively to enhance our attack power. The
deficient for detectors. rest of this section is organized as follows: section 2.1 illustrate our
Due to the complexity of object detection task, adversarial attacks patch design, section 2.2 expound the loss function in SAA and
on object detectors can be more diversified than that of classifiers. section 2.3 shed light on optimizition details.
In this paper, we propose a pixel efficient evasion attack method
SAA for object detectors. Concretely, the perturbation generated by 2.1 Patch Design
our SAA can make all the objects in the image to evade detections
Our SAA can generate sparse adversarial noise. Concretely, we
of target detectors. We design different loss functions according to
apply perturbation with bounded π 0 norm (no more than 2% of the
the model we attack and choose pivotal positions in the image to
image size) on the image. The perturbation is sparse in space but is
make our adversarial perturbation sparse yet powerful.
efficient in fooling the detectors due to our purposely design of its
Copyright Β© 2020 for this paper by its authors. Use permitted under Creative Commons spatial distribution.
License Attribution 4.0 International (CC BY 4.0). In the training phase of detectors, the center of an object plays a
In: Dimitar Dimitrov, Xiaofei Zhu (eds.): Proceedings of the CIKM AnalytiCup 2020, 22
October, 2020, Gawlay (Virtual Event), Ireland, 2020, published at http://ceur-ws.org. vital role in the detection of this object. Take the one-stage detector
YOLOv4 [1] as an illustration, which will allocate anchor box that
28
resemble ground truth mostly for an object. Even for two-stage de- FasterRCNN [10] is a typical two-stage object detector that firstly
tectors and anchor-free detectors, object centerness directly relates extracts bounding box proposals before the subsequent classifica-
to the computation of intersection over union (IOU). Boxes that tion and regression processes. The definition of positive and neg-
deviate object centerness too much are more likely to be erased ative samples is more complex in FasterRCNN [10] than that of
during the non-maximum suppression (NMS) process. YOLOv4 [1]. However, we merely consider the inference phase of
Traditional adversarial patches with square or circle shape [2] FasterRCNN [10] to simplify our attack. That is, we increase the
[8] are too concentrated in a local area of the image. They have softmax output probabililty of background while decrease that of
great physical realizability but are pixel inefficient in this task. We any other categories (foreground objects). However, FasterRCNN
argue that a highly centralized structure limits the success rate of [10] typically generates more than one hundred thousand region
attacks especially when the π 0 norm constraint is too strong. proposals by RPN, which is more than ten times that of YOLOv4
In view of those cases, we design a kind of cruciform patch [1]. The enormous number of region proposals makes it hard to
with itβs intersection locates at the centerness of object bounding erase all the objects with an extremely limited attack budget. So we
box. Exploiting the long-span patch, we can mislead the results of make some compromises and design the total evasion loss function
detectors severely with very few pixels changed. of FasterRCNN [10] as follows
Due to the difficulty of π 0 norm constrained optimization problem,
we exploit a mask with values of 0 and 1 to decide the location of
the patch, also to limit π 0 norm of the perturbation. We design the πΏππ π πΉ π
πΆπ π = πΌ 1 Β· πΏππ π 1 + πΌ 2 Β· πΏππ π 2 (3)
mask empirically based on the assumption that object centerness is
a vulnerable area against adversarial attacks, combined with other where πΌ 1 and πΌ 2 are hyperparameters, and we have
conditions of the patch (shape, size) mentioned before. Hence the
input of the detector is formulated as follows:
πΏππ π 1 = max π (π, π) (4)
π βπΆ,π βπ΅
πΌππ = πΌ β (1 β π) + π β π (1) 1 Γ
πΏππ π 2 = max π (π, π) (5)
π π βπΆ
Here πΌ denotes clean image, π denotes adversarial patch, π is the π βπ΅
mask we design purposely and β denotes element-wise product where πΆ denotes the set of all object categories, π΅ denotes the set of
in this paper. As illustrated in figure 1, we include preprocesses of all bounding boxes predict by FasterRCNN [10] and π is the number
detectors into the forward phase of our attack in order to optimize of elements of set π΅. We design πΏππ π 1 to attack the bounding box
the patch directly with gradient. with highest object probability, which is hard in experiments, hence
πΏππ π 2 is added to erase as many bounding boxes as possible.
2.2 Loss Function We ensemble the two detectors to train adversarial patches, thus
The goal of our SAA is erasing all the object detections in an image. the final loss function we use is as follows. Without deliberately
And we define this type of attack on object detectors as evasion balancing the weights of two terms in πΏππ π , we can achieve high
attack. The evasion attack is closely related to the definition of attack success rate.
positive sample (foreground) and negtive sample (background) of
the detector. We erase an object in an image by making it become a
negative sample of the target detector. Consequently we design loss πΏππ π = πΏππ π πππΏπ + πΏππ π πΉ π
πΆπ π (6)
functions according to the definition of foreground and background
of target detectors.
YOLOv4 [1] is a one-stage object detector with excellent speed
performance and considerable precision performance. YOLOv4 [1]
exploits a confidence branch to distinguish foreground and back-
ground. The bounding box with its confidence lower than a thresh-
old in YOLOv4 [1] will be recognized as background and will be
discarded in the postprocess phase. Based on these principles, we
design the evasion loss function of YOLOv4 [1] as follows
πΏππ π πππΏπ = max (ππππ (π, π)) (2)
π βπΆ,π βπ΅
Here πΆ denotes the set of all object categories, π΅ denotes the set of
all bounding boxes and ππππ is the object confidence in YOLOv4 Figure 2: Adversarial examples generated by SAA.
[1]. So the loss function extracts the maximum object confidence
of an image. We minimize it to achieve our attack purpose.
29
2.3 Optimization Details Table 1: Evasion Scores
With only 2% of pixels allowed to perturb, itβs hard to generate
adversarial examples with high attack success rate. We exploit es-
Detector(s) Evasion Score
sential tricks in the training process of adversarial patch to generate
YOLOv4 1610.03
powerful adversarial examples.
FR-RES50 1174.21
To overcome the preprocess distortions, we include preprocess
Black-Box*2 355.69
into the forward and backward phase of adversarial patch training,
which including image resize, image normalization and other neces-
sary processes. The strategy avoids unnecessary adversarial attack
conduct experiments on two black-box detectors and achieve con-
enhancements and thus accelerate our adversarial patch training
siderable transferability. The evasion scores of these detectors are
process.
listed in table 1. In white-box settings, our strategy achieve pretty
We exploit multi-scale steps to update adversarial patch and set
high evasion scores of more than 1000 on both YOLOv4 [1] and
all steps to integer multiples of 1/255. We accelerate the training
FasterRCNN [10]. Moreover we achieve 355.69 evasion score on
process tremendously by using large steps first. Also, itβs a great
two completely different black-box detectors, without any transfer-
way to avoid local minima in optimization. We decrease the update
ability enhancement strategy introduced.
step gradually to refine our adversarial patch as the training goes
on. It is worth mentioning that we set update steps to be special
values (integer multiples of 1/255) to filter out the quantification
effect.
We also design multi thickness cruciform patch to adaptively
attack images with different number of object detections. We be-
lieve thinner cruciform patch produce stronger attack effect for its
adversarial effect on a wider range of image feature. Experiment
results confirm its validity in practice.
In experiments, we conduct two phases attack to FasterRCNN
[10]. At the first phase, we set πΌ 1 = 1 and πΌ 2 = 0 in πΏππ π πΉ π
πΆπ π ,
aiming at suppress the bounding box with highest object probability.
Due to rough patch location selection, we rarely succeed in the first
phase to attack FasterRCNN [10]. And the second phase will be
started once the first phase failed, we set πΌ 2 = 1 and πΌ 1 = 0 to attack Figure 3: Other shapes of adversarial patch.
as many as bounding boxes as possible. We donβt set πΌ 1 β 0 and
πΌ 2 β 0 together for better optimization of each term in πΏππ π πΉ π
πΆπ π We also make preliminary attempts to design different shape of
at different phase. the sparse patch, like what is shown in figure3, and get good results
in some cases. We argue that better design of sparse adversarial
3 EXPERIMENTS patches is more effective in π 0 norm bounded settings. And we
We select 1000 images from MSCOCO2017 dataset with all images believe strategies like attention and saliency map can be exploited
resized to 500*500 size. And we choose YOLOv4 [1] and FasterRCNN to improve our method.
[10] as target models to conduct evasion attacks using our sparse
adversarial patch. 4 CONCLUSION
In order to comprehensively evaluate the performance of our In this work, we propose a sparse adversarial attack on object
attack method, we use evasion score as the evaluation indicator. detectors. We design evasion loss functions to blind detectors with π 0
For an image π₯ and its adversarial version π₯ β² ,the evasion score in norm bounded perturbations. Our method achieve very high attack
model π is defined as follows success rate on two state-of-art detectors and manifest considerable
transforability even in black-box settings. Even so, we believe that
our method can be further improved via selecting better locaions
πππ(π΅(π₯, π), π΅(π₯ β², π))
Γ
π π
π of the adversarial patch in the image.
π (π₯, π₯ β², π) = (2 β ) Β· (1 β ) (7)
5000 π΅(π₯, π)
where π
π is πΎπ‘β connected domain of adversarial patch and π΅(π₯, π) REFERENCES
denotes the number of bounding boxes of image π₯ predicted by [1] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. 2020. YOLOv4:
Optimal Speed and Accuracy of Object Detection. arXiv preprint arXiv:2004.10934
detector π. That means the fewer pixels you change and the more (2020).
bounding boxes that disappear, the higher the evasion score. You [2] Tom B Brown, Dandelion ManΓ©, Aurko Roy, MartΓn Abadi, and Justin Gilmer.
2017. Adversarial patch. arXiv preprint arXiv:1712.09665 (2017).
get no more than 2000 evasion score in each detector. [3] Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness
We ensemble two state-of-the-art object detectors YOLOv4 [1] of neural networks. In 2017 ieee symposium on security and privacy (sp). IEEE,
and FasterRCNN [10] to generate sparse adversarial examples us- 39β57.
[4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual
ing our SAA. The FasterRCNN we choose is ResNet-50 [4] based learning for image recognition. In Proceedings of the IEEE conference on computer
and FPN [6] is introduced to optimize feature extraction. We also vision and pattern recognition. 770β778.
30
[5] Lifeng Huang, Chengying Gao, Yuyin Zhou, Cihang Xie, Alan L Yuille, Changqing [9] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and
Zou, and Ning Liu. 2020. Universal Physical Camouflage Attacks on Object Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks.
Detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and arXiv preprint arXiv:1706.06083 (2017).
Pattern Recognition. 720β729. [10] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster r-cnn: To-
[6] Tsung-Yi Lin, Piotr DollΓ‘r, Ross Girshick, Kaiming He, Bharath Hariharan, and wards real-time object detection with region proposal networks. IEEE transactions
Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceed- on pattern analysis and machine intelligence 39, 6 (2016), 1137β1149.
ings of the IEEE conference on computer vision and pattern recognition. 2117β2125. [11] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan,
[7] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks.
Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. arXiv preprint arXiv:1312.6199 (2013).
In European conference on computer vision. Springer, 21β37. [12] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan
[8] Xin Liu, Huanrui Yang, Ziwei Liu, Linghao Song, Hai Li, and Yiran Chen. Yuille. 2017. Adversarial examples for semantic segmentation and object detection.
2018. Dpatch: An adversarial patch attack on object detectors. arXiv preprint In Proceedings of the IEEE International Conference on Computer Vision. 1369β1378.
arXiv:1806.02299 (2018).
31