Deep Neural Networks based Object Detection algorithms have shown their remarkable performance and been widely applied in various aspects in recent years. However, for those areas which draw much attention to the robustness and security of the model, like Autonomous Driving and Biomedical Image Analysis, there are still challenges to make domain users look at these detectors as reliable methods. In this paper, we mainly focus on object detection and propose our method to fool object detectors with with L0-Norm patch attack.
• Computing methodologies → Object detection; • Security and privacy;
Deep Neural Networks based computer vision algorithms have
shown their remarkable performance but also faced with
vulnerability and security problems. Adversarial attack on Image
Classification has made significant progress [
In this section, we first make description and understanding of the competition, then provide some background knowledge and review the related works about adversarial attack on Object Detection. *Corresponding author 2.1
The competition adopts two object detectors known by the
competitors (known as white-box attacks) and another two black-box object
detectors for evaluation. The competitors are asked to generate
adversarial examples by adding a small number of patches (less than
10) to each image (also known as L0 attacks) offline.
2.1.1 White-box Models: The 2 white-box model are the famous
Faster R-CNN [
First limitation is the maximum limit of changed pixels rate. Second
is the maximum limit of patches’ number. The goal of the
adversaries is to make all bounding boxes failed to be returned, by adding
the patches to images. As for evaluation score, on one hand the less
bounding boxes given by the adversarial example, the higher the
score, on the other hand the less changed pixel rate, the higher the
score. More specific definition can be found in [
A number of attacks for object detectors have been developed
recently [
The PGD [
For some attack problems where the ground-truth label is not given, we can propose a loss function towards bad prediction by setting synthetic bad label, e.g. in classification problem we may set all labels into a specific category, like 0 in MNIST. Then the original PGD can be used by min{Jysynthetic, ypred }. 2.4
Sparse l0-attack
In an l0-attack one is interested in finding the smallest number of
pixels which need to be changed so that the decision changes. We
show an adversarial image with l0-attack in Fig.1. From a practical
point of view the l0-attack tests basically how vulnerable the model
is to failure of pixels or large localized changes on an object e.g. a
sticker on a refrigerator or dirt/dust on a windshield [
N i=1 Where N is the number of predicted objects, and sml is the softmax output, and τ is a threshold more strict than NMS threshold. 1 is a indicator function that return 1 if smlx > τ else 0. The target of this function is trying to make all foreground prediction can not pass NMS. 3.2
Our attacking framework is summarized in Algorithm 1. Input
sample x to the given detectors, then the softmax probability of a specific
category can be obtained. Since background will be neglected by
NMS, we only store the foreground’s probability. Above input to
store process can be denoted as smlkx. By calculate the loss function,
we can make back propagation the gradient on x is got.(We just give
a simple description here about how to iterate the momentum, more
details like its cold start problem and various variants can be seen
in [
Algorithm 1 PGD l0-attack on ensemble of models Input: A sample x, the foreground’s softmax function of K models: sml1, sml2, ..., smlK and weights: w1, w2, ..., wK .
Hyper-params: Training epochs T , gradient momentum update factor µ, learning rate lr.
Output: An adversarial example x∗ .
x0∗ = x, momentum g0 = 0; for t = 0 to T − 1 do
Input xt∗ , get sml0xt∗ , sml1xt∗ , ..., smlK xt∗ ; Fuse the K logits as lxt∗ = kK=1 wksmlk; Get loss Jxt∗ based on lxt∗ and Eq.(1); Obtain current gradient ∇xJxt∗ gt1 = gt ∗ 1 − µ ∇xJxt∗ ∗ µ xt∗1 = xt∗ − lr ∗ gt1 update xt∗1 by projection onto l0-norm box end for return x∗ = xT∗ 3.3
We solve the connected domains rule of this competition by K-means
and then convert sub-problem to a MST. We first set a specific pixels’
number as β for l0-attack, and train the adversarial sample with
Algorithm.1 for certain epochs. Then, we group these pixels into K
areas by K-means, which can be visualized in Fig.4. The distance
metric of two pixels is given by Chessboard Distance, or known
as Chebyshev Distance[
In this section, we show our experiments based on the methodology in section 3. In our experiments, we only attack the 2 provided white-box models due to the computational resources limitation. YOLOv4 make binary classification on whether the object is foreground, and its NMS threshold (0.4) is different from Faster RCNN(0.3). We set the threshold τ in Eq.1 as 0.15 for YOLO and 0.25 for Faster R-CNN. The latter output 81 class while the former is binary, so the corresponding τ is relatively bigger. The value also take the output’s unbalanced distribution into consideration. We set our learning rate as 60000, because the attack on image’s value are 0 − 255 while the gradients are quite small. The momentum update factor µ = 0.5 is soft, and weights relatively high on the gradients of current iteration compared with the usage other task, like traditional image classification. We set the first 30 epochs to train sample with 8000 pixels l0 limitation, then use 200 epochs to train sample with less pixels with connecting operation, finally we train 30 epochs to make sure the the sample quantized into uint8 type which helps maintain most adversarial ability. The first 30 ‘pre-train’ epochs’ improvement can be visualized in Fig.5.
We show our result in Table.1 by adding the methods during the competition, the overall score list are recorded at the our subscription result on the competition entrance. And we visualized a perturbed image and its patch in Fig.6. 5
In this paper, we show our PGD l0-attack adversarial solution towards the competition’s target. We find that there is no need to make hand-crafted patches into the image, because by PGD searching
handcraft patch, A k-means
momentum
quantized training 31 683 651 997 1255 1335 61 304 783 1036 1635 2547 2836 23 51 154 357 472 546 crafted patches and only attack A, the pixels side patches will be modified. Then we use K-means to get 10 proposal areas, and the score gets better. We also attack YOLO only, and find that YOLO is much easier to be destroyed. By training on ensemble of the 2 models, we get higher black box score, even though the white-box score shows a little decrease. By adding momentum, the increase is promising. To lower the number of pixels, we optimized the connecting method by Prims together with hybrid stage-1 of A into loss function, and we get a notable improvement. Finally, we quantize the adversarial sample after every 3 epochs, which lowers the train/test error.
(a) perturbed image (b) modified areas of original image and perturbed image. Most of the perturbed pixels lie near to the 3 zebra objects. this can be done automatically. We also use K-means and Prims to handle the game’s constrain. For l0 attack, the top-K selection is intuitive because the amplitude of gradients on the input interpret how important they are for the prediction target. For the reason that the connecting process is too slow, we just connect once and keep patches fixed, which may hinder the searching result and would be optimized in future work.