Although there are a great number of adversarial attacks on deep learning based classifiers, how to attack object detection systems has been rarely studied. In this paper, we propose a Half-Neighbor Masked Projected Gradient Descent (HNM-PGD) based attack, which can generate strong perturbation to fool diferent kinds of detectors under strict constraints. We also applied the proposed HNMPGD attack in the CIKM 2020 AnalytiCup Competition, which was ranked within the top 1% on the leaderboard. We release the code at https://github.com/YanghaoZYH/HNM-PGD.
• Computing methodologies → Neural networks; • Security and privacy → Software and application security.
Object detection is one of the most fundamental computer vision
tasks, which not only performs image classification [
Method [
In this paper, we introduce an adversarial attack framework,
called HNM-PGD, which can fool diferent types of object
detectors under two strict constraints concurrently. Our method first
identifies a mask that meets the constraints, and then generates an
adversarial example by perturbing a specific area that is constrained
by the mask. Adversarial examples generated in this way are
guaranteed to satisfy the limitation in terms of the number of perturbed
pixels and connectivity regions, while remaining a high eficiency.
One key novelty in HNM-PGD lies on that it enables an automatic
process without handcraft operation, which provides a practical
solution for many real-world applications. As a by-product of this
attack strategy, we found that some perturbations contain clear
semantic information, which are rarely identified by previous studies
and provide some insights regarding the internal mechanisms of
object detectors.
Given an input example , an object detector can described as
( ) = , where represents the output vector of the detector.
Considering YOLOv4 [
In this paper, the restrictions of the adversary are i) the number of
perturbed pixels is not more than 2% of the whole; ii) the number
of 8-connectivity regions is not greater than 10. Except for these
two constraints, there are no limitations on the magnitude of the
adversarial perturbation. Because both constraints are related to
the number of pixels, this belongs to the ℓ0 norm attack.
Salience map is a common tool to analyze and interpret DNN
models’ behaviors. Smilkov et al. [
1 Õ =1 ∇ ( ( + )) , (1) where are white noise vectors that sampled i.i.d. from a Gaussian distribution. 3 3.1
We first propose a mask generation method to locate perturbation regions for any given examples. Apparently, the size and shape of perturbation regions are critical to conduct a successful adversarial attack under the constraints. Therefore, we use salience map to capture the model’s response toward each pixel in at beginning. After compute an example’s salience map, we initialize the mask via only keep pixels that the model’s respond is larger than a threshold resp. To automatically carry out this initialization, we borrow the idea of standard deviation and coverage from Gaussian distribution, and compute resp via the mean and standard deviation of , which can be described as resp = Mean( ) + Std( ), where is a control parameter.
To meet the pixel constraints, we follow the spirit of K Nearest Neighbor algorithm to refine the mask. Specifically, if half of a pixel’s neighbors have been chosen by the current mask, then this pixel would also be chosen, otherwise it will be discarded. We employ two convolution kernels whose parameters are all 1 to conduct this Half-Neighbor (HN) procedure. The first kernel aims to reduce the number of pixels in a mask, and its size is gradually changed during iterations. The second kernel is fixed to 3 ×3, it can guarantee that there are no isolated pixels in the mask and reduce the number of connectivity regions (See lines 5–9 in Algorithm 1). If the mask still does not meet the constraints, the algorithm will adjust accordingly and search again. 3.2
Once the perturbation regions are located, we generate adversarial examples via PGD iterations. The basic idea here is summarized in Algorithm 1, where the selected regions are perturbed by a PGD adversary via conducting element-wise products between perturbation and mask . The workflow of the proposed defense is shown in Fig. 1.
Due to the diference in the object detectors’ output , we need to consider YOLOv4 and Faster RCNN separately. As we discussed in section 2.1, YOLOv4 directly outputs its confidence, so Binary (a) (b) (c) Cross-Entropy (BCE) loss is a suitable option to conduct adversarial attack. Suppose there are pre-defined boxes, and the maximal confidence is 1, BCE loss can be simplified as (2) (3) () = Õ log conf, =1 where is the output of a detector and conf ∈ .
Diferent from YOLOv4, there are + 1 classes in Faster RCNN’s classification result, including foreground objects and 1 background class. To force the detector to classify an adversarial example into the background class, we conduct a targeted adversarial attack with a negative Cross Entropy (CE) loss, which can be written as () = cl+s1 − log Õ exp( cls) , . 4
where cls is the classification output of Faster RCNN detector, and cls ∈ . Note that we can attack YOLOv4 and Faster RCNN simultaneously by simply using HN masked PGD mixmize + To demonstrate our method, we select 100 images from MS COCO dataset as a toy dataset and conduct experiments for comparison on two white-box models, i.e. YOLOv4 and Faster RCNN. 4.1
YOLOv4 Regarding the provided model YOLOv4, the input size is set to 608×608 while the original image has 500×500, to allow diferential, we employ the function torch.nn.Upsample with bilinear interpolation to resolve the resize problem. Due to the approximation computation of torch.nn.Upsample, we need to allow more boxes to be detected to stabilize the adversarial perturbation. As YOLOv4 method only outputs the foreground objects with conf > 0.5, we adjust the confidence threshold from 0.5 to 0.3 during attack.
Faster RCNN Similar to the configuration of YOLOv4, we resize the input to 800×800 with bilinear interpolation. As the permitted threshold for Faster RCNN is 0.3, which is relatively lower than YOLOv4. In practice, we assign a smaller threshold 0.1 when calculating the loss to enable more boxes to appear.
PGD Settings In this paper, the HNM-PGD is carried out with 40 steps, and the step size is 16/255. In this part, we employ the formula in the description of AnalytiCup to calculate the score, which provides a criteria to evaluate the performance of the proposed method. Our code is available on Github1.
We produce 100 adversarial examples on the white-box models with two loss together: + . Figure 2 gives several successful examples for the targeted models. We can observe that the proposed method does locate the object correctly, and the added patches normally target on their sensitive parts. Figure 3 illustrates the overall score with the increasing upper bound of the number of pixel among 100 selected images, and the scores achieved by YOLOv4 and Faster RCNN, respectively. We find that with the increase of the quantity of pixel, Faster RCNN performs better, while this is not the case for YOLOv4. This is because the provided whitebox Faster RCNN uses a low tolerate threshold, where suficient pixel is needed for successful attack. In terms of YOLOv4, the performance fluctuates at about 53. Therefore, there is a trade-of when choosing the amount of the pixel.
We apply the same strategy and perform the adversarial attack with more steps (800) and smaller step size (4/255) for all 1000 images under the diferent quantity of pixel, then we pick the best result obtained on the white-box models as our solution. In the ifnal stage of the AnalytiCup competition, we had also tried to improve the generalization of the attacking approach on the unseen model, i.e. black-box. In detail, we add some transformations (like lfipping/cropping) to the input image, which is expected to not overfit the known white-box models too much. Our final score is 2414.87, ranked 17 in the competition. 5
This competition leads to a few interesting research directions.
Intuitively, due to the ℓ0 norm constraint, both location and shape of
the perturbation are critical to the attacking performance. We have
reviewed other top contestants’ solutions and found that linear
adversarial patches have a higher impact on the target model’s output
and use less number of pixels than blocky ones, while location is
less important. This seems because blocky perturbation can only
influence a relatively small range of a convolution kernel’s output,
while linear perturbation can cross a wider area. To verify such
conjecture, we wish to adopt verification technologies on neural
networks [
In conclusion, we propose a PGD-based approach to attack object detectors using Half-Neighbor masks. In the proposed HNM-PGD, the automatic pipeline allows it to craft adversarial examples/patches automatically under ℓ0 constraint, which can be applied in many applications, even physical-world attacks. On the other hand, this end-to-end attack framework also benefits further studies on defending object detectors against adversarial attacks and verifying their robustness.
The work was partially supported by the Guangxi Science and Technology Plan Project under Grant AD18281065, the Guangxi Key Laboratory of Cryptography and Information Security under Grant GCIS201817. Fu Wang is supported by the study abroad program for graduate student of Guilin University of Electronic
Technology under Grant GDYX2019025, and the Innovation Project of GUET Graduate Education under Grant 2020YCXS042.