Towards Understanding How Self-training Tolerates Data Backdoor Poisoning

Abstract

Recent studies on backdoor attacks in model training have shown that polluting a small portion of training data is suficient to produce incorrect manipulated predictions on poisoned test-time data while maintaining high clean accuracy in downstream tasks. The stealthiness of backdoor attacks has imposed tremendous defense challenges in today's machine learning paradigm. In this paper, we explore the potential of self-training via additional unlabeled data for mitigating backdoor attacks. We begin by making a pilot study to show that vanilla self-training is not efective in backdoor mitigation. Spurred by that, we propose to defend the backdoor attacks by leveraging strong but proper data augmentations in the self-training pseudo-labeling stage. We find that the new self-training regime help in defending against backdoor attacks to a great extent. Its efectiveness is demonstrated through experiments for diferent backdoor triggers on CIFAR-10 and a combination of CIFAR-10 with an additional unlabeled 500K TinyImages dataset. Finally, we explore the direction of combining self-supervised representation learning with self-training for further improvement in backdoor defense.

Keywords

• eol>backdoor attack
• data poisoning
• self-training
• deep learning

1. Introduction

posing a real threat to deep learning security [ 8 ]. One of such data poisoning attacks is the backdoor attack (also Deep neural networks (DNNs), key components of deep known as Trojan attack) [ 9, 7 ], where a fraction of the learning, have prompted a technological revolution in training data is corrupted by the addition of a trigger. In artificial intelligence through various applications in com- this paper, we focus on defense against such backdoor puter vision [ 1, 2, 3 ] and other realms [ 4, 5 ]. Due to the attacks. ever-growing capacity of DNNs, the models are capable In many applications of deep learning, there is the of learning better and more accurately during the train- availability of large quantities of unlabeled data - labeling ing phase. This can sometimes lead to DNNs being brittle is often cumbersome due to time and resources. Hence, - (1) Well-crafted imperceptible perturbations on test im- semi-supervised learning [ 10 ] has been a growing area ages can cause the model to misclassify images during of research, which aims to leverage such unlabeled data the inference stage (known as evasion attack) [ 6 ]. (2) to improve the performance of DNNs. Self-training is Another attack called data poisoning attack [ 7 ] can occur one such popular paradigm, which has been proven to ifrst during training by manipulating the training data by perform really well in large data settings [ 11 ]. In this the introduction of toxic artifacts. These are memorized context, we aim to address the following question: by the model and are carried on to the inference stage.

Attack type (2) is the major focus of this work. (Q) How does self-training relate to robustness against

Recent DNNs are extremely data-hungry - they are backdoor attacks ? often trained using data from anonymous or unverified sources from the internet. This makes it particularly convenient for adversaries to manipulate datasets, leading to various kinds of stealthy data poisoning attacks, thus

© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License into probable clean and poisoned samples. Then semiCPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) supervised learning (MixUp [ 16 ]) is performed with the

Self-training has been recently shown to have some

capability of using diverse feature priors during training [ 12 ] and help alleviate spurious correlations under certain sets of assumptions [ 13 ]. This inspires our study towards understanding if this paradigm may be helpful in backdoor defense.

To the best of our knowledge, the most relevant work to ours is [ 14 ]. In [ 14 ], self-supervised learning and symmetric cross entropy loss [ 15 ] is used to separate the data 2. Related Works ifltered clean samples as labeled data and the rest as un- preprocessing module with the intent of damaging the labeled data (by removing their labels). However, this trigger pattern before passing it into the DNN. (2) Demethod is not able to answer our question (Q). Diferent tection based defense [ 29, 30, 31, 32, 33 ]: The aim here from [ 14 ], we do not aim to filter out clean samples using is to detect the presence of possible malicious samples heuristic detection method. Though we use self-training or backdoored models. Then the method either denies as a semi-supervised learning algorithm for mitigating the use of such suspicious object or filters the suspicious backdoor, we aim to get insights on how self-training as input samples for re-training. (3) Erasure based or model a sole learning paradigm can help in backdoor mitigation. reconstruction based defense [ 34, 35, 36, 37 ]: This type We summarise our contributions as follows: of defense aims to erase the efect of triggers from an already backdoored model such that it performs well in • We show that self-training can mitigate backdoor both clean samples and in the presence of triggers. (4) using additional clean unlabeled data Trigger synthesis [38, 39, 40, 41]: Here the trigger is po• We propose that self-training combined carefully tentially detected and synthesized in the first step and with data augmentation has the capacity to de- then the efect of such a trigger is suppressed. (5) Poison fend against backdoor to a certain extent, even suppression defense [42, 43]: This kind of defense tries to when the unlabeled data is poisoned. suppress the efectiveness of hidden triggers in the input samples during training, thus preventing the model from • Stronger defense is possible if stochastic data aug- learning any correlation with the trigger. mentation schemes (like in SimCLR [ 17 ]) are em- In this paper, we aim to suppress the poison using data ployed with self-training augmentation and erase its efect from a trained poisoned model by self-training. Thus, our work falls within the scope of poison suppression and erasure-based defense.

2.1. Backdoor Attacks

Backdoor attack is one of the emerging fields of research

in data poisoning while training neural networks. We focus on two types of trigger-driven backdoor attacks poisoned label attacks and clean label backdoor attacks.

The poisoned label attacks constitute of poisoning the training dataset by injecting a trigger in a small portion of the dataset and mislabeling them to a target class. This was fundamentally demonstrated in BadNets [ 18 ] which used a rectangular patch and stamped it on an area of an image. Subsequently, more sophisticated triggers have been developed [ 19, 20, 21, 22 ].

Another type of backdoor attack constitutes the clean label backdoor attack [ 23, 24 ]. The images belonging to the target class are adversarially perturbed away from their true class and then injected with the trigger. Training with such images establishes the correlation between the trigger and the target class. This type of attack is more stealthy because the labels of the target class are consistent with the ground truth labels.

In this paper, we consider both types - the basic poisoned label and the clean label backdoor attack for our experiments.

2.2. Backdoor Defense

Due to the emerging threat of backdoor attacks, several

kinds of defenses have been proposed. These roughly belong to the following categories : (1) Input preprocessing [ 25, 26, 27, 28 ]: This kind of defense introduces a

2.3. Self-training

Self-training is a form of semi-supervised learning [10]

which attempts to leverage unlabeled data to improve classification performance in the limited data regime. Diferent types of semi-supervised learning paradigms have been explored such as consistency training [44, 45, 46, 47] and pseudo-labeling [ 48, 49, 11 ].

In self-training, a good teacher model is initially trained using the labeled data. This model is used to generate pseudolabels for the unlabeled data which are then used to train a student model. This same process is repeated iteratively.

The main rationale behind this method is that a trained teacher model would provide better predictions on unlabeled data than pure chance. Because of the uncertainty of the correctness of predicted pseudolabels, a confidencebased example selection scheme [50] is often employed. Here, a fraction of pseudolabels for which the teacher model assigns the highest probability is used to train the student model. This is repeated with increasing fractions of unlabeled data till completion.

In recent studies, self-training has been shown to have some capacity to incorporate diverse feature priors in learning [ 12 ]. Thus, self-training may be able to use more robust features in the data and not rely on the backdoor trigger, if designed properly. Moreover, under certain assumptions, it was shown that self-training could avoid spurious correlations [ 13 ]. Thus, in this paper, we study the usefulness of self-training in mitigating stealthy backdoor attacks. 3. Preliminaries and Setup on clean samples and attack success rate (ASR), which is the adversarial performance of the model on samples stamped with the train-time backdoor trigger. ASR is given by the fraction of the poisoned test samples from the non-target classes which have been predicted as the backdoor target class.

4. Backdoor Defense Via

Self-Training With Data

Augmentation

In this section, we describe our pilot study and the resultant proposed approach for defending against backdoor attacks using self-training.

Backdoor Attacks. We briefly describe the general steps to a backdoor attack. We consider a clean dataset = {(xi, )}=1 where xi is an image and is 4.1. Self-training meets Backdoor: a pilot the corresponding label. Based on a poisoning ratio , study the clean dataset is divided into and such that = || and = ⋃︀ . is modified with an In what follows, we present an experiment that motivates attack|er|defined poisoned image generator such that our further investigation in this direction. = {(x′, ) |x′ = (x), (x, ) ∈ }. For example, We consider a small part of the CIFAR-10 dataset [51] one of the ways of poisoning images is to stamp a small as labeled data and the rest as unlabeled data . We checkerboard pattern (called trigger) at a fixed location pretrain a model on and perform self-training with of the image and change the labels to a target label . this trained model using and the rest of the unlabeled Finally, the poisoned dataset = ⋃︀ is sent to data . Detailed experimental settings are described the users who may train a DNN on this dataset leading to in Section 5.1. We consider the self-training algorithm the creation of a model vulnerable to backdoor attacks. described in [ 12 ], which selects samples in each iteration Threat Model. In this paper, we consider that the train- based on their confidence levels. ing dataset is maliciously poisoned using backdoor trig- In Table 1, we report the performance of the model gers. However, the user has no prior knowledge on such when it is self-trained with additional unlabeled data with train-time data poisoning. The user can obtain such a varying poison ratio. The pseudo-labeling in self-training dataset, for example, by scraping images from the inter- can be performed on the unlabeled data only ( ) or on net. The goal of the user is to develop a training scheme to all of the data ( ⋃︀ ). We eliminate any supervisory train models that are not vulnerable to backdoor attacks loss from our self-training schemes because including even at the presence of poisoned data samples. such a loss helps in successful backdoor creation, due to Problem Setup. Diferent kinds of defenses against back- the presence of backdoor triggers and malicious targets. door attacks have been proposed. However, defenses The key insights that we get from this are as follows: based on self-training with blind data poisoning infor- • Additional clean unlabeled data may be able to mation are still less explored. In this paper, we ask: How erase the backdoor efects from a poisoned model is self-training with additional unlabeled data useful in [Table 1 row 1, ASR = 0.81%] backdoor defense when the defender has no knowledge of backdoor attack and no access to clean samples? • However, naive pseudo-labeling of poisoned data

Formally, let be the labeled dataset and be can nullify this efect. [Table 1 row 2-4, ASR the unlabeled dataset which the user has at their dis- around 100% ] posal to train a model. We assume the worst case, This presents the opportunity for designing a more carewhere is always poisoned with poison ratio (). ful self-training scheme to prevent backdoor attacks in The unlabeled data can be clean and in the worst case, our problem setting. heavily poisoned. The poison ratio of the unlabeled data is denoted as ( ). However, the user has no attack 4.2. Alleviating Backdoor Via Data knowledge about any data. The model is trained using Augmentation self-training with only and . The performance of the trained model is measured in terms of standard ac- Spurred by the previous finding that naive pseudocuracy (SA), which is the benign accuracy of the model labeling in self-training cannot mitigate backdoor, we explore the landscape of data augmentations as “fea- tions involving stochasticity like RCS, color jitter, CutOut, ture manipulation” mechanism to alleviate the efect of Gaussiam blur and Gaussian noise, we average the rebackdoor trigger. Data augmentations not only help in sults of 6 independent runs. For color jitter, we randomly augmenting the dataset with additional data, but they choose brightness, contrast, saturation between 0.4-0.8 also make it harder for the model to overfit to “easy to and hue between 0.1-0.2. learn but bad” features [52]. This efect is especially pro- We perform the aforementioned augmentations sepnounced in non-linear models like neural networks. We arately on a fully poisoned test data and observe the can think of the backdoor trigger as the “easy to learn but performance of these models as shown by the box plot in bad” feature that has a strong correlation with the target Figure 1. Details of the attack and training settings are label, and hence we try to leverage the potential of data included in Section 5.1. From Figure 1, we find that there augmentations in this case. In this context, [43] showed are large variances in ASR reduction across augmentathat extremely strong data augmentation techniques like tions which signifies that there is no single augmentation MixUp [ 16 ] can mitigate backdoor attacks in a supervised that can combat backdoors. However, we observe that training scheme. In this paper, we explore the efect of the augmentation of random cropping of 0.5 part of the data augmentations in self-training. image combined with vertical flip ping (RCS0.5+VFlip)

We consider a wide variety of data augmentations to reduces ASR considerably. We consider this particular understand their efect in the presence of a backdoor trig- augmentation for our future experiments. ger. VGG-16 models are trained on the labeled part of CIFAR-10 with three diferent types of backdoor attacks. 4.3. Self-training with data For data augmentation, we consider rotation at diferent augmentations angles, adding Gaussian noise with varying variances of normal distribution, horizontal and vertical flipping, The pseudo-labeling scheme in self-training enables us random crop and resize (with and without flip), grayscale to decouple any malicious targets from the training imconversion, color jitter, Gaussian blur, and CutOut [54]. ages. However, because of the strong correlation between We also combine suitable augmentations with YOCO [53] the backdoor trigger and the given target label, pseudoto improve the diversity of augmentations. For augmenta- labeling would most likely predict the target label in the

Algorithm 1 Self-training with Data Augmentation

Params: Number of iterations N. Fraction added per

iteration k.

Input: Labeled data = {(, )} with classes, Unlabeled data = {(, )}, model trained on .

Data Augmentation: for iteration n ∈ 1, ..., do forward-pass () through model to create pseudo-labels * forward-pass through model to create pseudolabels * = {(, * ) ⋃︀(, *)} = []; for each class c do

Select the || most confident examples from predicted by the model as class c

Add those examples to with class c; end for

Re-train (warm start) the model on until convergence; end for

Train a standard model from scratch on presence of the backdoor trigger. This could be one of the possible reasons of the high ASR in Table 1.

To take advantage of this decoupling phenomenon in self-training, we propose pseudo-labeling on training images with strong data augmentation. As mentioned before, we choose “RCS0.5+VFlip” as data augmentation. The proposed algorithm is given in Algorithm 1. Rationale. The main rationale behind this algorithm is that pseudo-labeling a transformed backdoored image would reduce the chances of the model predicting the malicious target label, as exhibited in Figure 1. However, we propose to only pseudo-label a part of the total data (in our case, we choose that to be the ), to prevent a large reduction in standard accuracy.

Description. In the algorithm, we commence selftraining by taking a model pre-trained on the labeled data as the teacher model. At the start of each iteration, the teacher model predicts the pseudolabels of data augmented labeled data () and unlabeled data . For each predicted class , a fraction of the most confident examples are chosen to retrain a student model. In our experiments, for each iteration, the same teacher model is used as the student model for training, and then the trained student model is treated as the teacher model in the next iteration. The fraction of the data chosen to train the model in each iteration is proportional to the iteration number. Thus, this process continues till the whole pseudolabeled data (Algorithm 1) is exhausted. It is important to note that we do not include any supervisory loss in our training which is usually done in standard self-training.

4.4. Self-Training via self-supervised representation learning

In this section, we explore a stronger backdoor mitigation

strategy using self-supervised representation learning with self-training.

We aim to use the exemplar-based self-supervised algorithm SimCLR [ 17 ]. SimCLR is an instance discrimination based self-supervised method using contrastive loss, with instances created using stochastic data augmentation. This contrastive learning framework learns a neural network-based encoder that outputs a representation embedding of the input data. The use of stochastic data augmentation in SimCLR creates a similar opportunity for alleviating backdoor using data augmentation and combining it with self-training. In this context, [ 14 ] high

Algorithm 2 Self-training with SimCLR

with ⋃︀

Params: Number of iterations N. Fraction added per

iteration k.

Input: Labeled data = {(, )} with classes, Unlabeled data = {(, )}, model trained on .

Train SimCLR representation encoder network (· ) Representation embedding ℎ = () clusters ← K-Means clustering of normalized ℎ ← Cluster Pseudolabels through Majority Voting for iteration n ∈ 1, ..., do if == 1 : then

Predict pseudolabels * for ( ⋃︀ ) using else forward-pass ( ⋃︀ ) through model to create pseudo-labels * end if = {( ⋃︀ , *)} = []; for each class c do

Select the || most confident examples from predicted by the model as class c

Add those examples to with class c; end for

Re-train (warm start) the model on until convergence; end for Train a standard model from scratch on lighted the diference in representation space learned by SimCLR and that learned by a supervised algorithm from poisoned data, which may help in backdoor mitigation. We describe our proposed method in Algorithm 2.

We initially train a SimCLR representation encoder network (· ) as in [ 17 ] using the complete dataset. This trained representation encoder is used to find the representations embeddings ℎ of the labeled data. We cluster these embeddings into 10 (number of classes) clusters using K-Means clustering [55]. Since, for each of these clusters, we are aware of the ground truth labels, we pseudo-label the data in each cluster through majority voting.

For self-training, pseudolabeling in the first iteration is done using the previously attained clusters. For any given sample , we can simply find the representation vector ℎ = () and predict its corresponding cluster by the minimum Euclidean distance between the cluster centers and ℎ. The rest of the self-training proceeds as described in Section 4.3.

5. Experiments

8/255 and attack learning rate = 2/255. The images are then stamped with a BadNet like grayscale trigger.

The data poisoning ratio in the labeled dataset is set to be usually 10 % for the BadNet attack and 5 % (50 % from the target class) for the Clean Label attack for successful poisoning. The poison ratios for the unlabeled dataset is given in Table 2. While using the 500K TinyImages dataset as unlabeled data, we reduce the poisoning ratio to prevent the absolute number of poisoned samples from being too high.

Training and evaluation. For both pretraining and self-training, we train our models with random cropping of padding=4, random horizontal flips and random rotation of 2 degrees. We use a SGD optimizer with a momentum of 0.9 and a weight decay ratio of 1 × 10− 4. Pretraining. We train the models on the labeled dataset for 200 epochs with a batch size of 128. For CIFAR-10, the initial learning rate is 0.01 which is decayed by 0.5 at epoch 100. For CIFAR-10 + 500K TinyImages, the initial learning rate is 0.1 which is decayed by 0.1 at epoch 90 and 180.

5.1. Implementation details

Self-training. We perform self-training for = 4 iteraDatasets and networks. We consider VGG-16 model tions and in each iteration, fraction of data added = 0.3. [56] for training using CIFAR-10. We treat 20% of the In each iteration, using the pseudolabeled dataset. In dataset as labeled data and consider the rest to be unla- each such iteration, we train the models for 150 and 110 beled. epochs for CIFAR-10 and CIFAR-10 + 500K TinyImages

Additionally, we also perform a set of experiments respectively. For CIFAR-10, the initial learning rate is 0.01 using the complete CIFAR-10 dataset as the labeled data. which is decayed by 0.5 at epoch 100, while for CIFAR-10 For the unlabeled counterpart, we consider 80 Million + 500K TinyImages, the initial learning rate is 0.1 which Tiny Images (80M-TI) dataset [57]. CIFAR-10 is a subset is decayed by 0.1 at epoch 50 and 100. of this dataset - however, many images in this dataset Finally, to end self-training, the model is trained do not belong to any of the classes of CIFAR-10. For from scratch using (Algorithm 1, Algorithm 2). For this purpose, an unlabeled dataset of 500K images were CIFAR-10, this is done for 300 epochs using SGD with constructed and made publicly available in [58]. Thus an initial learning rate of 0.01 which is decayed by 0.5 we use CIFAR-10 + 500K unlabeled data as our dataset at 100 and 200 epochs. The corresponding training and perform experiments with ResNet-18 [59]. using CIFAR-10 + 500K TinyImages is performed for 250 epochs with a learning rate of 0.1 which is decayed by 0.1 at epoch 90 and 180.

Backdoor attacks and configurations. We consider two main types of backdoor attacks for our experiments which are as follows: (a) BadNet Backdoor Attack and (b) Clean Label Backdoor Attack. For these attacks, we use a trigger of size 5 × 5, which is stamped at a fixed position in the images (lower right corner). The BadNet trigger can be a gray-scale patch or a RGB like [60] trigger and the target label is always taken as 1.

For the Clean Label Backdoor attack, images from the target class are perturbed by an adversarial perturbation so that the learned representations are distorted away from the true class. The adversarial perturbation was performed using a 10 step-PGD attack on a clean trained ResNet-18 model with the maximum perturbation =

SimCLR training. For SimCLR, we use ResNet-18 as the

base-encoder network and a 2-layer MLP projection head that produces a 128-dimensional representation space.

We use the NT-Xent loss [ 17, 61 ] (with a temperature parameter of 0.5) for training SimCLR using SGD with a 0.6 learning rate, a momentum of 0.9 and a weight decay ratio of 1 × 10− 6. This is trained for 1000 epochs with a batch size of 512 with standard data augmentations as used in [ 17 ].

Table 3 presents the performance of the proposed Al

SA ASR SA ASR gorithm 2 involving self-training with self-supervised 81.65 % 100 % 71.95 % 0.92 % representation learning, SimCLR. We test the efectiveness of this algorithm using BadNet GrayScale as the backdoor attack (poison ratio 0.1) on CIFAR-10. As we 5.2. Experimental results can see, this method is successful in improving the defense against backdoor, but it comes with a trade-of with 5.2.1. Self-training with data augmentation the standard accuracy. Although only preliminary results We present the performance of Algorithm 1 in Table 2. are presented, this shows the potential of self-training The performance is measured in terms of standard ac- integrated with SimCLR in backdoor defense. curacy (SA) and attack success rate (ASR) over diferent backdoor attacks. 6. Conclusion

As mentioned in Section 4.3, we start self-training with a pre-trained model trained on the labeled portion of the In this paper, we take a step towards understanding the data. We present the performance of the pre-trained potential of self-training as a learning paradigm for backmodel for comparison. Moreover, a semi-supervised door mitigation without any available clean data and baseline is included. The semi-supervised baseline con- without any prior knowledge of train-time poisoning. stitutes of Algorithm 1 without data augmentations i.e. We propose the use of strong data augmentations on self-training is performed only through pseudo-labeling part of the available data before pseudo-labeling in selfthe labeled and unlabeled data without augmentations. training and also explore SimCLR as a stochastic data augNo supervisory loss is included in the baseline, because mentation framework in this context. We demonstrate that would help in backdoor attack and not provide a rea- the potential of our method across diferent triggers and sonable baseline for ASR. We observe that our algorithm datasets. is successful in combating backdoor using self-training Our self-training scheme, while successful in reducing (Table 2: ASR of Proposed Method). backdoor, also leads to drop in standard accuracy. We

In our experiments, we use the aforementioned strong attribute this to mainly the usage of strong data augmenaugmentation of random cropping of 0.5 part of the im- tation which leads to severe SA loss (Figure 1). However, age combined with vertical flipping. From Figure 1, we self-training is capable of recovering SA, while preservfound that the SA reduces to about 20% when such aug- ing the benefit of backdoor suppression from data augmentation is applied. However, we observe from our mentation. This points to the potential development of experiments that in our algorithm, the drop in SA is sig- trigger-agnostic sophisticated augmentation techniques nificantly less with considerable ASR reduction i.e. SA that can leverage the self-training framework to reduce

ASR while maintaining SA. We hope that this work helps to motivate a deeper understanding of self-training towards its potential of backdoor mitigation, thus leading to more secure deep learning algorithms.

tion distillation: Erasing backdoor triggers from deep neural features from tiny images (2009).

networks, arXiv preprint arXiv:2101.05930 (2021). [52] R. Shen, S. Bubeck, S. Gunasekar, Data augmentation as feature [35] P. Zhao, P.-Y. Chen, P. Das, K. N. Ramamurthy, X. Lin, Bridging manipulation, in: K. Chaudhuri, S. Jegelka, L. Song, C. Szepesmode connectivity in loss landscapes and adversarial robust- vari, G. Niu, S. Sabato (Eds.), Proceedings of the 39th Internaness, arXiv preprint arXiv:2005.00060 (2020). tional Conference on Machine Learning, volume 162 of Pro[36] Y. Zeng, S. Chen, W. Park, Z. Mao, M. Jin, R. Jia, Adversarial ceedings of Machine Learning Research, PMLR, 2022, pp. 19773– unlearning of backdoors via implicit hypergradient, in: Inter- 19808. URL: https://proceedings.mlr.press/v162/shen22a.html. national Conference on Learning Representations, 2022. URL: [53] J. Han, P. Fang, W. Li, J. Hong, M. A. Armin, I. Reid, L. Petersson, https://openreview.net/forum?id=MeeQkFYVbzW. H. Li, You only cut once: Boosting data augmentation with a [37] X. Liu, F. Li, B. Wen, Q. Li, Removing backdoor-based water- single cut, in: K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, marks in neural networks with limited data, in: 2020 25th G. Niu, S. Sabato (Eds.), Proceedings of the 39th International International Conference on Pattern Recognition (ICPR), IEEE, Conference on Machine Learning, volume 162 of Proceedings of 2021, pp. 10149–10156. Machine Learning Research, PMLR, 2022, pp. 8196–8212. URL: [38] W. Guo, L. Wang, Y. Xu, X. Xing, M. Du, D. Song, Towards https://proceedings.mlr.press/v162/han22a.html. inspecting and eliminating trojan backdoors in deep neural [54] T. DeVries, G. W. Taylor, Improved regularization of connetworks, in: 2020 IEEE International Conference on Data volutional neural networks with cutout, arXiv preprint Mining (ICDM), IEEE, 2020, pp. 162–171. arXiv:1708.04552 (2017). [39] R. Wang, G. Zhang, S. Liu, P.-Y. Chen, J. Xiong, M. Wang, [55] C. M. Bishop, Pattern Recognition and Machine Learning (InPractical detection of trojan neural networks: Data-limited formation Science and Statistics), Springer-Verlag, Berlin, Heiand data-free cases, in: European Conference on Computer delberg, 2006.

Vision, Springer, 2020, pp. 222–238. [56] K. Simonyan, A. Zisserman, Very deep convolutional net[40] K. Xu, S. Liu, P.-Y. Chen, P. Zhao, X. Lin, Defending against works for large-scale image recognition, arXiv preprint backdoor attack on deep neural networks, arXiv preprint arXiv:1409.1556 (2014).

arXiv:2002.12162 (2020). [57] A. Torralba, R. Fergus, W. T. Freeman, 80 million tiny images: A [41] T. Chen, Z. Zhang, Y. Zhang, S. Chang, S. Liu, Z. Wang, Quar- large data set for nonparametric object and scene recognition, antine: Sparsity can uncover the trojan attack trigger for free, IEEE transactions on pattern analysis and machine intelligence in: Proceedings of the IEEE/CVF Conference on Computer 30 (2008) 1958–1970.

Vision and Pattern Recognition, 2022, pp. 598–609. [58] Y. Carmon, A. Raghunathan, L. Schmidt, J. C. Duchi, P. S. Liang, [42] M. Du, R. Jia, D. Song, Robust anomaly detection and back- Unlabeled data improves adversarial robustness, Advances in door attack detection via diferential privacy, arXiv preprint Neural Information Processing Systems 32 (2019). arXiv:1911.07116 (2019). [59] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for [43] E. Borgnia, V. Cherepanova, L. Fowl, A. Ghiasi, J. Geiping, image recognition, in: Proceedings of the IEEE conference on M. Goldblum, T. Goldstein, A. Gupta, Strong data augmenta- computer vision and pattern recognition, 2016, pp. 770–778. tion sanitizes poisoning and backdoor attacks without an ac- [60] A. Saha, A. Subramanya, H. Pirsiavash, Hidden trigger backcuracy tradeof, in: ICASSP 2021-2021 IEEE International Con- door attacks, in: Proceedings of the AAAI conference on ference on Acoustics, Speech and Signal Processing (ICASSP), artificial intelligence, volume 34, 2020, pp. 11957–11965.

IEEE, 2021, pp. 3855–3859. [61] A. v. d. Oord, Y. Li, O. Vinyals, Representation learning with [44] A. Tarvainen, H. Valpola, Mean teachers are better role models: contrastive predictive coding, arXiv preprint arXiv:1807.03748 Weight-averaged consistency targets improve semi-supervised (2018). deep learning results, Advances in neural information processing systems 30 (2017). [45] T. Miyato, S.-i. Maeda, M. Koyama, S. Ishii, Virtual adversarial training: a regularization method for supervised and semisupervised learning, IEEE transactions on pattern analysis and machine intelligence 41 (2018) 1979–1993. [46] B. Athiwaratkun, M. Finzi, P. Izmailov, A. G. Wilson, There are many consistent explanations of unlabeled data: Why you should average, in: International Conference on Learning Representations, 2019. URL: https://openreview.net/forum?id= rkgKBhA5Y7. [47] V. Verma, K. Kawaguchi, A. Lamb, J. Kannala, Y. Bengio,

D. Lopez-Paz, Interpolation consistency training for semisupervised learning, arXiv preprint arXiv:1903.03825 (2019). [48] A. Iscen, G. Tolias, Y. Avrithis, O. Chum, Label propagation for deep semi-supervised learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5070–5079. [49] Y. Zou, Z. Yu, X. Liu, B. Kumar, J. Wang, Confidence regularized self-training, in: Proceedings of the IEEE/CVF International

Conference on Computer Vision, 2019, pp. 5982–5991. [50] D.-H. Lee, et al., Pseudo-label: The simple and eficient semisupervised learning method for deep neural networks, in: Workshop on challenges in representation learning, ICML, volume 3, 2013, p. 896. [51] A. Krizhevsky, G. Hinton, et al., Learning multiple layers of

References

[1] A. Krizhevsky , I. Sutskever, G. E. Hinton , Imagenet classification with deep convolutional neural networks , Communications of the ACM 60 ( 2017 ) 84 - 90 .

[2] I. Goodfellow , J. Pouget-Abadie , M. Mirza , B. Xu , D. WardeFarley , S. Ozair , A. Courville , Y. Bengio , Generative adversarial networks , Communications of the ACM 63 ( 2020 ) 139 - 144 .

[3] S. Ren , K. He , R. Girshick , J. Sun , Faster r-cnn: Towards realtime object detection with region proposal networks , Advances in neural information processing systems 28 ( 2015 ).

[4] Y. Wang , A. Fathi , A. Kundu , D. A. Ross , C. Pantofaru , T. Funkhouser , J. Solomon , Pillar-based object detection for autonomous driving , in: European Conference on Computer Vision , Springer, 2020 , pp. 18 - 34 .

[5] A. Li , S. Zhao , X. Ma , M. Gong , J. Qi , R. Zhang , D. Tao , R. Kotagiri , Short-term and long-term context aggregation network for video inpainting , in: European Conference on Computer Vision , Springer, 2020 , pp. 728 - 743 .

[6] X. Yuan , P. He , Q. Zhu , X. Li , Adversarial examples: Attacks and defenses for deep learning , IEEE transactions on neural networks and learning systems 30 ( 2019 ) 2805 - 2824 .

[7] M. Goldblum , D. Tsipras , C. Xie , X. Chen , A. Schwarzschild , D. Song , A. Madry , B. Li , T. Goldstein , Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses , IEEE Transactions on Pattern Analysis and Machine Intelligence ( 2022 ).

[8] R. S. S. Kumar , M. Nyström , J. Lambert , A. Marshall , M. Goertzel , A. Comissoneru , M. Swann , S. Xia, Adversarial machine learning-industry perspectives, in: 2020 IEEE Security and Privacy Workshops (SPW) , IEEE, 2020 , pp. 69 - 75 .

[9] A. Schwarzschild , M. Goldblum , A. Gupta , J. P. Dickerson , T. Goldstein, Just how toxic is data poisoning? a unified benchmark for backdoor and data poisoning attacks , in: International Conference on Machine Learning, PMLR , 2021 , pp. 9389 - 9398 .

[10] O. Chapelle , B. Scholkopf , A . Zien, Eds., Semi-supervised learning (chapelle, o . et al., eds.; 2006 ) [book reviews] , IEEE Transactions on Neural Networks 20 ( 2009 ) 542 - 542 . doi: 10 . 1109/TNN. 2009 . 2015974 .

[11] Q. Xie , M.- T. Luong , E. Hovy , Q. V. Le , Self-training with noisy student improves imagenet classification , in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020 , pp. 10687 - 10698 .

[12] S. Jain , D. Tsipras , A. Madry , Combining diverse feature priors , in: K. Chaudhuri, S. Jegelka , L. Song , C. Szepesvari , G. Niu, S. Sabato (Eds.), Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research, PMLR , 2022 , pp. 9802 - 9832 . URL: https://proceedings.mlr.press/v162/jain22b.html.

[13] Y. Chen , C. Wei , A. Kumar , T. Ma, Self-training avoids using spurious features under domain shift , Advances in Neural Information Processing Systems 33 ( 2020 ) 21061 - 21071 .

[14] K. Huang , Y. Li , B. Wu , Z. Qin , K. Ren , Backdoor defense via decoupling the training process , in: International Conference on Learning Representations , 2022 . URL: https://openreview. net/forum?id=TySnJ-0RdKI.

[15] Y. Wang , X. Ma , Z. Chen , Y. Luo , J. Yi , J. Bailey , Symmetric cross entropy for robust learning with noisy labels , in: Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019 , pp. 322 - 330 .

[16] H. Zhang , M. Cisse , Y. N. Dauphin , D. Lopez-Paz, mixup: Beyond empirical risk minimization , in: International Conference on Learning Representations , 2018 . URL: https://openreview. net/forum?id= r1Ddp1 - Rb .

[17] T. Chen , S. Kornblith , M. Norouzi , G. Hinton , A simple framework for contrastive learning of visual representations , in: International conference on machine learning, PMLR , 2020 , pp. 1597 - 1607 .

[18] T. Gu , B. Dolan-Gavitt , S. Garg , Badnets: Identifying vulnerabilities in the machine learning model supply chain , arXiv preprint arXiv:1708.06733 ( 2017 ).

[19] X. Chen , C. Liu , B. Li , K. Lu , D. Song , Targeted backdoor attacks on deep learning systems using data poisoning , arXiv preprint arXiv:1712.05526 ( 2017 ).

[20] M. Barni , K. Kallas , B. Tondi , A new backdoor attack in cnns by training set corruption without label poisoning , in: 2019 IEEE International Conference on Image Processing (ICIP) , 2019 , pp. 101 - 105 . doi: 10 .1109/ICIP. 2019 . 8802997 .

[21] A. Nguyen , A. Tran , Wanet-imperceptible warping-based backdoor attack , arXiv preprint arXiv:2102.10369 ( 2021 ).

[22] Y. Li , Y. Li , B. Wu , L. Li , R. He , S. Lyu , Invisible backdoor attack with sample-specific triggers , in: Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021 , pp. 16463 - 16472 .

[23] A. Turner , D. Tsipras , A. Madry , Label-consistent backdoor attacks , arXiv preprint arXiv: 1912 . 02771 ( 2019 ).

[24] S. Zhao , X. Ma , X. Zheng , J. Bailey , J. Chen , Y.-G. Jiang, Cleanlabel backdoor attacks on video recognition models , in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020 , pp. 14443 - 14452 .

[25] B. G. Doan , E. Abbasnejad , D. C. Ranasinghe , Februus: Input purification defense against trojan attacks on deep neural network systems , in: Annual Computer Security Applications Conference, ACSAC '20, Association for Computing Machinery, New York, NY, USA, 2020 , p. 897 - 912 . URL: https://doi-org.login.ezproxy.library.ualberta.ca/10.1145/ 3427228.3427264. doi: 10 .1145/3427228.3427264.

[26] Y. Li , T. Zhai , Y. Jiang , Z. Li , S.-T. Xia, Backdoor attack in the physical world , arXiv preprint arXiv:2104.02361 ( 2021 ).

[27] M. Villarreal-Vasquez , B. Bhargava , Confoc: Content-focus protection against trojan attacks on neural networks , arXiv preprint arXiv: 2007 . 00711 ( 2020 ).

[28] S. Udeshi , S. Peng , G. Woo , L. Loh , L. Rawshan , S. Chattopadhyay , Model agnostic defence against backdoor attacks in machine learning , IEEE Transactions on Reliability ( 2022 ).

[29] H. Chen , C. Fu , J. Zhao , F. Koushanfar , Deepinspect: A blackbox trojan detection and mitigation framework for deep neural networks ., in: IJCAI , volume 2 , 2019 , p. 8 .

[30] Y. Gao , C. Xu , D. Wang , S. Chen , D. C. Ranasinghe , S. Nepal , Strip: A defence against trojan attacks on deep neural networks , in: Proceedings of the 35th Annual Computer Security Applications Conference , 2019 , pp. 113 - 125 .

[31] B. Tran , J. Li , A. Madry , Spectral signatures in backdoor attacks , Advances in neural information processing systems 31 ( 2018 ).

[32] S. Kolouri , A. Saha , H. Pirsiavash , H. Hofmann , Universal litmus patterns: Revealing backdoor attacks in cnns , in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020 , pp. 301 - 310 .

[33] B. Wang , Y. Yao , S. Shan , H. Li , B. Viswanath , H. Zheng , B. Y. Zhao , Neural cleanse: Identifying and mitigating backdoor attacks in neural networks , in: 2019 IEEE Symposium on Security and Privacy (SP) , IEEE, 2019 , pp. 707 - 723 .

[34] Y. Li , X. Lyu , N. Koren , L. Lyu , B. Li , X. Ma , Neural atten-