An adversarial example (AX) is a maliciously crafted input that humans can recognize correctly, while machine learning models cannot. This paper considers how to turn deep learning-based face recognition systems to be robust against AXs. A large number of studies have proposed methods for protecting machine learning-classifiers from AXs. One of the most successful methods among them is to prepare an ensemble of classifiers and promote diversity among them. Face recognition typically relies on feature extractors instead of classifiers. We found that directly applying this successful method to feature extractors leads to failure. We show that this failure is due to a lack of true diversity among the feature extractors and fix it by synchronizing the direction of features among models. Our method significantly enhances the robustness against AXs under the white box and black box settings while slightly increasing the accuracy. We also compared our method with adversarial training.
Deep neural networks (DNNs) have become core
components of many essential services as their performance
has gone beyond the human capability of recognition in
many tasks
An adversarial example (AX) is an inconceivably perturbed input that deceives the machine learning model into miss-classification of it, while a human can correctly classify it. Some attacking methods need an entire code of the model, a white-box setting, while some need only oracle access to the model, a black-box setting. The threat became very plausible when Sharif et al. (2016) showed physical glasses could deceive the machine learning model in the black-box setting. As a result of such an attack, a person on the blacklist may wear a slightly fancy glass to evade immigration inspection performed by machines.
Many works proposed how to prevent AXs, and many
of them are broken back, leading to an arms race between
defenders and attackers. A model ensembling
We are interested in applying the ADP method to enhance the robustness of face recognition against AXs. Face recognition commonly relies on a machine learning feature extractor since it enables the service to register a huge number of new faces without retraining the network. We experimented with ADP’s direct application and found that it does not improve feature extractors’ robustness against AXs.
We consider the cause of failure is that the directions of different models’ features are not comparable in a meaningful way, and thus their diversity is insignificant. We propose letting all the models in the ensemble share weight vectors of their final layers so that features in different models can compare themselves in a common coordinate. We also propose to promote the diversity of feature vectors directly rather than the diversity of the classifier. Figure 1 illustrates our method.
We have experimented with ADP, our methods, and
several variants. We trained them in various methods such as
ArcFace
Despite its enhancement, our model’s robustness was not so distinctly high that we can still generate successful AXs for all legitimate samples in the white-box setting if with sufficiently large perturbation. Although our method does show significantly lower transferability than others in a black-box setting, we found that very well forged AXs often, with sufficiently large perturbation, can again deceive the machine recognition with very high probability.
Face recognition typically belongs to either face identification or face verification. The former is also called closed-set face classification and assumes the probed image belongs to one of the objects enrolled in the gallery. The latter is also called open-set face classification and can reject a probed image with no corresponding object in the gallery. We focus on open-set face recognition.
Modern open-set face recognition systems commonly
consist of a DNN-based feature extractor that maps an input
image into a low dimensional feature space
There are two major ways of training the DNN-based
feature extractors. One way is to train a normal multiclass
DNN classifier initially and then regard the output of the
penultimate layer of the DNN as the feature vector (i.e.,
the DNN without the last fully connected layer is a
feature extractor)
Feature extractors trained through simple softmax outputs
are effective for closed-set classification tasks but are not
discriminative enough for open-set verification. An angular
margin penalty such as ArcFace
We briefly review the angular margin penalty. We start from the cross-entropy loss LCE (x; y) for a trainable parameter , an input x, and its true label y. We assume g(x; ) is the classification over n classes and is the fully connected final layer’s softmax output whose input is the output f (x; ) 2 Rd of the d-dimensional penultimate layer. Then, letting Wj 2 Rd and bj 2 R for j = 1; : : : ; n, respectively, an final weight vector and a final bias for the j-th prediction, eWy f(x)+by LCE (x; y) := T 1y log g(x) = log Pn `=1 eW` f(x)+b` :
The loss function of ArcFace LARC; ; (x; y) L2normalizes f and Wj , lets b` = 0, and introduces penalty and smoothness hyperparameter . With cos (x; `) = W` f(x) kW`k kf(x)k , it is;
LARC; ; (x; y) = log
e cos( (x;y)+ ) e cos( (x;y)+ ) + P`2f1;:::;ngny e cos (x;`) : 1Because of this property, we need to evaluate the accuracy of feature extractor based face recognition by datasets that do not share the labels (identities of the owners of faces) with the training dataset.
The loss function of CosFace is with a slightly different form of the penalty. We call each model of both ArcFace and CosFace as a single model and compare it with other models in our experiments. The feature extractor trained by this network is f~( ; ) 2 Rd, which is the normalization of f ( ; ).
While deep neural networks are successful to show high
accuracy in their tasks, they are vulnerable to AXs
Various studies have proposed methods to make DNNs
robust against AXs. They include adversarial
training
Some works such as
Despite the relative success of the ensemble model, all of the works are validated only for classifications. We can directly integrate the adaptive diversity promoting (ADP) method into the angular margin penalty for feature extraction. We assume that our ensemble is composed of K models, and thus all parameters have K duplicates. Each model predicts over n labels in training. Let the output of the k-th model that represents the j-th label in the ensemble be gk;j . The ensemble prediction Gj for the j-th label is the average of all the output of predictions. That is, Gj (x) = K1 PkK=1 gk;j (x). Then, Shannon Entropy of the distribution fGj gj=1;:::;n is H(G(x)) =
N X Gj (x) log(Gj (x)): j=1
Let gk;ny 2 Rn 1 be such an (n 1)- dimensional vector that is gk 2 Rn except for the y-th element, i.e., true prediction. Let g~k;ny 2 Rn 1 be L2-normalized gk;ny 2 Rn 1, and let an (n 1) K matrix M (x; y) = (g~1;ny(x); : : : ; g~K;ny(x)) 2 R(n 1) K . Then, the ensemble diversity of non-maximal (y) prediction of x is
ED(x; y) = det(T M (x; y) M (x; y)): Here, the operation of ” ” is the multiplication of K (n 1) matrix and (n 1) K matrix, whose result is K K matrix. Geometrically, it is the volume of spaces that fg~1;ny(x); : : : ; g~K;ny(x)g spans. With hyperparameters and , the regularizer for promoting adaptive diversity is ADP ; (x; y) =
H(G(x)) +
log(ED(x; y)): k
With fLM; ; (x; y)gk=1;:::;K where M represents either ArcFace or CosFace by ARC or COS, the ADP method trains the model by optimizing parameters for minimum K X k=1 LE;M;ADP; ; ; ; :=
k LM; ; (x; y)
ADP ; (x; y):
The feature extractor is the normalized penultimate layer output (f~k 2 Rd)k=1;:::;K . We let the ensemble features F be the average of all the output of features extractors as which we can compare to the original k(x; `) such that F (x) = 1 N
X f~k(x): K k=1 Here, we omitted . We use the Euclidean distance between the above ensemble feature F (x) of the input x and the registered feature F (x0) of another input x0 as a measure of similarity for our verification task.
We will later see in our experiments that ADP scarcely enhances the robustness of the network against AXs. We see its cause as below. Let normalized weight vectors of the final layer by all the models be fW~ `kg(k;`)2f1;:::;Kg f1;:::;ng. Here, k runs for models in the ensemble, and ` runs for labels. Let ff~kgk be the feature extractors. Again, k runs for models in the ensemble. The diversity promotion in the ensemble aims to enforce any perturbation to input x brings it to such x0 that features ff~k(x0)gk of different models are close to W~ `k of different `’s. The ADP is likely to achieve such a property. However, the difference in directions of features does not imply the difference of directions from learned features, i.e., the weight vectors. For example, suppose that f~1(x) near W~ 11 moves to a very different direction from the direction of which f~2(x) near W~ 12 moves when we perturb x by . However, if these directions are such that f~1(x) moves to W~ 21 and that f~2(x) moves to W~ 22, the difference in directions does not prevent AXs. Since weight vectors are independent among different models, the previous approaches do not prevent such situations from occurring. Therefore, we need to promote the diversity of features among all the ensemble models to be diverse relative to respective weight vectors’ positions.
We propose to share weight vectors fW~ `kg(k;`)2f1;:::;Kg f1;:::;ng of the final layer by all the models. We propose W~ `k for all k are the same and let W~ ` denote it. The change promotes all the label y features to be close to the same W~ y independent of models. Hence, the adversary needs to find the perturbation of x, which moves all features ff~k(x)gk=1;:::;K from W~ y to W~ y0 . Suppose the directions of the susceptibility of features to perturbation are different among different models. Then, it is hard for an adversary to find perturbation that moves features in the same direction. If we promote diversity of features, we can expect the chart of features around weight vector are different among models and can expect that the ensemble increases its robustness to AXs.
~
We call the weight vector W` that all models share as shared representative weight vectors (SRV). Let k(x; `) be the angle, i.e., the arc, between W~ ` and f~k(x) in Rd. That is, k(x; `) is such that, cos k(x; `) = W~ ` f~k(x); cos k(x; `) =
W`k fk(x) kW`kk kfk(x)k = W~ `k f~k(x): We have the loss function with the shared representative weight vector as =
LE;ARC;SRV; ; (x; y) K X log k=1
e cos( k(x;y)+ ) e cos( k(x;y)+ ) + P`2f1;:::;ngny e cos k(x;`) : We expect that feature extractors trained by the following loss function are robust to AXs.
LE;ARC;SRV;F DP; ; ; (x; y) =
LE;ARC;SRV; ; (x; y)
FDP (x): We define LE;COS;SRV;F DP; ; ; (x; y) similarly. Feature Diversity Promotion: In ADP, we promote nonmaximal predictions of models in the ensemble to be diverse. However, it is a feature that we want it hard for adversaries to manipulate rather than predictions. Hence, we choose to promote the diversity of ensemble features directly. We can measure the diversity EDfeat of ensemble features at x in the same manner as before. Let F~(x) = (f~1(x); : : : ; f~K (x)) 2 Rd K be d K matrix. Then,
EDfeat(x) = det(T F~(x) F~(x)) Here, the determinant is on the K K matrix. We promote the ensemble feature extractors to be such that their features are diverse by the following regularizer.
FDP (x) =
log(EDfeat(x)) The weighting coefficient is a hyperparameter. The regularizer has no term of Shannon entropy, unlike ADP ; , since we do not need to balance features with values such as maximal prediction in ADP.
We followed the same training process that is in
For the embedding network, we employ the widely used
CNN architecture, MobileFacenet
We follow
We set the batch size to 256 and train the ensemble consisting of three feature extractors on one NVIDIA Tesla V100 (32GB) GPU. We set the initial learning rate is 10 3, and we divide it by 10 at 12, 15, and 18 epoch. The training process finishes at 20 epoch.
We have experimented with the effectiveness of
regularizers ADP ; (x; y) and FDP (x) to the robustness of
feature extractors trained by the ensemble model of
ArcFace and CosFace. The hyperparameter of regularizers we
tested are ( ; ) = (2:0; 0:5); (2:0; 10:0), and (2:0; 50:0)
for ADP, and = 1:0; 10:0, and 50:0 for FDP. We also
compared our method with one of the best adversarial training,
which exploits margin-based triplet embedding
regularization
We applied attacks such as I-FGSM, BIM, and CW to the following models in the LOTS framework. (1) ”single model,” which is the original model, (2) ”baseline,” which is a simple ensemble of original models, (3) ”ADP,” which is a simple ensemble model with ADP regularizer, and (4) ”FDP” which is a simple ensemble model with FDP regularizer. (5) ”SRV,” which is an ensemble with a shared representative vector, (6) ”SRV+ADP,” which is SRV with ADP regularizer, (7) ”AdvT for m = 0:2; 0:6; 1:4; 3:0,” which is adversarial training, and (8) ”Our method,” which is our full model that has SRV with FDP regularizer. The number of models in each ensemble is three (K = 3).
All the attacks are adaptive, which means we applied methods to robustified networks. We could compare our method to only those previous methods we experimented with ourselves since their reported results are in different conditions.
We verified the accuracy of verifications on the VGG2 dataset. Table 1 shows ROC-AUCs of the single model, ADP, AdvT, and our method (SRV+FDP) for ArcFace and CosFace. The hyperparameters resulting from best ROCAUC are ( ; ) = (2:0; 0:5), m = 0:6, and = 10:0. Our proposed SRV+FDP is not only no worse than the original single model but performs best.
We also show the accuracy of verification on several legitimate datasets of different conditions in Table 2.
method single model Baseline
ADP FDP
SRV SRV+ADP
AdvT
single model SRV+ADP
89:60 89:71 86:20 89:94 90:94 86:62 90:80 89:97 88:29 88:33 91:14 94:22 94:15 90:52 94:13 94:93 93:98 94:38 95:15 93:07 92:43 94:97
These datasets are FW (Huang et al. 2007),
AgeDB30
We generate AXs by LOTS with I-FGSM, BIM, and CW in a white-box setting. We did not limit the number of iterations in all attacks, which swells more than 1,000 in some cases. We chose a rather large boundary ( = 0:1) in BIM. We randomly sample 1000 pairs of different identity images from the VGG2 test dataset and generated 500 AXs. As we imposed less on AXs, all the attacks have been successful with their different perturbation sizes. We let the learning rate of underlying SGD to be 0:1. We note that we can still recognize images with the largest perturbation by our human eyes.
We compared the robustness of various methods through the size of perturbation. We define the -attack success rate as
Acc = jfxadvjxadv 2 AX ; kxadv xsk2 <
gj : (1) jAX j Here, xs is a legitimate sample, and xadv is the AX created from xs. AX is a set of all the successful AXs generated in the white-box setting. This measurement represents the proportion of adversarial samples whose perturbation size is less than to all legitimate images. We can say that the larger perturbation we need to fool feature extractors, the more robust they are.
Graphs in the upper row of Figure 2 show the Acc of LOTS via I-FGSM, BIM, and CW, respectively, for ArcFace. We observed that (1) the baseline, ADP, and the FDP do not enhance the robustness against AXs in a noticeable manner, (2) SRV alone enhances robustness, (3) the addition of ADP to SRV does not enhance robustness, (4) Our
LOTS via BIM (CosFace)
FDP+SRV has the best robustness. We see the same trends for another common loss function of CosFace in graphs in the lower row of Figure 2.
Graphs in Figure 3 compare our method with adversarial training by the Acc of LOTS via I-FGSM, BIM, and CW, respectively, for ArcFace. The adversarial training is with several values of hyperparameter ”m.” Differences in robustness among them are marginal. We see our method enjoys the stronger robustness in all the cases.
We first generate AXs by LOTS with I-FGSM, BIM, and CW to the single model in a white-box setting. We generated several sets of AXs with different distances between their features and the target features. All of their distances are sufficiently small that all AXs are successful attacks. Although a class with a smaller distance is hard to generate, we successfully generated AX for all input images by searching for a longer time. On average, it took 10.97 seconds to generate an AX whose distance from the target is 0.2. As in the white-box setting experiments, we did not restrict the number of iterations; we chose a rather large perturbation boundary. We applied these AX to each model for the evaluation, whose results are in graphs in Figure 4.
The results show that our method has significantly suppressed the transferability of AXs. We could also see a clear relation between the transferability of the AXs and the distance between features. We consider it because our ensemble uses the same single model. The transferability surprisingly reaches 1 when the distance between AX and the target in feature space becomes sufficiently small. However, this comes with a much larger perturbation in input images, as shown in Figure 5. The transferability of all models approaches zero as the distance between features becomes large. It is so because AX becomes no longer a successful one with such a large distance. That we eventually have complete transferability with a small distance in feature space indicates that an essential improvement within a single model is necessary, unless we can strictly forbid the small size of perturbation by some means. We consider this is a clear limitation of the ensemble model method.
Introducing ensemble diversity in feature extractors for adversarial robustness was not direct. We proposed a novel method that could effectively increase the robustness of feature extractors. Our method might not completely prevent adversarial examples if the perturbation is large. However, it can potentially be used with other tandem methods to increase their robustness. Our method also showed stronger robustness than one of the adversarial training methods. AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, 6829–6836. AAAI Press. URL https://www.aaai.org/ocs/index.php/ AAAI/AAAI18/paper/view/17334.