Nowadays biometric user authentication systems have become widespread. These systems are implemented not only in enterprises, controlledaccess facilities, but also on smartphones of ordinary users and in online applications. There are special attacks designed to impersonate another person by submitting fake biometric data to the authentication system. The composition, purpose and main components of biometric face recognition systems are considered in the article, the essence of some known types of attacks, existing methods of counteracting counterfeit attacks (spoofing attacks) are analyzed and a new method for detecting them is proposed. The method is based on the use of an artificial convolutional neural network which was trained using a Replay-Attack Database from Idiap Research Institute. The results of modeling attacks on training, validation and test data sets are presented. The obtained results show high efficiency of the proposed method: the probability that an attack will be detected is 94.98%. This indicates the prospect of further studies of artificial neural networks for detecting attacks of counterfeit biometric images.
Biometrics refers to the science of defining a personality basing on analysis of its
behavior or physical or chemical characteristics [
Depending on the application, biometric systems are divided into two types: to authenticate or identify a person in the system. The identification system compares the input data with each image that lies in the database, and then makes its decision about the identity of the person. The authentication system compares the input data only with an image that matches certain unique identifier in the database.
Since there are too many features that can be removed from a person's biometric
data, for a feature to be considered useful, it must meet certain conditions. There are
several of the following conditions given in [
One of the biometric systems that meet these conditions is the face recognition system. In this system the biometric data scanner is a typical or specially designed camera for this task. In the article we will keep in mind that the scanner will use a regular smartphone camera or webcam for biometric data capture in the system.
The objective of this work is to develop a method for detecting a spoofing attack on a biometric face recognition system. The proposed method is based on the use of modern elements of deep learning convolutional neural networks, which is used to detect attempts to fake biometric images (for example, by replacing biometric data when scanning the face with appropriate photo or video images). 2
Deep learning methods that have developed significantly in recent years are used in
our face recognition system. Their application allows achieving high accuracy [
In the general case, the process of face recognition using deep learning methods can be described as follows: 1. User takes a photo of his face in a real-time mode. 2. Next, the module that finds the person's face in the image works. The found face is sent to a module that processes unique features. 3. The unique features of a mathematical image along with a unique user ID are compared with those already stored in the database. 4. The similarity of these two mathematical images is verified. If the difference between them does not exceed a specified value, then the user is allowed to access some resource or other type of service, that is, the authentication was successful. If the difference between the images exceeds the specified value, the user is informed that the authentication attempt has failed and the access to resource is failed.
A trained deep convolutional neural network is used as a module that finds the face
in the image. The algorithm which is used to find the object is described in [
The described system does not work in real time mode, that is, the input of
algorithms is submitted with only one image of the user. Therefore, such a system is
vulnerable to special attacks on face recognition algorithms. Mainly, such as substitution
or spoofing attacks [
The general scheme of such attacks implementation is shown in Figure 2. If Figure 1 shows the correct biometric system, an attempt to spoof the attack will look like a substitution of real data of the user to the image of his data (in our case, this is the face).
Let's analyze known attacks on face recognition systems that are built using deep learning methods. 3
Currently, there are two types of attacks on the face recognition systems:
Adversarial attacks [
To begin with, adversarial attack information will be outlined and analyzed. This type of attack is aimed specifically at convolutional neural networks. 3.1
The essence of Adversarial attacks and methods of counteraction
In 2014, a team of Google and NYU researchers found out [
We start with an image of a panda that is correctly recognized by deep learning
methods (neural network) as a “panda” with a confidence of 57.7%. If carefully
constructed noise is added to this, the neural network itself will assume that it is a Gibbon
image with 99.3% confidence. This is obviously an optical illusion, but only for the
neural network. For human vision, nothing has changed, and it is safe to say that both
images look like pandas – in fact, one cannot even say that some noise was added to
the original image to build an example on the right. But adding ordinary noise is not
the only type of attack. Another example is already aimed at bypassing of biometric
facial recognition system via creating special glasses. It is showed in [
The most interesting thing about this type of attack is that it is completely
impossible to counteract it. There are only a few ways that can just reduce the likelihood of
an attack of this kind [
The second way is Defensive Distillation. Its main point is that it trains the second
model, the upper layers of which are smoothed in the places that the attacker will
usually try to use, complicating their detection, which can lead to incorrect
categorization. The reason it works is that, unlike the first model, the second one is trained on
the “soft” output performance of the primary model, rather than the “hard” (0 or 1)
ones with real training data. This technique has been shown to be successful in
protecting against simple variants of such attacks, but has become vulnerable to newer
attacks. In particular, one can take the Carlin-Wagner attack as an example [
Thus, as one can see, there are no explicit methods of defense against the attacks
under consideration. Therefore, the task of solving this issue is still a method of
research in the field of computer science or artificial intelligence [
This is more complicated attack than video playback. This allows you to use facial movements and to bypass some additional layers of protection, such as depth sensors.
There are many different approaches to counteracting attacks of this type. The most popular modern solutions include: Live Face Detection: A mechanism based on an analysis of how “live” is a test face. This is usually done by checking eye movements, such as blinking and face movement. Contextual information exploration: By exploring the surroundings of an image, we can try to detect whether a digital device or photo paper was in the area found. Texture analysis: In this method, small textured portions of the input image are probed to find patterns in fake and real images. User Interaction: By inviting the user to perform a specific action (turn their head left or right, smile or blink their eyes), the algorithm can detect whether the action was performed naturally.
Of course, one cannot ignore the special technical means specially designed to
counteract such attacks. In practice, it was possible to offer Apple a list of
deeprendering and 3D-usage methods that allow for fake support with great precision [
During research and testing, it has been found that an extremely high level of realtime counterfeit detection can be achieved with a standard medium quality camera. We offer a new method of recognition spoofing attacks, which is based on the use of convolutional neural networks. 4
As mentioned above, convolutional neural networks perform very well in computer vision tasks. Therefore choosing the approach to developing the method, it was decided to use them.
As is known from [
As a result, the proposed method consists of using a convolutional neural network that receives biometric images of human faces at the input (128x128 pixels in our experiments) and outputs a binary solution: whether the input image is spoofed or not.
The convolutional neural network (CNN) was first proposed by Jan Lekun in 1988
and aims at effective pattern recognition [
Usually standard methods are used to train convolutional neural networks, most
often the backpropagation method is the gradient calculation method used for updating
the weights of a multilayer perceptron [
The Replay-Attack database consists of 1,300 video clips of attempted photo and video attacks for 50 different people under different lighting conditions. Data from this database is already divided into 3 subgroups, including: Train data that will be used to train the anti-forgery classifier. Validation data used to find the optimal classification threshold. Test data to measure the effectiveness of the method developed.
Persons who appear in one of the datasets (train, valid, or test) do not appear in any other dataset. After some experiments, it was decided to integrate this method immediately after the face image was found on the input. This is due to the fact that, in some examples, a spoofing attack, the face finding algorithm did not find the face in the glossy type photos.
The problem in our work is formulated as follows: a biometric image of the face is fed to the input of the spoofing attack recognition algorithm. At the output, the algorithm should give one of two possible binary values: 1 (or «yes») - if the input is classified as a spoofing attack, 0 (or «no») - if the input is classified as a real image.
An image argumentation algorithm was developed to extend our data, and the following types of argumentation were used:
Adding a Gaussian noise with a value scale 0.05 255 , where scale is a standard deviation from the normal noise-generating distribution; Adding a Gaussian Blur with a value sigma 0.5 , where sigma is the standard deviation of a Gaussian filter; Rotations from -45 to 45 degrees; Zoom in or zoom out.
The next stage of the study was the justification of special metrics to measure the effectiveness of the developed method. Because as a result of detecting an attack, a binary decision is made (yes / no), we have the following set of possible situations (Table 1). In our case, the null hypothesis is defined as follows: at the entrance to the spoofing recognition algorithm, we consider that the input image is an attempt to spoof the attack, which is denoted as H0. Then, the alternative hypothesis is defined as this: the input image that is fed to the input of the recognition algorithm is not an attempt to spoof the attack and is denoted as H1. For each input image, a decision or conclusion is drawn that accepts or rejects the null hypothesis, i.e. whether it is an attempt to spoof the attack or not.
For all input data, the decision rule is: if the output of the algorithm is less than 0.5, then it is assumed that the image is not an attempt to spoof the attack, otherwise it is assumed that the image is a spoofing attack.
Table 1 shows the following terms: True positive – the model correctly recognized the spoofing attack; False positive – the model recognized the spoofing attack as a real face image; True negative – the model correctly recognized the real face image; False negative – the model recognized the real face image as a spoofing attack.
False positives and False negatives are also called 1st and 2nd order errors. Depending on the system in which the algorithm is used, attention to one of the errors becomes greater. For example, if we take our situation into account, attention should be paid to a 1st-order error, since, when a model recognizes a spoofing attack for a real face, it can lead to critical consequences. Otherwise, for example, we have a system for recognizing spam emails on the e-mail box, then there will already be more attention to the error of the 2nd kind, since, when a non-spam email is recognized by the algorithm as spam, it can lead to that an important email may be transferred to the spam folder and not reach the end user.
After defining the basic concepts involved in evaluating the performance of a classification algorithm, let's determine what our model performance test is. Suppose we have trained our network on training data, and we need to know how the model would behave when dealing with data that was not in the training sample. To do this, there is or creates a special set of images that the algorithm has not yet seen. During testing, all results are stored in a special form, depending on Table 1. Thus, after testing throughout the test sample, we have four values that are the results of the model. Such values are the number of results True positive, True negative, False positive, False negative. But these are just basic metrics and, to better evaluate the performance of the model, use other custom metrics. In our work, such metrics are Precision and Recall:
True positives
True positives True positives False negatives , (1) (2) True positives - the total number of correctly recognized attacks; False positives - the total number of false unrecognized attacks; False negatives is the total number of mis-recognized attacks.
Thus, in formulas (1) and (2), Precision shows the proportion of detected attacks
from all attacks that were in the data set, whereas Recall shows the proportion of
correctly recognized attacks from all images that were recognized as attacks [
After defining the necessary metrics and data processing methods, the next step is to develop a neural network architecture. Figure 5 shows its architecture.
From Figure 4 you can see that our neural network has 5 convolutional blocks
consisting of a Convolutional (Conv2D) layer and a MaxPooling2D layer. The last digit
in the Output Shape value indicates the number of filters in the convolutional layer, in
all convolutional layers the filter size is 3x3, and the relu activation function [
The optimization of the neural network weights was performed using the Adam
algorithm [
The input to the algorithm: : Learning speed (standard setting 0.001 ); 108 ; 1,2 0,1 : Exponential decay rates for moment estimates (default settings m0 0 (Initialization of the first moment vector); 0 0 (Initialization of the second moment vector); t 0 (Initializing time step).
Until t converges, we perform the following operations in a loop: t t 1 ; gt ft (t 1) (Take a gradient with respect to the stochastic function of each time step t); mt 1mt1 (1 1)gt (Update the rejected value of the first moment); t 2 t 1 (1 2 )gt 2 (Update the rejected value of the second moment); mt t
mt 1 t1 t 1 t 2 t t 1 .
(Calculate the rejected value of the first moment); (Calculate the rejected value of the second moment);
mt t
(Update parameters).
At the output of the algorithm we get t (Result parameters).
The training of the neural network was held with the following parameters:
learning_rate = 0.001;
epochs = 25;
batch_size = 32;
loss: binary_crossentropy,
learning rate – a training step or rate at which weights change. If you choose too
much of a learning step, the function can jump over the global minimum and
never collapse, and vice versa, if set too small, the function can get stuck in the local
minimum and never get out of it;
epochs – number of training cycles, that is, how many times the model should go
through all training data;
batch size – the number of images that are fed into the model at the same time (to
made the model geared towards generalizing the data and to speed up the learning
process);
loss - a special function that is used to optimize the neural network. Loss helps
optimize neural network parameters. Our goal is to minimize loss to the neural
network by optimizing its parameters (weights). Loss is calculated by using a
special function, comparing the target (real) value and the predicted value with a
neural network. Then we use Adam gradient descent method to update the neural
network weights so that loss is minimized. So we train a neural network.
Therefore, loss is a function of losses, it is in charge of what the neural network will
study, in our case the task of binary classification, so the choice falls on
binary_crossentropy loss [
H p (q) 1 N
yi log( yi ) (1 yi ) log(1 yi ) , N i1 (3) yi is a label predicted by the neural network.
That is, binary_crossentropy loss measures the effectiveness of a classification model whose output is a probability value between 0 and 1. Loss increases when the predicted probability deviates from the real value. The ideal model should have a value of loss function close to zero.
All software was programmed in Python using the following frameworks:
tensorflow [
After 20 epochs, the model began to show signs of retraining, so the end of training occurred on the 20th epoch. Loss of our network was 0.1796 and the overall accuracy on the validation data was 0.9485. Figure 6 shows how the training loss and validation loss of our model varied over the training period. You can also see in Figure 7 how the model's accuracy in training and validation data has changed.
You can see that in Figure 6, our loss function drops throughout the training. This means that the model was able to find the difference in the distribution of data in the real image of the human face and in the fake. Figure 7 confirms the above, we see how accuracy (the number of correctly predicted values) increases with each epoch. You can also see that the results of the test set are worse on both charts. This is because the training data is seen and memorized by the model and not by the test data. Therefore, the effectiveness of the test sample is less than the training sample.
The next step to improving the model's efficiency is finding the best decision
threshold, which is 0.5 by default. The main criterion is the balance between false
positive rate (FPR) and false negative rate (FNR) values [
As can be seen in Figure 8, the optimal decision threshold should be 0.59 in order to minimize the first-order error and the second-order error. From the figure you can see that further false negative rate does not increase much relative to how false positive rate decreases. The decision was made to set a decision threshold of 0.7 because it is more important for our system to reduce 1st order error.
When the optimal decision threshold has been found, the final step of our experiment is to measure the performance of our model on new data. In our case, such data is the test data specified in paragraph 4 of this article. Thus, according to the metrics of paragraph 4, the results of the algorithm are shown in table 2.
The values in the table were calculated on an already trained model. Metrics were calculated using formulas (1) and (2). As you can see from these metrics, the probability that an attack will be detected is 94.98%. It should be noted that models with different parameters and architectures were tested. The table shows the results of the best model, the details of which were given in paragraph 4. 6
This paper examines existing attacks on face recognition systems. A method of
counteracting Spoofing attack based on the use of artificial convolutional neural network
was developed. The probability that an attack will be detected by our system is
94.98%. Out of the box, only adversarial attacks remain, and still remain unbeatable.
It can be noted that if you use real-time recognition, adversarial noise-based attacks
will lose their relevance. Further work should focus on the recognition of the special
eyepieces that were mentioned in paragraph 3 of this article. In addition, the obtained
research results can be useful for various methods to improve the reliability and
security of biometric systems [