To combat the spread of the COVID-19 pandemic, it is essential to strictly obey social distancing measures, as well as have the possibility to possess and wear personal protective equipment. This paper proposes a mask and face recognition algorithm based on YOLOv3 for individual protection applications. The proposed method processes images directly in raw data format input to a neural network trained with deep learning techniques. System training was performed on a set of images appropriately obtained from the MAFA dataset by selecting those with surgical masks for a total of about 6,000 cases. The performances obtained indicate 84% accuracy in recognizing a mask and 96% in the case of a face.
ditions, and, possibly, where necessary, compliance with the restrictions on individual protection (masks, gloves, TIn an emergency phase, the fight against the spread overalls etc.). There are several important advantages: of COVID-19 contamination is regulated by procedures the safeguard of people’s health, the mitigation of the of medical-scientific rigor and oficial protocols adopted risk of contamination return, the possibility of timely as regulations until the epidemic is definitively defeated interventions by the law enforcement engaged in preon a global scale. For the return to normality, which serving public health orders, as well as safe and fast is expected to be gradual and of medium-long dura- return to work. tion, it is essential to strictly obey social distancing The key issues forming the basis for the proposal measures, as well as have the possibility to possess described in this paper arises are the following: and wear personal protective equipment for those who continue to work in potentially contagious environ- • need for social distancing outdoors (streets, squaments. Thus, it becomes strategic to focus on solu- res, parks, etc.) and indoors (ofices, schools, tions that can remotely and non-intrusively monitor shopping centers, theaters, restaurants, pubs, shopeople’s behaviour and health, while ensuring respect ps, etc.); for privacy. One solution is represented by innovative video intelligence technologies for the automatic • need to manage quotas for access and use of pubdetection of body temperature and the proximity dis- lic areas and public carriers; tance between individuals in order to guarantee, and • need for timely notification of gatherings to the possibly certify, in outdoor or indoor environments, managers of the frequented areas and, in the most compliance with the regulations on the constraints of serious cases, to the law enforcement, possibly the distance between individuals (and/or the maximum via the certification of critical events; capacity in a given environment), access to indoor environments for individuals without critical health con• need to monitor the state of health (by checking the temperature) of people who access an indoor environment; • need to monitor compliance with the use of protective equipment (masks, gloves, overalls), especially in the most at-risk work contexts.
on by proposing a mask/face recognition algorithm.
In the state of the art there are many studies dealing with face recognition and, in particular, recognition of masked faces.
In [1] the authors propose a masked face detection technique useful for monitoring and identifying criminals or terrorists. They propose a CNN-based cascade framework, which consists of three carefully designed convolutional neural networks to detect masked faces.
The accuracy in recognizing masked faces is 87.8%.
In [2] the authors propose a further method of
identifying masked faces based on the LLE-CNN network
and MAFA database [
The study proposes a face-eye based multi-granular recognition model. With this approach, the accuracy of masked face recognition goes from the initial 50% to 95%.
In the present study, a mask/face recognition technique is proposed using a very performing type of convolutional neural network called YOLOv3. This method Figure 1: Block diagram of the proposed method. allows to derive the detection and classification performance of the "faces" and "masks" within the video and/or images. 3. Adopted Neural Network
The paper is structured as follows: Section 2 describes the proposed method; Section 3 illustrates the neural network used; Section 4 describes the database used; section 5 shows the performances obtained by the proposed technique; the last section is dedicated to conclusions.
The application of artificial intelligence and machine
learning algorithms turns out to be a very complex
approach if the problems requiring a solution are not
highlighted [
Figure 1 shows the block diagram of the proposed ing and object detection/tracking. From an application technique. point of view, it is possible to group object detection
The first block represents the acquisition of the video into two macro-categories: signal by means of cameras, which can be installed in indoor or outdoor environments. • General object detection: the goal is to inves
Once the video signal is acquired, a pre-processing tigate methods for identifying diferent types of phase is performed (Video processing block) which is objects using a single framework, in order to simresponsible for extracting the frames with a frame-rate ulate human vision and cognition; equal to 30 fps. Subsequently, the frames are fed into the previously trained YOLOv3 neural network. The output neural network provides a percentage of detection and classification accuracy of the face and masks present in the input data frames. • Detection applications: refers to the recognition of objects of a certain class in specific application scenarios. For example, there may be various applications for pedestrian detection, face detection or for text detection.
One of them is the Faster R-CNN [14] which rep- Table 1 resents the current state of the art for models that di- Objects in training e testing dataset for each classs. vide the task of identifying objects into several phases.
This network allows you to simultaneously train a rec- Dataset Training Testing ognizer and a bounding box designer within a single Mask 5555 1855 model. The procedure carried out by this network is Face 4173 1299 of the "proposal detection and verification" type.
A second model is YOLO (You Only Look Once). Table 2 In [15, 16] the authors have completely abandoned the Confusion matrix. pre-existing paradigm of "proposal detection and veri- Face Mask ifcation". Instead, YOLO follows a completely diferent philosophy: applying a single model to the entire im- Face 0.94 0.06 age. YOLO, in fact, divides the image into regions, pre- Mask 0.14 0.86 dicts the bounding boxes and for each of them, determines the probabilities of belonging to a certain class, all using a single network. In this dataset, a great presence of images was noted
In [17] the authors define the SSD (Single Shot De- in which the masks were not suitable for individual tector) model. This method has greatly contributed to protection, such as: scarves, sweaters and hands covthe change of perspective towards the generation of ering the face, full masks used for masquerades, etc. bounding boxes: unlike the previous models that were For this reason, image filtering was performed; in parconcerned with accurately predicting the location of ticular, selecting those that contained surgical masks. an object within the image, SSD starts from a set of Subsequently, a re-labeling of the dataset was perbounding boxes by default. Starting from this set a de- formed, in order to obtain an automatic recognition viation and its classification are predicted for each of system of the presence of a protective mask on a face. these boxes. Thanks to a set of operations and SSD fil- Via the new labeling, a dataset of 5,800 images was ters, it also obtains excellent accuracy in the prediction extracted, where 3,800 images were used for training of object classes. the neural network and 2,000 images were used for
In order to make an exhaustive comparison of the testing. various convolutional models presented, to maintain Each image can contain one or more “Mask” and a certain consistency in the results, it was decided to “Face” objects. In this regard, Table 1 shows the numuse the work done in [18] as a framework to compare ber of the two types of objects for the training and testthe performances. In this study, the authors indicate ing dataset. that YOLOv3 is clearly superior, compared to the other CNNs, both in terms of computational time and accu- 5. Performance Evaluation racy. However, it should be noted that Fast R-CNN, despite the huge gap in terms of computational time, al- Once the neural network model and the dataset in use lows, among others, to identify very accurate segmen- are defined, it is possible to analyse the performances tations (polylines) when compared with the "simple" obtained when the dataset described above is fed to bound boxes provided by YOLOv3 or SSD. Therefore, the network. based on the specific application context there may be After a training phase of the neural network model, some cases in which Fast R-CNN is the optimal solu- the testing dataset was applied, containing images comtion. pletely unknown to the network. The performances on the testing dataset obtained 4. Database from the network are shown in Tables 2 and 3. Table 2 shows the confusion matrix produced by the neural network. The performances obtained are quite high, implying that the network is able to perform good detection of the two classes on images that it has never seen before.
Table 3 shows the performances, in percentage, obtained using the statistical classification parameters: accuracy, recall or sensitivity, precision and F1 score
to the search for a database containing faces and masks to train the model.
At first, MAFA [