1. Introduction
The use of face masks has become a critical preventive
particularly in the context of the ongoing COVID-19
pandemic. In medical settings, such as hospitals and
clinics, healthcare professionals and patients are required to
wear surgical masks to minimize the risk of transmission.
However, the use of masks can hinder facial recognition
object detection, image classification, and image
segmentation. They have the potential to learn complex patterns
and representations from large datasets, which can aid
In this paper, we propose a CNN-based approach for
detecting faces covered by surgical masks in medical
settings. We aim to develop a model that can accurately and
robustly identify masked faces, considering the unique
challenges posed by diferent mask types, colors, and
ifcation and security purposes. Accurate and eficient
technology, which is commonly used for identity veri- fit variations commonly encountered in medical
envidetection of faces covered by surgical masks is thus cru- individuals wearing surgical masks, and fine-tuned a
ronments. We collect a dataset of facial images with
ResNet50 model on the specific task. The contributions
of our work include the development of a CNN-based
Traditional approaches for face detection and recog- approach tailored for face mask detection in medical
setcial for maintaining security measures while adhering to
infection control protocols.
nition may face challenges in the presence of masks, as
masks can alter facial features, obstructing key facial
landmarks and reducing facial visibility. To address this
challenge, convolutional neural networks (CNNs), a type
ically learn hierarchical features from images, have been
proposed as a promising solution. CNNs have shown
great success in various computer vision tasks, including
(C. Napoli)
CEUR
htp:/ceur-ws.org
ISN1613-073
CEUR
generalization in the presence of diferent mask types
and fit variations. The proposed approach has the
potential to enhance security measures while maintaining
applications in various real-world scenarios. The rest of
the paper is organized as follows: in Section 2, we review
related works in the field; in Section
3, we describe the
dataset used in our study; in Section 4, we introduce the
proposed method; in Section 5 we present the
experimental results and discuss the findings; and finally, in Section
6, we conclude the paper and outline future directions of research in this area.
2. Related Works
learning algorithms. The latest algorithms for face
recognition are based on deep learning architectures, 3D face
Machine learning and convolutional neural networks recognition, and hybrid approaches. As face recognition
have been widely applied to the general field of face recog- technology becomes more prevalent, there is a growing
nition. Face recognition is a rapidly developing field that need to address privacy and security concerns. ArcFace
has seen significant advancements in recent years, driven [10] is a face recognition algorithm that uses a
marginby the increasing availability of large-scale datasets and based softmax loss function to optimize feature
reprepowerful machine learning algorithms. Deep learning sentations for face recognition, similarly CosFace [11],
has become the dominant approach for face recognition another deep learning-based face recognition algorithm,
due to its superior performance on large-scale datasets. uses a cosine-based loss function to improve the
discrimSome of the most widely used deep learning architec- inability of the learned features. Like ArcFace, it has
tures for face recognition include Convolutional Neural achieved state-of-the-art performance on several
benchNetworks (CNNs), Siamese Networks, Triplet Networks, mark datasets. Another algorithm, named SphereFace
and Deep Belief Networks (DBNs). These models are [12], uses a angular-based softmax loss function to
imtrained using large-scale datasets such as VGGFace [
1
], prove the discriminability of the learned features. While
FaceNet [2], and IE-CNN models [3], which contain mil- the said algorithm have many similarities, another
syslions of face images. Deep learning models have achieved tem named DeepID [
13, 14
], has beeen developed to ofer
state-of-the-art results on various benchmarks such as a multi-task learning approach to learn multiple levels
the Labeled Faces in the Wild (LFW) [4] and MegaFace [5] of features for face recognition. However it has been
datasets. Many other applications have been developed surpassed by more recent approaches such as VGGFace2
using machine learning and face recognition algorithms [
1
]. VGGFace2 is a large-scale face recognition dataset
[
6, 7, 8, 9, 8
] 3D face recognition is an emerging field that has been used to train several deep learning-based
that uses 3D information to improve the accuracy of face face recognition algorithms, including some of the
aprecognition systems. 3D face models can capture addi- proaches mentioned above. It contains over 3 million
tional facial details such as the depth of the facial features, face images of over 9,000 subjects, making it one of the
which are not present in 2D images. Some of the popular largest face recognition datasets available. Another
conapproaches for 3D face recognition include the use of 3D volutional model is FaceNet [2] that, diferently from the
morphable models (3DMM), depth-based methods, and previous approaches, uses a triplet loss function to learn
multi-view based methods. Face recognition can be di- discriminative features for face recognition, being widely
vided into two categories: verification and identification. adopted in industry. With the outbreak of the
COVIDVerification aims to determine if two face images belong 19 pandemic, the use of face recognition algorithms has
to the same person, while identification aims to identify been applied to the recognition of people wearing (or
a person from a set of images. Face verification systems not wearing) face masks. One common approach is to
are commonly used for security applications, while face use deep learning-based object detection methods to
deidentification systems are used in large-scale surveillance tect the presence of a face and then classify whether the
applications. Recent advancements in face recognition face is wearing a mask or not. This approach typically
have focused on improving the accuracy of both veri- involves training a CNN on a dataset of masked and
unifcation and identification systems. Hybrid approaches masked faces. The CNN learns to extract features from
combine multiple techniques to improve the performance the face images and use them to classify whether a mask
of face recognition systems. For example, a hybrid ap- is present or not. Several studies have reported high
accuproach may combine deep learning models with 3D face racy rates for face mask detection using deep learning
alrecognition techniques to improve accuracy. Another gorithms. In fatct during the COVID-19 pandemic the use
approach is to use facial landmark detection algorithms of convolutional neural networks (CNNs) for face mask
to improve the alignment of face images before recog- detection has gained significant attention in literature.
nition. As face recognition technology becomes more Several studies have proposed CNN-based approaches
prevalent, there are growing concerns about privacy and for detecting masked faces. In [
15
] the authors appliy
security. Several approaches have been proposed to ad- a Long Short-Term Memory (LSTM) network to model
dress these concerns, such as anonymization techniques, the time-dependencies in order detect whether a person
which modify the facial features to protect the privacy of wears a face mask while speaking. In [
16
] the authors
the individuals in the images. Other approaches include are able to detect if face masks are worn by people in a
adversarial attacks, which aim to fool the face recognition closed enviroinment. Overall, deep learning algorithms
system into misidentifying a person. Face recognition have shown promise for detecting whether a person is
is a rapidly developing field that has seen significant wearing a face mask [17], and their use could help to
advancements in recent years, driven by the increasing improve public health measures during the COVID-19
availability of large-scale datasets and powerful machine pandemic. For example, the authors of [18] propose a
slightly modified version of LeNet [ 19] to detect masked than other facial image datasets such as the popular
Lafaces in the wild. Similarly the authors of [
20
] propose beled Faces in the Wild (LFW) dataset. The FFHQ dataset
a novel CNN architecture tailored for the specific task also includes annotations for facial landmarks, which can
and evaluate their model on a custom dataset. For more be used for tasks such as face alignment and face tracking.
information about the existing methods of covered face The FFHQ dataset has been used in a range of computer
detection we refer the reader to the recent survey [
21
]. vision research projects, including the development of
Our method difers from the other works in literature generative models for face synthesis and style transfer,
both in the dataset used and the training strategy. In as well as the development of deep learning-based
modparticular we will address the task as a multi-label clas- els for facial expression recognition and emotion
detecsification problem. Previous works [
22, 23
] have shown tion. The availability of high-quality facial images in the
that this approach allows a better understanding of the FFHQ dataset has helped to advance the state-of-the-art
context. We will see that this approach allows to reduce in these and other computer vision tasks, and it is likely
overfitting and consequently to generalize better on un- to continue to be an important resource for researchers
seen data. working in the field of computer vision. In this study
1000 images of faces uncovered and covered with masks
have been selected from the FFHQ dataset and merged,
3. Dataset to preserve generality, with a large portion of Kaggle face
mask dataset[
25
] while others were added manually in
order to increase the variability in the masks types and
colors; 160 images of masks with no faces scraped from
the web and 150 images containing diferent classes of
objects all unrelated to faces and to masks also scraped
from the web.
To train our model we created a custom dataset con
taining 151 images of uncovered faces extracted from
the Flickr-Faces-HQ dataset[
24
]. The Flickr-Faces-HQ
(FFHQ) dataset is a large-scale dataset of high-quality
facial images collected from the photo-sharing website
Flickr. The dataset was created by NVIDIA Research in
2019 and contains 70,000 images of 1,024x1,024
resolution, with a diverse range of ages, ethnicities, and genders. 4. Method
The images in the FFHQ dataset are highly curated and
ifltered for quality, ensuring that they are of high fidelity, For the classification task we used ResNet50 [ 26] a
dReshigh resolution, and well-lit. The dataset is designed to Net50 is a 50-layer deep neural network that is based
be used for training and evaluating machine learning on residual connections, which allow for the reuse of
models for various computer vision tasks, including face features from previous layers in the network. The
archirecognition, facial expression analysis, and face synthe- tecture includes several blocks of convolutional layers,
sis. One of the key features of the FFHQ dataset is that batch normalization, and pooling layers, as well as
shortit includes a wide range of facial expressions, poses, and cut connections that bypass one or more layers. These
lighting conditions, which makes it more challenging shortcut connections enable the network to learn
residual functions, which can be more easily optimized during
ResNet50 pretrained on ImageNet and we added a single
training and help to mitigate the vanishing gradient
probclassification layer with sigmoid activation (remember
lem that occurs in very deep networks. ResNet50 has
that the labels are not mutually exclusive). We used the
been trained by means of ImageNet [27]. ImageNet is a
standard binary cross entropy loss: considering the batch
5. Results
We fine-tuned the network for 200 epochs on images with
size 224x224 pixels. We used the feature extractor of the
size B, the predicted value ̂ and the true value we get:
( , )̂ = −
3
1
=1 =1
∑
∑
where
= [] log( ̂ []) + (1 − []) log(1 − ̂ [])
With this training strategy we obtained an accuracy
of 96% on a balanced testset, against the 90% of accuracy
obtained by the network trained to classify only covered
versus uncovered faces. To make a fair comparison
between the two approaches, the accuracy in the multilabel
classification problem is computed considering only the
label that indicates if the face is covered or not, which is
in fact a binary classification problem. Figure
2 shows some samples along with the network predictions.
6. Conclusion
Our findings suggest that CNNs can efectively detect
faces covered by surgical masks in medical settings,
which can be beneficial for enhancing security measures
while maintaining infection control protocols. The
proposed model has potential applications in healthcare
facilities, airports, and other settings where face mask
detection is critical for security and safety purposes. Future
work can explore additional data sources and further
optimization techniques to improve the model’s performance
and real-world applicability.
815–823.
Vggface2: A dataset for recognising faces across
pose and age, in: 2018 13th IEEE international
conference on automatic face & gesture recognition
(FG 2018), IEEE, 2018, pp. 67–74.
[2] F. Schrof, D. Kalenichenko, J. Philbin, Facenet: A
unified embedding for face recognition and
clustering, in: Proceedings of the IEEE conference on
computer vision and pattern recognition, 2015, pp.
Similar face recognition using the ie-cnn model,
IEEE Access 8 (2020) 45244–45253.
[4] G. B. Huang, E. Learned-Miller, Labeled faces in
the wild: Updates and new reporting procedures,
Dept. Comput. Sci., Univ. Massachusetts Amherst, Amherst, MA, USA, Tech. Rep 14 (2014).