Since the publication of the AlexNet in 2012, Deep Convolutional Neural Network models became the most promising and powerful technique for image representation. Speci cally, the ability of their inner layers to extract high level abstractions of the input images, called deep features vectors, has been employed. Such vectors live in a high dimensional space in which an inner product and thus a metric is de ned. The latter allows to carry out similarity measurements among them. This property is particularly useful in order to accomplish tasks such as Face Recognition. Indeed, in order to identify a person it is possible to compare deep features, used as face descriptors, from di erent identities by means of their similarities. Surveillance systems, among others, utilize this technique. To be precise, deep features extracted from probe images are matched against a database of descriptors from known identities. A critical point is that the database typically contains features extracted from high resolution images while the probes, taken by surveillance cameras, can be at a very low resolution. Therefore, it is mandatory to have a neural network which is able to extract deep features that are robust with respect to resolution variations. In this paper we discuss a CNN-based pipeline that we built for the task of Face Recognition among images with di erent resolution. The entire system relies on the ability of a CNN to extract deep features that can be used to perform a similarity search in order to ful ll the face recognition task.
Content based image retrieval (CBIR) is one of the most active research eld
in the computer vision community. In this context, commonly faced tasks are
instance-level retrieval and class retrieval [
Before the advent of the Convolutional Neural Networks (CNN), the
scale-invariant feature transform [
Under the hood, these methods rely on the ability of deep models to extract the so called deep features from given input images. From a theoretical perspective, the inner layers of a CNN realize an abstraction of the input that describes speci c concepts contained inside the data. Moreover, due to the typical structure of deep models architecture, inner layers combine the information available from previous layers thus achieving a higher level of abstraction that summarizes the overall content of the input data. Based on this observation, deep features are usually adopted as global descriptors for input images. Thus, the deeper the layer from which we extract deep features is, the more descriptive of the input they are. It is common practice to extract them from the penultimate layer of a CNN.
As previously said, deep features are high dimensional vectors de ned in a space on which it is de ned an inner product and thus a metric. This property is fundamental since it allows to evaluate similarities among descriptors, extracted from di erent images, that can be used as indicators of the similarities of the content of the original data. An example of this principle is sketched in Figure 1.
This concept is typically applied in the context of surveillance systems. Indeed, in a scenario where an input face image is acquired by a camera, its global descriptor is extracted and used in order to perform a similarity search on a database (db) containing a gallery of features vectors that belong to known identities. For example, the research in the db can be accomplished by evaluating the cosine similarity, or the Euclidean distance, among the probe and gallery vectors.
The similarity search becomes even more challenging when gallery and probe come from di erent resolution domains.
This is the background from which we started the study we present in this
paper. Our nal goal is to conceive a pipeline for face recognition based on neural
networks. In order to extract deep features we used a pre-trained ResNet-50 [
The performance of the deep model used for features extraction has been
evaluated on the 1:1 veri cation protocol on the IARPA Janus Benchmark-B (IJB-B)
dataset [
Deep learning techniques are currently experiencing a huge expansion in their
eld of application mainly as a result of the extremely high computational
power reached by the modern GPUs. Moreover, the existence of big
datasets [
To this end, several techniques have been developed in order to train deep models to deal with low resolution images. Some examples are Super Resolution and Common Space Projection techniques.
Super Resolution is a technique based on the ability of a neural network to
synthesize a high resolution image starting from a low resolution one. The
recognition task is later ful lled in the high resolution domain [
Instead, Common Space Projection techniques concern the ability of a neural
networks to minimize the distance among deep features, in a common space,
extracted from a low resolution image and its high resolution counterpart.
For example, in [
In this section we brie y describe the main modules of the pipeline we developed. A schematic view is shown in Figure 2.
Face Detector
CNN for features extraction
Probe Gallery
There are three main components at the heart of the system: a face detector,
a features extractor and a classi er. We will be focusing on the rst two.
The face detection task is accomplished by means of a multi-stage architecture
that, given an input image, delivers the coordinates of the bounding boxes that
are centred around each face visible in the image. Speci cally, we used the
Multi-task Cascaded Convolutional Neural Networks (MTCNN) [
Features vectors are extracted before the classi cation layer, they have dimensionality equals to 2048 and they have been L2-normalized before to evaluate any metric on them. 4
In order to test the performance of the features extractor we used the IJB-B
dataset [
The resulting Receiver Operating Characteristic (ROC) for the 1:1 veri cation task is shown in Figure 4.
In order to evaluate the similarity among the di erent features vectors we measured the cosine similarity among them. As it is clear from Figure 4 the performance of the features extractor are degraded for lower resolution, especially below 32 pixels. Moreover in Table 1 we reported the True Acceptance Rate (TAR) at a reference value for the False Acceptance Rate (FAR) of 1.e 3.
ROC curve
Up until now, we have considered the case in which the images had the same resolution. In a real scenario it usually happens that probe and gallery have di erent resolutions. For example, in the case of surveillance systems it is common that the gallery database is populated with high resolution images descriptors while the probe has a lower resolution.
In Table 2 we reported the value for the TAR @ FAR equals to 1.e 3 for the cross resolution 1:1 veri cation task.
It is clear from Table 2 that the cross resolution face recognition task is very challenging for a deep neural network model, especially when the images have very low resolutions. 5
Surveillance systems require high performance from CNN-based systems on the face recognition task. Precisely, the deep models have to be robust with respect to variations of the input image resolution since it is usually the case that low resolution images, from surveillance cameras, have to be matched against a database containing deep features extracted from high resolution images. In fact, the descriptor extracted for each human face have to be robust with respect to resolution variations otherwise any kind of similarity search across the identities database will fail. We have seen that, even though there are models that perform well on the FR task, their performances suddenly drop when we used low resolution images.
Although we have shown the feasibility of a pipeline for FR based on deep models, we need to improve upon its performance especially in the case of mixed resolutions. Thus, we are planning a new training campaign focused on the low resolution domain below 32 pixels. Finally, we will pay particular attention to the case of FR in which probe and gallery have di erent resolutions. What we expect from such a campaign is that, even if we might obtain a small drop in the performance at high resolution, the improvement at low and mixed resolutions should outweigh that drop.