Introduction

Face recognition technologies have emerged as a powerful tool for identifying individuals by analyzing their facial features. Over the recent years, face recognition has garnered substantial attention owing to its utility across diverse domains, encompassing applications such as security and surveillance, biometric authentication, human-computer interaction, healthcare, and other pertinent security-related fields [1]. As with most biometric applications, changes in appearance caused by an unconstrained environment tend to cause problems with face recognition. Let's consider some problems that need to be solved in the near future [1,2]:

1. Occlusion. A face can be photographed in an arbitrary pose in a certain environment and without any user input, so it is possible that the image will only contain a partial face.

2. Aging of the face. As the age gap between the query image and the reference image of the same person increases, the accuracy of recognition systems usually decreases.

3. A single sample. In real-world applications (e.g., passports, immigration systems), only one model of each person is registered in the database and available for the recognition task.

4. Video surveillance. Camera focus issues that can lead to image blur, low-resolution or compression errors, and blocky effects.

Face recognition technologies have become widely used in many applications [3]. Some of the most common areas of application nowadays are criminal investigations and identification of missing individuals. Moreover, face recognition technology can already be deployed on drones as part of special operations missions to help operators with intelligence gathering, reconnaissance, and identifying targets [4]. The fact of these means that face recognition technologies are increasingly being used in sensitive areas, as military operation, where any mistake can be crucial. Therefore, it is important to explore in detail information technologies of face recognition and identification to reduce the possibility of failure occurrence before they will be widely used in any purposes.

Task definition and solution methods

In our previous studies [5] there was introduced and explored a novel information technology for face recognition and identification based on local-texture descriptors. This technology relies on an algorithm that incorporates several methods such as Haar features, Gabor wavelet transform, local binary patterns in a 1-dimensional space and histogram of oriented gradients. These methods are employed for tasks of face localizing, image processing, extraction of feature vectors, respectively. More elaborate explanation of the algorithm's workings is provided as follows.

The algorithm takes a portrait image of a person as its input. This image is then transformed into grayscale and scanned to localize the face region. This localized face region is subsequently singled out for more targeted processing. To localize the face region the Haar features are used [6]. These features encompass a collection of elemental patterns, comprising both white and black blocks. Upon training the image features fj, it becomes feasible to derive the threshold value θj and the modulated comparability value modulo pj. The basic classifier can be described as:

ℎ 𝑗 (𝑥) = { 1, 𝑖𝑓 𝑝 𝑗 𝑓 𝑗 (𝑥) < 𝑝 𝑗 𝜃 𝑗 , 0, 𝑒𝑙𝑠𝑒.(1)

The outcome of applying the classifier to identify faces within an image comprises a collection of rectangles. These rectangles essentially represent a series of coordinates, denoting the corners of the region where a human face is positioned within the image. Following this detection, the original image matrix undergoes a reduction, retaining only the elements corresponding to the area where the human face is positioned.

Once the face region of the image is recognized, it undergoes processing through the Gabor wavelet transform [7]. This transformation is applied multiple times to the facial area image, varying certain parameters of the wavelet function. The objective is to aggregate all the outcomes to generate a comprehensive global facial representation of the person.

Within the Gabor representation [8], the arbitrary function F(x) is expanded by considering both symmetric and asymmetric elementary signals.

𝑆 𝑠 (𝑥) = 𝑒𝑥𝑝 [− (𝑥 − 𝑥 𝑚 ) 2 4𝜎 2 ] 𝑐𝑜𝑠[2𝜋𝑓 𝑛 (𝑥 − 𝑥 𝑚 )](2)𝑆 𝑎 (𝑥) = 𝑒𝑥𝑝 [− (𝑥 − 𝑥 𝑚 ) 2 4𝜎 2 ] 𝑠𝑖𝑛[2𝜋𝑓 𝑛 (𝑥 − 𝑥 𝑚 )](3)

These signals are centered at the position x = xm and at the spatial frequency f = fn with a Gaussian envelope described by the standard deviation 𝜎.

The complex Gabor function in the spatial domain can be denoted as follows:

𝑔(𝑥, 𝑦) = 𝑠(𝑥, 𝑦)𝜔 𝜏 (𝑥, 𝑦),

where s(x, y) is a complex sinusoid known as the carried and ωτ(x, y) is a 2D Gaussian function known as the envelope function. Complex sinusoid can be described with the following:

𝑠(𝑥, 𝑦) = exp (𝑗(2𝜋𝐹 0 (𝑥 cos 𝜔 0 + 𝑦 sin 𝜔 0 ) + 𝑃)).

(

)5

where parameters x • cos ω0 and y • cos ω0 define spatial frequency of the sinusoid in polar coordinates, F0 is magnitude and ω0 defines direction. Gaussian function can be defined as:

𝜔 𝜏 (𝑥, 𝑦) = 𝐾 𝑒𝑥𝑝(−𝜋(𝑎 2 (𝑥 − 𝑥 0 ) 𝜏 2 + 𝑏 2 (𝑦 − 𝑦 0 ) 𝜏 2 )),(6)

where (x0, y0) is the peak of the function, a and b are the Gaussian scaling parameters, and the subscript τ denotes the rotation operation as follows:

(𝑥 − 𝑥 0 ) 𝜏 = (𝑥 − 𝑥 0 ) cos 𝜃 + (𝑦 − 𝑦 0 ) sin 𝜃,

(𝑦 − 𝑦 0 ) 𝜏 = −(𝑥 − 𝑥 0 ) sin 𝜃 + (𝑦 − 𝑦 0 ) cos 𝜃.

A family of two-dimensional Gabor wavelets that satisfies wavelet theory and neurophysiological constraints for simple cells can be obtained using the following formulas:

𝜓(𝑥, 𝑦, 𝜔 0 , 𝜃) = 𝜔 0 √2𝜋𝜅 𝑒 − 𝜔 0 2 8𝜅 2 (4(𝑥 cos 𝜃+𝑦 sin 𝜃) 2 +(−𝑥 sin 𝜃+𝑦 cos 𝜃) 2 ) • [𝑒 𝑖(𝜔 0 𝑥 cos 𝜃+𝜔 0 𝑦 sin 𝜃) − 𝑒 − 𝜅 2 2 ],(9)

where ω0 is the radial frequency in radians per unit length and T is the orientation of the wavelet in radians. The Gabor wavelet is centered at the position (x = 0, y = 0), and the normalization coefficient is such that <ψ, ψ> = 1, i.e. normalized by L 2 . κ is a constant, with κ ≈ π for a one-octave frequency range and κ ≈ 2.5 for a 1.5-octave frequency range. The technique of employing local binary patterns in a 1-dimensional space [9] is harnessed to derive a feature vector from the comprehensive face image acquired following the Gabor wavelet transformation. This approach has demonstrated robust and efficient performance outcomes even when dealing with variations in angles of rotation and lighting conditions. It involves generating a 1dimensional row projection for each image matrix level, which serves as a descriptor for capturing and examining the texture within the facial image. The computation of the local binary patterns descriptor can be achieved using the following formula:

1DLBP = ∑n=0 N-1 S(gn -g0) • 2 n , (10)

where g0 is a value of the central element, gn is the value of 1-dimensional neighboring element and S(x) is defined with the following:

𝑆(𝑥) = { 1 𝑖𝑓 𝑥 ≥ 0; 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒.(11)

Based on preceding research, it was deduced that employing a single method for feature vector extraction is less effective when compared to their combination. Consequently, the decision was made to enhance the feature vector obtained from 1-dimensional local binary patterns with another vector derived through the creation of a histogram of oriented gradients [10]. The essence of this technique lies in preserving information regarding image shape characteristics within histograms that pertain to object boundaries found within sub-ranges of images post wavelet transformation. The count of object boundary orientations falling within specific ranges is depicted by each interval in the histogram. In the context of a grayscale image, derivatives are computed along both the x and y axes for every pixel. The magnitude of the gradient can be expressed as follows:

|𝐺| = √𝐼 2 (𝑥) + 𝐼 2 (𝑦). (12)

The orientation calculation can be described as follows:

𝜃 = 𝑎𝑐𝑟𝑡𝑎𝑛 𝐼 𝑦 𝐼 𝑥 . (13)

To form the comprehensive feature vector, both the resulting vectors from the 1-dimensional local binary patterns and the histogram of oriented gradients are combined. This resultant global feature vector can then be employed for subsequent person classification and identification based on the input image containing their face. The purpose of this research is to explore the conditions of image sample formation, to which the algorithm is applied, to investigate the algorithm performance in condition of its appliance to images captured in constrained and unconstrained conditions, as well as the ways of improving the resulting identification accuracy rates.

Experimental research

With an aim to explore and improve performance of the information technology for face recognition and person identification by face image, based on Haar features, Gabor wavelet transform, histograms of oriented gradients (HOG) and local binary patterns in one-dimensional space (1DLBP), it was decided to conduct the experiments on different databases, that contain images captured under different conditions of fixation regarding the intensity of lighting, the presence of cosmetics, makeup or occlusive elements, subjects' age variability, head postures and facial expressions variability, etc. For experimental research several databases were chosen such as Database of Faces (DoF, formerly "The ORL Database of Faces") [11], FERET (Face Recognition Technology database) [12], SCface [13,14], CFP (Celebrities in Frontal-Profile data set) [15], Tinyface [16], LFW (Labeled Faces in the Wild) [17] and AgeDB [18].

The initial set of experiments was focused on determining the efficiency of the algorithm, underlying in the basis of information technology of face recognition and person identification by face image under exploration, after its appliance to original face images containing in the databases. To conduct the experiment the face images of 40 individuals were used to form the etalon and test samples. Such number was conditioned by the minimum number of individual images of which are contained in one of the databases, which is the Database of Faces. To perform experiments more clear, the face images for the same number of individuals were selected from other databases as well. In Table 1 the obtained results of these experiments are presented. During the experimental research on original face images and after analyzing its results, it was established that on some images the algorithm is not capable to localize face region. Moreover, the efficiency of the researched algorithm on the face images from such databases as Database of Faces, FERET and SCface significantly exceeds the algorithm performance on the face images from CFP, Tinyface, LFW and AgeDB databases. For the set of first three databases the identification accuracy rate of the algorithm is greater on 70%-77.5% in comparison to other databases.

In order to explain such significant variation in algorithm performance results, it is worth to explore in detail the databases on the image from which the experiments were conducted. Database of Faces contains 92×112-pixeled face images of 40 people, which were captured on the dark uniform background in a vertical frontal position of the subject under conditions of varying lighting, facial expressions and facial details. FERET (Face Recognition Technology database) is a database that contains 1564 sets of 256×384-pixeled images of 1199 people taken during 2 years in a semiconstrained environment with the same physical settings for each session. SCface contains 4,160 static images of 130 people with the size of 75×100, 108x144 and 168×224 pixels captured on the same background, under unconstrained lighting conditions, with variation of fixed head positions. CFP dataset contains frontal and profile images for 500 individuals from open access captured in both constrained and unconstrained environments, with certain change of poses while other variations of image characteristics are unrestricted. Tinyface dataset consists of face images of 5,139 individuals with average size of 20×16 pixels captured under unconstrained conditions regarding background, lighting, face positioning and the presence of occlusion. LFW contains 250×250-pixeled images of 5,749 different people captured in the environment, that is as close as possible to the natural environment, and characterized with a wide range of variations in background, lighting, pose, facial expression, race, ethnicity, age, etc. AgeDB contains images for 568 individuals taken with huge age variety of the same subject in unconstrained conditions regarding poses, facial expressions, occlusion, and noise.

Analyzing the specifics of the conditions, under which the face images from all selected databases were captured, it can be concluded that the researched algorithm performed more efficiently during its appliance on the images captured in semi-constrained or constrained conditions regarding the background, lighting, subject's head position, camera position in relate to the subject, and other physical settings. Also, the images from the Database of Faces, FERET and SCface database are uniformed within the single database, as far as they were not taken in conditions close to the real-world conditions. On the other hand, CFP, Tinyface, LFW and AgeDB databases contain face images, that were captured in unconstrained conditions and more often are not uniformed within one database, as far as they were taken from the open access. Therefore, these databases contain images that are highly variative regarding the background, amount and intensity of lighting, head positions, occlusive elements, age and time intervals of capturing, makeup and cosmetics, etc.

So, considering the aforementioned problems, that significantly affect the performance of the algorithm under research, it was decided to perform the next set of experiments with several changes relatively to the first set of experiments. To overcome the problem of inability to localize face region on some of the images, the etalon and test image samples was reformed by extracting those images, in which the face region in not recognizable due to extreme angle of head rotation, excessive lighting or other conditions, that make face features not fully visible on the image. As for the images captured under unconstrained conditions, the etalon image sample was expanded with other images with recognizable face features of the same individuals, which face images were already in the sample. The results of experiments are presented in Table 2. Comparison diagram of described two sets of experiments is depicted in Figure 1. As can be seen, the extraction of images, in which the face region was not recognizable, allowed to increase the identification accuracy rate of the algorithm after its appliance on the images from the Database of Faces on 7.5%, FERET on 20%, CFP on 27.5%, Tinyface on 7.5%, LFW on 45% and AgeDB on 25%. Also, the expansion of etalon image samples for databases that contain images captured in unconstrained conditions has led to the improvement of the efficiency of the algorithm on 15% for the CFP and AgeDB databases, while result for Tinyface database remained the same and for LFW database decreased on 10%.

Obtained results of experiments are varying in the wide range of identification accuracy rates. Such variability may be caused by the fact that the selected databases contain images with different file format and resolution. Therefore, it was decided to conduct the experiments in order to determine whether it possible to improve the algorithm performance by eliminating image properties variability and by that, possibly, reduce the range of algorithm performance results. Also, as was described in our previous research [19], variety in quality of the images, which is as well determined by format and resolution, may cause the failure of identification process if images in etalon and test image samples are not unified. On the assumption that such properties of images as format and resolution may affect the researched algorithm performance, it was decided to convert original images from selected databases to the most common formats and resolutions and those on which the algorithm efficiency was the highest among all sets of experiments and conduct the experiments by applying the algorithm on converted images.

Figure 1: Experimental results obtained on initial samples with face region extractable images in

To perform the experiments on format-converted images the following formats were selected: JPG, PNG, and BMP. Experimental results obtained after appliance of the researched algorithm to the samples of images with format conversion are presented in Table 3. Comparative diagram of above-described set of experiments demonstrated in Figure 2. Analyzing the results, it can be concluded that the format conversion was effective only after appliance of the algorithm under research to the samples of images from Database of Facesthe identification accuracy rate of the algorithm increased on 5% after converting images to JPG format. Also, variation of results is observed in case of LFW-efficiency of the algorithm has decreased on 5% in case of expanded etalon images sample, while the rate of identification accuracy remained the same in case of initial number of images in the sample. The performance of the algorithm after its appliance on the images samples from other databases have not been influenced by format conversionidentification accuracy rates for samples of images from those databases remained stable for all experiments. Nevertheless, even such minor changes in identification accuracy rates indicate that the format conversion in individual cases of the algorithm appliance can be crucial for success of identification process. The next set of experiments was conducted with resolution conversion. The values of resolution were chosen with regard to those images, on which the previous experiments showed the highest rates of identification accuracy. Since all of the selected databases contain images with different resolutions, and simultaneous change of height and width may cause the alteration of face features depicted in the image, which are essential for successful performance of the algorithm, it was decided to automatically define the width value of image resolution in relation to the value of height that was set as constant.

According to this, the following values of resolution were chosen to form the image samples: width ×91, width × 100, width × 128, width × 144. Results obtained on resolution-converted images are presented in Table 4.

Analyzing the results, presented in the diagram in Figure 3 and obtained after appliance of the researched algorithm on the samples of resolution-converted images in compare to the results of previously described experiments, as can be seen from the identification accuracy rates of the algorithm changed in the following way for the samples of face images from databases. Database of Facesincreased on 5%-10%, FERETdecreased on 5%-35%, SCfacedecreased on 10-30%, CFPincreased on 5%-15% on initial and 5-10% on expanded samples, Tinyfaceincreased on 10-15% on initial and 25-35% on expanded samples, LFWdecreased on 10-20% on initial sample and increased on 25% on expanded sample, AgeDBincreased on 10-20% on initial sample and decreased on 5-10% on expanded sample. Also, after resolution conversion preserving the aspect ratio, some of the images were not face features extractable.

Conclusion

This work is devoted to the research of information technology of face recognition and person identification based on such methods as Haar features, Gabor wavelet transform, histograms of oriented gradients (HOG) and local binary patterns in one-dimensional space (1DLBP). The purpose of the research is to improve the performance of the algorithm underlying in the basis of information technology by exploring the conditions of image sample formation and effect of image properties on the algorithm efficiency.

To conduct the experiments there were selected several databases of face images. During the research it was found that the performance of the algorithm is affected by the presence in the etalon and test samples of images, in which it is difficult to localize face region due to such conditions of image capturing as extreme angle of rotation of a subject's head, excessive lighting, etc. Also, algorithm performance can be reduced during its appliance on samples of images captured under unconstrained conditions regarding the background, lighting, subject's head position, camera position in relate to the subject, and other physical settings. After extracting from the etalon and test samples of images with unrecognizable face regions and expanding the etalon samples of image from databases that were formed under unconstrained conditions, the accuracy identification rate of the algorithm was improved for images from the Database of Faces on 7.5%, FERETon 20%, CFPon 27.5%, Tinyfaceon 7.5%, LFWon 45%, AgeDBon 25%.

The next sets of experiments were performed with conversion of such image properties as format and resolution with an aim to explore the possibility to reduce the variety of algorithm performance results on samples of images from different databases. As a result of experiments, it was established that format of images, to which the explored algorithm was applied, in individual cases affects the efficiency -the identification accuracy rate increased on 5% after converting images from the Database of Faces images to JPG format and decreased on 5% in case of expanded etalon sample of images from LFW database. The conversion of resolution has also affected the algorithm performance. The results for were increased on 5%-10% for samples of images from the Database of Faces, decreased on 5-35% for samples from FERET, decreased on 10-30% for samples from SCface, increased on 5%-15% on initial and 5-10% on expanded samples from CFP, increased on 10-15% on initial and 25-35% on expanded samples from Tinyface, decreased on 10-20% on initial sample and increased on 25% on expanded sample from LFW, increased on 10-20% on initial sample and decreased on 5-10% on expanded sample from AgeDB.

The highest identification accuracy rate, which is 95%, was obtained on image sample of initial images from SCface database and on image sample after reforming of the etalon image sample by extracting images with unrecognizable face region.