Identifying faces in low-light situations can be a dificult feat because of the reduced visibility and substandard image quality. Conventional methods of face detection rely on visible light, which is insuficient in environments with low-light conditions. Our paper introduces a fresh approach to identifying faces in night vision. We make use of thermal infrared (IR) images to detect faces. Thermal IR images capture the thermal signatures of objects, which remain unafected by low-light conditions, providing valuable information for accurate face detection. Our method utilizes a deep learning model trained on thermal IR images to detect faces in conditions with low lighting. To evaluate our approach, we tested it on a dataset of thermal IR images captured in diferent lighting scenarios and compared its performance with traditional face detection methods. The results of our experiments indicate that our proposed approach surpasses traditional face detection methods in low-light conditions, achieving high accuracy in detecting faces. Through a qualitative analysis of thermal IR images, we delved into the key factors that lead to our approach's success. Our findings show that the thermal signature of the face ofers valuable insights that aid in precise face detection even in low-light environments. Furthermore, we assessed the efectiveness of our approach in detecting faces under diferent pose and expression variations, and the results indicate that our method is highly eficient in detecting faces in various pose and expression conditions. Our solution presents a novel and eficient method for detecting faces in low-light conditions using thermal infrared images, which has the potential to be utilized in various applications including surveillance, security and law enforcement.
Face detection is a critical task in computer vision with a wide range of applications such as surveillance, security, and law enforcement. However, detecting faces in low-light conditions is a valuable asset and is a challenging problem due to limited visibility and poor image quality. Traditional methods of face detection rely on visible light, which is inadequate in low-light environments. Thus, there is a need for approaches to detect faces in low-light conditions. The ability to see in the dark is a valuable asset in many situations, including security, law enforcement, and search and rescue. Traditional night vision methods are limited by the amount of light available, which can make it dificult to see objects in complete darkness, or in low-light conditions.
Night vision face detection is an area of research that has gained significant attention in recent years. Various approaches have been proposed for this problem, such as using visible light cameras with high-intensity infrared illuminators or active 3D imaging systems. While these approaches have shown promising results, they have some limitations, such as high power consumption, limited range, and high cost.
In recent years, thermal infrared (IR) imaging has emerged as a promising technology for night vision face detection. Thermal IR images are created by detecting the heat emitted by objects. This makes them ideal for night vision, as they can be used to see objects in complete darkness, which are not afected by low-light conditions, and can provide valuable information for face detection. Thermal IR cameras are also less afected by environmental factors such as fog, smoke, and dust, making them suitable for outdoor applications. However, using thermal IR images for face detection requires the development of new methods and models that can efectively utilize this modality.
In this paper, we propose a novel approach to night vision face detection using thermal IR images. Our approach involves the use of a deep learning model trained on thermal IR images to detect faces in low-light conditions. We evaluated our method using a dataset of thermal IR images captured in various lighting conditions and compared the performance of our approach with traditional face detection methods. Our approach has a number of advantages over traditional night vision methods. First, it is not afected by light conditions. This means that it can be used to see faces in complete darkness, or in low-light conditions. Second, it can be used to see through camouflage. This means that it can be used to identify people who are trying to hide their faces.
To investigate the factors that contribute to the success of our approach, we conducted a qualitative analysis of the thermal IR images. Our analysis reveals that the thermal signature of the face provides valuable information that can be used for accurate face detection in low-light conditions. We also evaluated the robustness of our approach to diferent pose and expression variations and found that our method is efective in detecting faces under various pose and expression conditions. Night vision face detection with thermal IR images has a number of advantages over traditional night vision methods. For example, it is not afected by light conditions, and it can be used to see through camouflage. Night vision face detection with thermal IR images has a number of potential applications in security and privacy. For example, it can be used to identify people in dark environments, and it can be used to detect people who are trying to hide their faces.
There has been a lot of research on night vision face detection in recent years. One of the most common approaches is to use a thermal IR camera to capture images of faces. The images are then processed using a computer vision algorithm to detect faces. One of the challenges of night vision face detection is that thermal IR images can be noisy. This can make it dificult for computer vision algorithms to detect faces. Another challenge is that faces can be obscured by objects, such as hats or sunglasses.
The method uses the reliable Hough transform approach to retrieve objects while filtering out
background activity noise. By using LC-Harris to extract 2D features, the depth of each detected
item is calculated. For eficient avoidance, the asynchronous adaptive collision avoidance
(AACA) method is used [
While the tracking phase incorporates mean shift tracking and Kalman filter prediction, the
detection phase uses a support vector machine (SVM) that uses size-normalized pedestrian
candidates. To improve the detection phase, the road-detection module verifies pedestrian
observations. using an onboard forward-looking infrared (FLIR) camera, an autonomous vehicle
detection system for pedestrians in poor light. To distinguish between infrared pedestrians,
low-level Haar-like characteristics are used, while the AdaBoost learning algorithm selects
the most pertinent information. A keypoint-based region of interest (ROI) selection approach
in IR imageries is suggested in order to eficiently scan sub-windows [
CNN designs employ hierarchical, discriminative processing for facial recognition and mark
regression. The hierarchical filtering approach uses both global and local filtering and addresses
the hard "face drifting" and "landmark shaking" challenges. The Landmark quality evaluation
system and the Kalman filter both ensure the stability and durability of regional face components
[
The Viola-Jones approach has the shortest detection time and an acceptable accuracy of
90.5±4.34% The pattern-matching method has an accuracy of 91.6±0.03%, while the
activecontour approach has an accuracy of 89.8±0.06%. A Viola-Jones and pattern-matching algorithm
combination might improve system accuracy [
The camera processes images using OpenCV software and a Raspberry Pi (RPI), a tiny
computer about the size of a credit card. It uses control algorithms to handle alerts and sends
recorded photographs through Wi-Fi to the user’s email. For both human and smoke detection,
the system has an accuracy of 83.56% and 83.33 % respectively. Comparing OpenCV and dlib,
the OpenCV library is more efective, performs better for face identification and detection, and
has greater productivity [
Multiprocess method based on networking middleware that enables real-time face tracking
and emotion identification in thermal infrared photos. Approch can detect facial landmarks and
recognize facial emotions, even when the faces move around. Furthermore, the approach can
recognize face and head postures, which increases its capacity to handle arbitrary head poses
[
Dynamic group diference coding is a screening method based on examining influencing
factors (D.G.D.C.). D.G.D.C. computes temperature diferences between the target person and the
recently passed crowd (dynamic group) by describing facial temperature with a face temperature
encoder (FTE) and constructing an embedding feature diference matrix. MLPs are used to
capture intrinsic information by describing the diference matrix in vertical and horizontal
directions [
In this section, we explain the approach to achieving our objectives by applying advanced
computer vision techniques to thermal-visual image data while constantly fine-tuning our
techniques through rigorous testing and experimentation. We will briefly overview the datasets
we use and where they come from to provide some context. Then, we will discuss the primary
methods we use, such as Dlib, Haar-cascade, and Yolo v8, and explain why they are important
for our analysis process.
3.1. Data
The dataset consists of a total of 2556 pairs of thermal-visual images. Each image pair features
participants with meticulously labelled facial areas, including manually defined face bounding
boxes and precisely positioned 54 facial landmarks. The dataset was constructed from our
large-scale (SpeakingFaces dataset) [
A complete and potent open-source software ml toolkit, the dlib library provides a wide range of features and capabilities for computer vision applications. The face identification capabilities of the dlib package are efective and precise. Modern algorithms for identifying faces in pictures or video streams are used. The Histogram of Oriented Gradients (HOG) approach and the Convolutional Neural Network (CNN)-based method are the two main face identification techniques ofered by dlib.
In order to capture the distinguishing characteristics of faces, the HOG-based face identification
method implemented in dlib makes use of the idea of local picture gradient orientations. It
entails calculating gradient orientations in tiny picture areas, creating histograms of these
orientations, and representing the local image structure using these histograms. To categorise
areas as either face or non-facial, these histograms are then given into a classifier, such as a
support vector machine (SVM). The HOG-based technique in dlib is renowned for its success
in identifying frontal and near-frontal faces and functions well in a variety of illumination
situations[
The gradient is obtained by combining magnitude and angle from the image. First Gx and Gy is calculated using the formulae below for each pixel value . (1) (2) (3) (4) where , refer to rows and columns respectively. After calculating Gx and, the magnitude and angle of each pixel is calculated using the formulae mentioned below.
(, ) = (, + 1) − (, − 1) (, ) = ( − 1, ) − ( + 1, )
Magnitude( ) = √︁2 + 2
Angle( ) = ⃒⃒ tan− 1 (/)⃒⃒
The CNN-based face identification technique in dlib makes use of deep neural networks to extract distinguishing characteristics from the raw image data directly. A convolutional neural network is trained using methods like convolutional layers, pooling layers, and fully connected layers using a sizable dataset of labeled face photos. The next step is to use the trained network to identify whether a section of an input picture contains a face or not. When detecting faces with complicated postures, occlusions, and scale fluctuations, the CNN-based method in dlib performs very well. It can record both subtle features and broad facial structures, producing extremely accurate face detection outcomes.
By examining the characteristics of an item, the Haar cascade approach uses machine learning to find it. It employs a collection of basic rectangular characteristics known as Haar-like features that are computed at various sizes and locations throughout a picture. The intensity changes between neighboring rectangular picture sections are the source of these characteristics.
The Haar-like features capture Local image fluctuations, which are then fed into a classifier
built on the AdaBoost algorithm—as input. AdaBoost creates a strong classifier by combining
a number of weak classifiers, successfully teaching the strong classifier how to distinguish
between diferent types of objects. The Haar cascade approach arranges the features in a cascade
structure, with a subset of all the characteristics comprising each step. Because the cascade
structure may swiftly reject portions of the picture that are unlikely to contain the item being
recognized, the processing is eficient[
At each cascade step, the classifier assesses the Haar-like characteristics and decides based on a threshold. The area moves on to the next step if the features meet the threshold for possible detection. If not, the algorithm rejects it and continues to the following area of the image.
At each stage, the Haar-like features are calculated for the image’s region of interest (ROI). Let’s denote the ROI as (, , , ℎ), where (, ) is the top-left corner coordinates of the ROI, and (, ℎ) is its width and height.
The diference between the sums of pixel intensities in two rectangular sections inside the ROI is used to calculate the Haar-like features. A single Haar-like feature can be described by the following formula:
= ∑︀ (pixels in the white rectangle) − ∑︀ (pixels in the black rectangle) where ∑︀ denotes the sum of pixel intensities.
A collection of weak classifiers, often based on straightforward decision functions, make up the classifier at each level. The weak classifier may be modelled as follows: ℎ( ) = {︃ if ≥ if < where ℎ( ) is the output of the weak classifier, and are the weights assigned to positive and negative detections, respectively, is the computed Haar-like feature, and is the threshold.
The outputs of the weak classifiers are combined at each stage of the cascade structure. Applying a combination rule, such as a weighted sum or a voting mechanism, to the outputs of all steps will yield the final choice.
An important development in real-time object identification, the YOLOv8 architecture provides
outstanding performance. The backbone, head, and neck, which together comprise its design
and are essential to its outstanding skills, are its three main parts. The backbone uses Powerful
convolutional layers to extract high-level information from the input picture, allowing the
model to collect detailed characteristics and semantic data required for accurate object detection.
YOLOv8 creates a strong foundation for following processing stages by using a backbone
network that is resilient[
The head module is essential for the extracted characteristics to be optimised and for reliable predictions to be produced. It uses cutting-edge methods to predict bounding boxes, class probabilities, and objectness ratings. The detecting head successfully accumulates object properties at multiple sizes using a variety of convolutional layers, including 1x1 convolutions, allowing for accurate localization and classification. The neck module in YOLOv8 also improves the model’s capacity to recognise objects of various sizes and aspect ratios. This intermediate component extends the backbone network’s feature collection by including extra context and semantic data. To further improve the model’s capacity to handle objects of various sizes and looks, feature pyramid networks (FPN) and spatial pyramid pooling (SPP) are often used for the neck module. (5) (6)
Our research proposes a hybrid approach that combines the strengths of Dlib and YOLOv8 for object detection, with the aim of optimizing performance in diverse lighting conditions. Initially trained a Dlib-based detector using a carefully annotated dataset collected specifically for low-light, night conditions. This model underwent thorough training and validation to improve its performance in nocturnal scenarios. The Dlib model was then integrated into the YOLOv8 framework, and additional training was carried out using datasets. In the second phase, we fine-tuned the model using a separate dataset captured in daylight conditions, allowing YOLOv8 to adapt and specialize for optimal performance during the day while leveraging the knowledge acquired during the initial Dlib training. This process leverages the strengths of each model to create a hybrid architecture that exhibits superior object detection proficiency in both day and night settings. We took care to maintain consistency in annotation format and hyperparameter tuning for enhanced accuracy. Comprehensive evaluations on independent test sets, including both day and night scenarios, validate the efectiveness of this hybrid model in achieving superior robust detection across diverse lighting environments.
Our results comprise using multiple models to perform better at night and day. Face detection can extend beyond day only or night only; the best way is to integrate multi-model architecture trained on various data domains. This multi-headed model can perform well in many conditions.
The U-net results can be observed in fig.3 for shape prediction of the face, which can be helpful in recognizing persons. The red dots show the predicted points, and the green points are the original points.
The dlib face detector is trained and tested on [
The dlib face detector, on top of its results, can be observed in fig.5 for detecting the persons
or people’s faces at night. It is trained with [
For daylight conditions, this model fails to predict all faces. Thus for the day condition, YoloV8 is used, fig 6. Which performs good for daylight conditions.
Training on a common dataset for day and light can lead to limitations like sometimes detecting noise instead of purely detecting faces, and some background clutter or occlusions can lead to failures in predicting. Combining both models can give face detection far better results than training on common ground, fig. 7 shows promising results.Our hybrid model combines day and night features through multiple models, resulting in a more robust model.
Testing revealed encouraging results for the face identification system built using the Dlib library, which combines HoG and CNN algorithms. The test error was slightly greater at 3.29 than the train error, which was measured at 2.72. This shows that the model has learned to generalize to new data quite efectively, even at night.
As we see in 1, it is clear that the Haar cascade algorithm exhibits a 48.15% accuracy in identifying faces in test photos that contain both day and nighttime faces, correctly identifying 13 out of 27 faces. However, the algorithm’s accuracy increases to 50%, correctly identifying 9 out of 18 faces, when examining only nighttime photos. These probabilities show the algorithm’s varying performance with respect to the time of day and indicate the need for more advancements to increase face detection rates in both situations.
The table 2 contains all the details for the day and night general case, comparing YoloV8, dlib, and hybrid models.
This research significantly contributes to the advancement of night vision face detection in thermal infrared (IR) imaging. A comprehensive examination of various techniques, including Haar Cascade, HOG+SVM, YOLOv8, and hybrid provides valuable insights for professionals in computer vision and surveillance technologies. Whereas the Yolo trained on the night data performs quite well than others, combining the weights of day and night light as a hybrid model can make a generalized and more accurate ground truth model. The study underscores the importance of method selection based on parameters like speed, accuracy, and computational demands while emphasizing the pivotal role of high-quality, diverse training datasets. The resulting improvements in accurate and eficient face detection systems for low-light environments have the potential to meaningfully enhance security and safety measures, aligning with the overarching objectives of this research endeavour.