An eficient computer system for regulating and monitoring the density of people in confined areas is very helpful. It becomes imperative to implement a solution that takes into account the processing power and pre-installed hardware in these places. Using computer vision, in particular, to make use of regular CCTV cameras that have been augmented by neural networks, solves the problem of precisely counting individuals in enclosed spaces. We describe a control system specifically designed for this goal, maximizing the capabilities of current infrastructure and enhancing neural networks to achieve higher frame rates.
in the neural networks field. It’s possible to adapt such
a framework to a series of diferent tasks, in
particuIn many enclosed spaces, crowd capacity management lar, it’s broadly used in the Super-Resolution of signals,
is a common challenge due to strict occupancy limits. such as images, videos, and audio and, generally
speakThese limits are critical for safety and regulatory compli- ing, in recreating or reconstructing parts of lost signals.
ance. To address this issue, we propose to leverage CCTV Given the potential of this framework, we decided to
cameras as a solution to more accurately count people implement a GAN regarding the frame-rate increase of
within a confined area. Leveraging advanced video ana- CCTV. The whole project tries to exploit the best
techlytics, our system aims to provide real-time monitoring, niques that require a saving of hardware resources, thus
helping companies and institutions maintain optimal au- allowing it to be used in as many environments as
posdience density and ensure a safe environment [
Some research [
• YOLOv8n trained with 416 × 416 images
• YOLOv8s trained with 416 × 416 images
• YOLOv8m trained with 416 × 416 images
The computer vision community has given significant
attention to the necessity of increasing the frame rate
and, consequently, the video frame interpolation. Many
uses for this issue exist, including the creation of slow 3.2. Tracking
motion and frame recovery for video streaming and
gaming. High-frame rate videos are visually more pleasing to To track people in the scene as reliably as possible, it’s
watch because they may avoid typical glitches like tem- needed a good balance between accuracy and speed;
poral jittering and motion blurriness. Several techniques while the chosen model ofers a good speed in the
detechave been used to overcome the issue of getting interme- tion, it lacks accuracy. To make up for this lack, we
cordiate frames from a limited collection, including frame rected and smoothed the predictions made by YOLO
usinterpolation and, more recently, DNNs. In Frame Inter- ing the SORT algorithm [20], which corrects and smooths
polation techniques, intermediate frames are generated the position of the bounding boxes using a Kalman filter
between the present frames using interpolation, as in the [21].
methods proposed by Choi et al. [
This section outlines the approach for obtaining the camIn this section, we describe the methods and the algo- era matrix and the algorithm employed to determine the rithms used to analyze the images and detect people 2D position of a person in the scene. inside the scene, and after that increase the frame rate.
YOLO [18] is the neural network framework we used for detecting persons in the scene, it is extremely popular and
The finite projective camera, denoted as , is characterized by its intrinsic and extrinsic parameters, given by: = [ | − ˜] = [| − ˜] Here, describes the orientation of the camera and ˜ is the world position of the camera center. is the calibration matrix and since the resolution is the same in both the x and y directions, the calibration matrix can be defined as: to train them at the same time, improving their performances to obtain a good model that generates the missing frames.
The generator takes as input two pictures of size ( ×
× 3). To minimize its dimensions, the encoder
employs two-dimensional convolutional layers with a stride
with being the focal length andcan be obtained using of two using a UNet [24]. LeakyReLU is the activation
the formula: function, and its slope is 0.2. On the other hand, the
decoder uses the LeakyReLU activation function with a
= 2 * 2( ) slope of 0.2 and consists of several 2-dimensional
con2 volutional layers with a stride of 2. The ℎ activation
Where is the field of view. Typically, obtain- function is used in the final output layer to make sure that
ing these parameters requires camera calibration using the outputs are inside the [
Summarizing, the 3 × 4 camera matrix transforms image coordinates (, , 1) to scene coordinates (, , , 1) . To obtain the scene coordinates from image coordinates, we aim to invert , considering that perspective projection is not injective. Assuming knowledge of the distance from the ground (height of the person), we utilize the pseudo-inverse + of . Two points on the back-projected ray are identified: the camera center and the point +. The ray is expressed as: ( ) = + +
Within our generative adversarial network (GAN), an adversarial discriminator D seeks to maximize the objective function, while the generator G strives to decrease it, resulting in a zero-sum game. The definition of the objective function is:
ℒ (, ) = = E,[(, )] + E,[(1 − (, (, )))]
The optimal generator denoted as * is determined by:
For a finite camera with = [ |4], the camera * = arg min max ℒ (, ) center is ˜ = − − 14. Back-projection of an image point intersects the plane at infinity at the point = We enhance the GAN objective function by adding the L1 (( − 1) , 0) , providing a second point on the ray. loss function, which is a conventional loss. The generaThe line is represented as: tor’s job is now to provide nearly optimum outputs using this conventional loss function, in addition to tricking ( ) = ︂( − 1(1 − 4))︂ tdhuetyd.iTschreimLi1nlaotsosr,, dweinthooteudt achsaℒn1giinsgdethfineeddiassc:riminator’s Solving for , considering the coordinate as the detected height, allows computation of the and coordinates in the scene.
We decided to implement a GAN solution for our framework, based on the Image2Image work [23]. The framework is composed of two models: a generator and a discriminator; the generator takes as input the frames and +1 and tries to infer the missing frame , while the discriminator takes the same input concatenated either with the real missing frame or with the generated one, to classify them as generated or real. The goal is ℒ1() = E,,[|| − (, )||] And now our final objective function is: * = arg min max ℒ (, ) + ℒ1() Here serves as a weighting parameter for the ℒ1 loss.
In this section we will describe the implementation details of our work, starting with the setup and the preparation of the simulator, the training phase of the neural network, and the whole system architecture.
GAN Network architecture Layer
Activation
Input Conv Conv ...
Conv Input Conv Conv ...
FC LeakyReLU LeakyReLU ...
Tanh LeakyReLU LeakyReLU ... Where the average of x; the average of y; 2 the variance of x; 2 the variance of y; the covariance of x and y; 1 = (1)2, 2 = (2)2 two variables to stabilize the division with weak denominator; L the dynamic range of the pixel-values (typically this is 2# − 1); 1 = 0.01, 2 = 0.03 by default. = 20 · log10 ︂( MAX {} )︂ √MSE Where MAX {} is the maximum possible pixel value of the image and with the mean square error (MSE) defined as: = =0 =0 1 − 1 − 1 ∑︁ ∑︁ ‖I (i , j ) −
K (i , j )‖2
Let I represent the original image and K denote the generated image, both of dimensions MxN. The results of our network, in comparison with other methodologies, are presented in Table 2.
Results compared with S.O.T.A networks This system can also be used with multiple cameras; when working with multiple cameras, each camera receives an image and elaborates that using YOLO and SORT, to extract the bounding boxes positions. Each
The whole project was developed using Python v3.8.10. For the detection and tracking part the following libraries were used: • OpenCV v4.5.2 compiled from source, to activate the ability to use CUDA drivers and CUDNN, obtaining faster results with YOLO.
• Numpy v1.21.4 For the neural network creation, training, and testing we used: • TensorFlow v2 • Keras for the creation of the layers • OpenCV for the pre-processing of the dataset and the data augmentation • Matplotlib to visualize our results
For training our network, we used the EPFL [25] dataset, which includes multiple scenes of moving pedestrians.
The training data were extracted by taking 3 frames at a time and adding noise to increase the number available. Table 2 Next, each triplet was saved in a file, with the first and last frames as input to the generator and the middle frame Net as reference. The dataset was divided into validation, training, and testing. The GAN network was trained EpicFlow[26] using the early stopping technique thus preventing the BeyondMSE[27] network from overfitting the data. The loss graph is MOCunreNt+eRtwESo[r2k8] shown in Figure 1 for the Generator and Figure 2 for the Discriminator.
To study the results of our neural network, we computed the SSIM and PSNR values which are used to mea- 4.3. System architecture sure the similarity between two images and are defined as: SSIM
frame is passed to the detection thread and can be stored, to be processed later by the Neural Network. The points centered in the top part of the bounding boxes generated In this section, we will show the results obtained. by the detection threads are passed to the camera models, to obtain the position of the persons on the plane. 5.1. Frame Rate and Crowd Counting Those positions are then merged by searching for each camera the nearest neighbor and in case of a mismatch between the number of people in the cluster, the bigger one is chosen; after matching is found, for each person, a dot is drawn on the map having the average position between the matched one. The whole system architecture is represented in the Figure 3.
As we can see in figure 4, the first and the last frame are the input, while the middle one was generated by the Generator of the GAN network.
After a series of comprehensive tests, our technology performed smoothly when properly identifying and counting people in enclosed spaces. With the addition of computer vision algorithms and the advances made possible by our improved neural network, accurate people counting and identification are guaranteed. The outcomes demonstrate the system’s capacity to monitor and control crowd density in confined areas eficiently. For a visual depiction, Figure 5 shows how our model could be used in a real-case scenario using only one camera.
In summary, our methodology ofers a dependable and precise means of detecting and measuring human beings in enclosed spaces. By utilizing the creative fusion of GAN-based networks and the efectiveness of lightweight YOLO models, our system not only ensures robustness but also demonstrates flexibility to operate on systems with limited technological resources. This clever approach strengthens security protocols and expedites operational workflows in addition to ofering a ifnancially sensible way to implement occupancy restrictions in a variety of scenarios. Our method, which makes use of cutting-edge AI technologies, is a big step toward improving space management and guaranteeing adherence to safety laws, making all people’s surroundings safer and more efective. Moreover, it is a useful tool in circumstances where precisely counting people is necessary to avoid crowding, making the environment safer and more efective for everyone. Our method, which makes use of cutting-edge AI technologies, is a big step toward improving space management and guaranteeing adherence to safety rules, which will eventually improve the general standard of public areas and facilities.