A method for recognizing the contours of objects in a video data stream is proposed. The data will be uploaded using the video camera. Objects will be recognized in real-time. We will use YOLO-a method of identification and recognition of objects in real-time. Recognized objects will be recorded in a video sequence showing the contours of the objects. The approach proposed in the project reasonably synthesizes methods of artificial intelligence, theories of computer vision on the one hand, and pattern recognition on the other; it makes it possible to obtain control influences and mathematical functions for decision-making at every moment with the possibility of analyzing the influence of external factors and forecasting the flow of processes and refers to the fundamental problems of mathematical modeling of real processes. The installation of the neural network is shown in detail. The characteristics of the neural network and its capabilities are shown. Approaches to computer vision for object extraction are shown. Well-known methods are methods of expanding areas, methods based on clustering, contour selection, and methods using a histogram. The work envisages building a system for rapid identification of combat vehicles based on the latest image filtering methods developed using deep learning methods. The time spent on machine identification will be 10-20% shorter, thanks to the developed new information technology for detecting objects in conditions of rapidly changing information.
Video surveillance is a common means of solving problems related to security and event monitoring [1–3]. One of the main tasks arising from video surveillance is detection [4], tracking [5] and identification [6] of moving objects. Video cameras are near us and record data about us. Therefore, there is a need to recognize data and objects. To recognize the data, you need to go through the stage of preprocessing, that is, improving the visual quality—increasing the contrast, distinguishing the boundaries, removing blurring, and filtering. Then there is an operation for the preparation of graphic images—selection of objects, segmentation, and selection of contours.
Tracking is determining the location of a moving object [7] or several objects over time using a video camera (Fig. 1). The algorithm analyzes video frames and outputs the position of moving objects relative to the frame.
Contrast enhancement
Blur reduction
Video stream preprocessing
The main tracking problem is matching the
positions of the target object in successive
frames, especially if the object is moving fast
compared to the frame rate. Thus, tracking
systems usually use a movement model [
Selection of objects
Segmentati on Examples of such simple movement patterns are flat object tracking—affine transformation or object image homography [9].
The target can be a rigid three-dimensional object, the motion model determines the appearance depending on its position in space and orientation.
For video compression, keyframes are divided into macroblocks. A motion model is a burst of keyframes where each macroblock is transformed using a motion vector.
The image of a deformable object can be covered with a grid and the movement of the object is determined by the position of the vertices of this grid.
When an object is to be searched and matched against a given one, a new set of key points is extracted into the test image the two sets are matched and a similarity score is calculated.
Before selecting an object from a video stream.
there are pixel-by-pixel, block-by-block, and
methods based on energy functional
minimization [
Pixel-by-pixel methods of object selection process all points of the image. These methods are highly accurate, but they are sensitive to noise.
Selecting an object from a video stream Pixel
Post-block
MInimization of the energy functional
Grayscale methods perform segmentation—
dividing a digital image into several sets of
pixels [
Block-based methods do not process
individual pixels [12], but groups of pixels
combined into blocks. If the block contains a
boundary, then in such areas the boundary of
the object is determined inaccurately [
The disadvantage of methods [15] based on the energy function is the low speed of operation. 2.2.
The methods of this group are based on the use of local features of the image [16]. The idea of the region expansion method is to analyze first the starting point, then its neighboring points according to the criterion of homogeneity of the analyzed points into one or another group. In more effective variants of the method, the starting point is not individual pixels, but the division of the image into several small areas. Each region is then checked for homogeneity, and if the result of the test is negative, the corresponding area is divided into smaller sections.
Threshold segmentation and segmentation [17] according to the homogeneity criterion based on average brightness (Fig. 4) often do not give the desired results.
Methods of expanding regions
By the criterion of homogeneity based on brightness
According to the texture
based homogeneity criterion
Threshold segmentation Such segmentation usually results in a large number of small regions. The most effective results are given by the segmentation based on the homogeneity criterion based on the texture [18].
In the video, heterogeneous objects are often observed, so you have to face the task of finding perimeters, curvature, form factors, specific surface area of objects, etc. All these tasks are related to the analysis of contour elements of objects.
Methods for highlighting contours in an image can be divided into three main classes: 1. High-frequency filtering methods [19]. 2. Methods of spatial differentiation [20]. 3. Methods of functional approximation [21] (Fig. 5).
Common to all these methods is the development of the boundary as a region of a sharp drop in the image brightness function ( ,), which is distinguished by the introduced mathematical contour model.
Selection of contours Methods of high-frequency filtering
Methods of
spatial differentiation
Methods of functional approximation. By the tasks, contour selection algorithms are subject to requirements: the selected contours must be thin, without gaps, and closed. The process of selecting contours is complicated due to the need to apply algorithms for thinning and eliminating gaps. However, the contours are not closed and unsuitable for analysis procedures. 2.4.
The -means method is an iterative method used to divide an image into clusters. The basic algorithm is given below [22]:
Step 1. Choose cluster centers, randomly or based on some heuristics.
Step 2. Place each image pixel in a cluster whose center is closest to that pixel.
Step 3. Recalculate the cluster centers by averaging all the pixels in the cluster.
Step 4. Repeat steps 2 and 3 until convergence (for example, when the pixels remain in the same cluster).
The distance is usually taken as the sum of squares or absolute values of the differences between the pixel and the center of the cluster.
The difference is most often based on color, brightness, texture, and pixel location, or a balanced combination of these factors. 2.5.
Histogram-based methods [23] are very efficient when compared to other image segmentation methods because they require only a one-pixel pass.
A histogram is calculated over all pixels in the image and its minima and maxima are used to find clusters in the image. Color or brightness can be used when comparing.
An improvement on this is to recursively apply it to the clusters in the image to divide them into smaller clusters. The process is repeated gradually with smaller and smaller clusters until the moment when new clusters stop appearing altogether.
Approaches based on the use of histograms can also be quickly adapted for multiple frames while retaining their single-pass speed advantage. 2.6.
You-Only-Look-Once (YOLO) [24] is an independent video object detection system that can operate in real-time at very high frame rates—the common limit is 45 frames per second, with a claimed useful frame rate of up to 155 frames per second. YOLO was originally released in 2015 with Facebook research.
YOLO consists of two main parts: a class detector and a framework detector. The class detector determines which objects are present in the image. The frame detector determines the location of objects in the image. The class detector works by using a regression neural network that learns to predict the value of a variable. It learns to predict the probability that a certain object is present in the image.
The YOLO class detector is a regression neural network with 24 deep layers. The input layer of the network receives a 448×448 pixel image. The output layer of the network contains 84 values. Each value corresponds to the probability that a certain object is present in the image.
Digital IP cameras are increasingly used in modern video surveillance systems. Connecting an IP camera to an already existing local network can guarantee minimal installation costs.
Let’s consider the characteristics that must be taken into account when choosing computer technologies for a digital video surveillance system.
The first characteristic is the number of physical ports to which other devices can be connected. Will this parameter determine the maximum number of IP cameras that can be connected? For a home video surveillance system, a switch that has 4 ports is often used. Equipment with 8–16–24 ports is used for professional systems [25].
The second characteristic is bandwidth. At the same time, the bandwidth of each port is taken into account. The most common values are 10/100 Mbps and 1 Gbps. It should be taken into account that often the total bandwidth of the switch can be lower than the total value of all ports. When choosing the bandwidth of a candle, you need to determine what data transfer rate your network can handle.
The third characteristic is the speed of data transmission, which will limit the possibility of receiving and transmitting information.
The fourth feature of PoE is the function that allows you to power other devices through the same cable that transmits data. This is very important for the organization of video surveillance, as it allows you to get rid of unnecessary wires, and also simplifies the process of installation and organization of the power supply of connected devices.
The fifth characteristic is management protocols. Yes, PoE switches are divided into managed and unmanaged. Managed switches are devices that support several protocols (functions) of network management and data transmission.
To build simple and small IP surveillance systems, physically isolated from networks in which other critical data is transmitted (telemetry data, banking and financial data, video conferences, etc.), it is possible to dispense with the use of unmanaged PoE switches.
Let’s install object tracking in the video stream and examine the speed of object detection.
To do this, we need to write the following command in the terminal: pip install ultralytics And then import it into the code: from ultralytics import YOLO
Now everything is ready to create a neural network model: model = YOLO(“yolov8m.pt”)
As mentioned earlier, YOLOv8 is a group of neural network models. These models were built and trained using PyTorch and exported as .pt files.
The first time you run this code, it will download the yolov8m.pt file from the Ultralytics server to the current folder. It will then construct a model object. You can now train this model, detect objects, and export them for use. There are convenient methods for all these tasks:
train({dataset descriptor file path})—used to train the model on the image dataset.
predict({image})—Used to predict the specified image, for example, to detect the bounding boxes of all objects that the model can find in the image.
export({format})—used to export the model from the default PyTorch format to the specified format.
All YOLOv8 object detection models are already pre-trained on the COCO dataset, which is a huge collection of images of 80 different types.
The prediction method accepts many
different types of input data, including a path
to a single image, an array of image paths, an
Image object from Python’s well-known PIL
library, and others [
After running the input data through the model, it returns an array of results for each input image. Since we only provided one image, it returns an array with one element, which you can extract like this: The result contains detected objects (Fig. 6) and convenient properties for working with them. The most important is the boxes array with information about the detected bounding boxes on the image (Fig. 7). You can determine how many objects are found by running the len function: After launch, “2” was received, which means that two boxes were detected: one for a mobile phone, and the other for a person (Fig. 8).
You can analyze each box either in a loop or manually. Let’s take the first object: The box object contains bounding box properties, including:
xyxy—coordinates of the box in the form of an array [x1,y1,x2,y2] cls—object type identifier conf—confidence level of the model regarding this object. If it’s very low, like<0.5, you can just ignore the field.
Let’s display information about the object: For the first object, you information: will receive Since YOLOv8 contains PyTorch models, the outputs from PyTorch models are encoded as an array of PyTorch Tensor objects, so you need to extract the first element from each of these arrays: Now we see the data as Tensor objects. To extract the actual values from a Tensor, you need to use the .tolist() method for tensors with an array inside, and the .item() method for tensors with scalar values.
Let’s load the data into the corresponding variables: Now we see the actual data. Coordinates can be rounded, and probability can also be rounded to two decimal places.
All objects that a neural network can detect have digital identifiers. In the case of the pretrained YOLOv8 model, there are 80 feature types with IDs from 0 to 79. The COCO feature classes are public. Additionally, the YOLOv8 result object contains a convenient names property to retrieve these classes.
The web application consists of 2 main files:
Flaskapp.py is a file responsible for the project itself, its appearance, and its structure.
YOLO_Video.py is a file that is responsible for the YOLO algorithm, namely for the implementation of object recognition in the video stream.
Implementation of the Flaskapp.py file: Configuring the Flask application:
A web application is created using the Flask class.
Configuration parameters such as the secret key and the file download folder are set.
Defines UploadFileForm class using FlaskWTF to handle file uploads.
Video processing functions:
Generate_frames and generate_frames_web functions are defined to generate frames based on output from YOLO detection.
These functions use the video_detection function from the YOLO_Video.py file to perform object detection on video frames.
Routes are defined for the home page (/ and /home), the webcam page (/webcam), and the video download page (/FrontPage).
The /video and /webapp routes are responsible for broadcasting video frames with object detection results.
The webcam and front routes render HTML templates for webcam pages and video uploads.
The UploadFileForm class is used to handle uploads of video files.
The application runs on the development server if the script is executed directly.
The video_detection function takes a video path as input and performs object detection using the YOLO model.
The YOLO model from Ultralytics is loaded from the specified checkpoint file (yolov8n.pt).
The frames of the detected objects on each frame are determined, and the processed frames are returned.
Class names corresponding to detected objects are defined in the classNames list.
Video capture and processing:
OpenCV is used to capture video frames from the specified path.
On each frame, detected objects are drawn along with class labels and confidence levels.
The processed frames are returned for streaming in the user’s browser.
The general course of work: • The user uploads a video file through the interface. • The file is saved and its path is stored in the Flask session. • The object detector is called from the received video path. • The processed video frames are transmitted for real-time viewing through the user’s browser.
This project uses Flask for the web application and integrates YOLO for real-time video processing. The YOLO_Video.py file isolates functionality related to YOLO, making it modular and reusable.
When entering the web application, we are greeted by the “title” page:
There are two buttons on this page:
The first Video button sends us to a page where we can upload a video, press the Submit button, and receive the processed video [27].
The second LiveWebcam button sends us to a page where the webcam is connected automatically and displayed on the screen in a processed format.
In the images Figs. 9–16 we can see that the YOLOv8 algorithm is running on the webcam.
Our model is based on pre-trained OSFA and is built on top of PyTorch. The training image size was up to 256×128. A batch size of 64 randomly selected data was then fed to the network. During the test, the test images are also resized to 256×128. Our model is trained on 100 epochs. The values of α1, α2 and the learning rate are the same as those set by OSFA. α1, α2 and learning rate are set to 1, 0.0007, 3.5×10−5, respectively. In SAM, the number of horizontal parts is 4. All experiments are performed with a hardware environment of 11th Gen Intel(R) Core(TM) i711800H at 2.30 GHz and NVIDIA GeForce RTX 3060. The data were taken from [28] to form the dataset. The protected technology was developed in [29], from where the protected communication channels are taken. The structural diagram of the model was taken in [30], and the use of methods in [31]. Also, the approaches used for image preprocessing were taken in [32] and [33]. Data processing was formed thanks to [34].
This study analyzed the YOLOv8 object recognition algorithm and its differences from other machine learning algorithms. A web application for object recognition in a video stream was also created, analyzed, and tested. A clear overview of the development tools for a web application using YOLOv8 has been provided. The PyCharm programming environment, the Flask framework, and its advantages compared to other frameworks were studied. Also explored is Ultralytics, which helps in the development and testing of web applications for video streaming.
As a result of the work performed, a high level of understanding of the YOLOv8 algorithm, its features, and its capabilities in the field of object detection in the video stream was achieved. The developed web application is not only a practical application of thialgorithm but can also serve as a basis for further developments and improvements in this direction.
P. Anakhov, et al., Protecting Objects of Critical Information Infrastructure from Wartime Cyber Attacks by Decentralizing the Telecommunications