The object identification system, given the correct model selection and settings, enables accurate and fast identification of graphical objects in video data. A deep learning neural network is the base for the identification system. The use of the CamVid benchmark video dataset for training the neural network model allows using of fundamental truth labels that associate each pixel with one of the 32 semantic classes of the identification system. The total number of used training images is 421, and the testing ones are 280. Selecting optimal parameters for the learning function and identification support, a method for measuring the distance between feature vectors gives the necessary result-the identification of objects from the video data stream of perimeter IP cameras demonstrated an average accuracy of 99.7% across all cameras in the test examples, consisting of 12 video fragments with a duration of 70 seconds each. The developed algorithm of the system is capable of identifying objects of 11 classes from the graphical information content of IP cameras.
The creation and implementation of object identification systems within various technical systems is explained by their demand in a large number of contemporary information systems.
Today, effective and high-quality identification of various objects can only be achieved by intelligent systems. These systems are based on artificial intelligence or machine learning algorithms. Frequently, neural networks in various variations serve as the basis for object identification systems. When using neural networks, it is crucial to correctly determine both the type of network and choose a test video database for proper training of the neural network [1, 2]. Otherwise, the accuracy of the classification in the created object identification system may be low. Another important aspect is the formation of the classifier’s gradation scale in the identification system.
If we need to analyze video data it is critical to create a function that analyzes video frames and utilizes test databases for initial object identification in individual static frames of the video sequence [3]. A separate challenge in object identification models based on neural networks is the classification of recognized objects [4].
Significant contributions to creating
algorithms and methods for object
identification have been made by: V. Kornienko,
L. Budkova, A. Korobov, A. Korobov [1],
V. Lakhno, V. Chubaievskyi, K. Palaguta [6],
V. Kornienko, I. Gulina, L. Budkova [7],
O. Kryvoruchko, A. Desiatko, A. Blozva,
V. Semidotska [8]. Publications on the use of
neural networks and machine learning
technologies for solving applied tasks in object
identification and cybersecurity are dedicated
to the research of the following scholars:
S. Schuster [9], L. Ljung, C. Andersson, K. Tiels
[
In most cases, overcoming the
aforementioned challenges allows obtaining a
correct mathematical model of the object
identification system. The same systems can be
used as a computer vision system or for other
practical applications. To obtain an object
identification system, it is crucial to obtain a
mathematical model on which such a system
will operate [
The advantage of a mathematical model based on neural networks is the ability to learn.
In computer intelligent systems for object identification, machine learning is a factor that significantly improves the adequacy and accuracy of recognition and identification algorithm performance. The most rational type of learning for the mathematical model of the created system is supervised learning. This type of learning involves using labeled or selected datasets that contain input data and expected output results. Thus, during the learning process, the model can perform internal iterations to approximate the specified error boundary. Once the learning boundary (error) ceases to exceed the specified limits, the model is considered trained.
Deep learning allows training a model to predict outcomes based on a set of input data.
For network training, both supervised and unsupervised learning can be used. The difference between these two learning methods is illustrated in Figure 1. In the neural network used as the basis for the object identification system, multiple layers of neurons are envisaged. The input layer of the neural network takes initial input data. In this case, there are four neurons in the output layer: intensity of each pixel, Haar features for each of the considered objects (trees, cars, roads, sky, pedestrians, etc.). The input layer passes this data to the first hidden layer.
Hidden layers perform mathematical computations with the input data. The term “depth” in “deep learning” refers to having more than one hidden layer. The output layer produces the final result. In this case, it is the identification of the type of object present in a specific image. The diagram of the obtained neural network model is shown in Figure 2. Identification of objects based on this machine learning method occurs through the reevaluation of the weights of connections between neurons. The weight factor determines the importance of the input data element. When classifying objects, the weight coefficient in interneuronal connections is the most crucial.
For the practical implementation of the created model, the vgg16() function in
MATLAB was chosen as the basis. This function represents the architecture of a deep neural We can define these two vectors as follows: network for image classification. It includes 16 = ( 1; 2; … ). ′ = ( 1′ ; 2′; … ′ ).
The number of filters in each convolutional truth labels associating each pixel with one of MaxPooling (2x2) layers are used after each free internet link. convolutional and fully connected layers.
The given function has 16 layers, including 13 convolutional layers and 3 fully connected layers. The overall architecture of VGG-16 includes: 1. Input Layer: Consisting of 224x224 pixels.
Layers: each convolutional layer has 3x3 filters and layer increases from 64 to 512. block of convolutional layers.
connected layers
Layers: three fully with 4096 neurons each. ReLU activation is applied to the output neurons of each fully connected layer. Dropout (random deactivation) may be applied to prevent overfitting. 4. Output Layer: the output layer has 1000 neurons (the
number of classes in
images).
The softmax activation function is used to obtain class probabilities.
The vgg16 function in MatLab returns a neural network object but does not include a specific method for measuring the distance between feature vectors.
There are some specific methods for measuring the distance between feature vectors: distance
squared; distance; Manhattan
Euclidean
distance;
Chebyshev distance; and hamming distance
[
In this research, the classical Euclidean distance is utilized. To calculate it, we use the following expression: ( , ′) = √∑( − ′)2, the second n-dimensional vector. where is the first n-dimensional vector, ′ is To train a neural network, prepared data needs to be fed into it, and the generated outputs should be compared with the results from a test dataset. For the training of the neural network, test examples from the video database labeled Cambridge (CamVid) were used.
The database provides fundamental ground 32 semantic classes. It was obtained from a This
database addresses the need for forming experimental data to quantitatively evaluate identification and classification algorithms. For each pair of objects, the “distance” between them is measured, representing the degree of similarity.
A model that provides a minimum of external criteria is considered optimal. With an increase in the number of variables in the model and the degree of the reference polynomial, obtaining the best forecasting model can increase significantly.
The obtained model is practically implemented in the MatLab environment. The aforementioned video database of test samples was used for training the neural network. This video database contains a 10-minute frame with a rate of 30 Hz. Images are segmented using corresponding semantic labels at a frequency of 1 Hz and partially at 15 GHz. The CamVid database has four datasets corresponding to the studied objects. They include: 1. Pixel-level semantic segmentation for more than 700 images (segmentation performed manually), later verified and confirmed by a second person for accuracy.
content. 2. High-quality color video images in high resolution, collected in the database. These images represent digital video recordings with a long duration of video 3. Contain calibrated sequences for color response and internal camera characteristics, considering the point of view, as a typical surveillance camera and
fixation of each frame in the sequences. 4. To support the expansion of the database, software is proposed for labeling (necessary to assist users who want to perform accurate labeling of classes for other images and videos).
The relevance of the database is evaluated by measuring the algorithm’s performance in each of the three different areas: object recognition in multiple classes, pedestrian detection, and label propagation. 3. Training and Testing Identification System Model the To ensure the operational functionality of the model, the following steps were taken: loading test datasets from the CamVid database sets and preparing a repository for test loading samples.
Declare individual classes of identified objects: “Sky,” “Building,” “Pole,” “Road,” “Pavement,” “Tree,” “SignSymbol,” “Fence,” “Car,” “Pedestrian,” “Bicyclist”.
The resolution of training frames is set to 360×480 points of the video stream: imageSize = [360 480 3].
A neural network model for the identification of graphical objects returns a specific set of numerical values for the identified objects. To obtain the central value of an ordered set of such data, we use the median () function in the Matlab environment. The initialization of parameters for neural network training and the definition of the error function for the neural network are provided in Table 1.
As we see from Table 1, the trainingOptions() function has 9 arguments: • sgdm (Stochastic Gradient Descent with
Momentum). • 'Momentum (momentum helps accelerate the optimization process by incorporating information from previous iterations). • InitialLearnRate (initial neural network learning rate). • L2Regularization (L2 neural network training regularization—weight decay); • MaxEpochs (number of full passes through the entire dataset during network training). • MiniBatchSize (mini-batch size, the number of examples used to update the gradient at each iteration). • Shuffle and every epoch (shuffling data at each epoch during training). • VerboseFrequency (the frequency of displaying training progress information in the command window).
Further, it is necessary to define the classifier’s grading scales to perform cluster analysis and the final classification of identified graphical objects. As reference data, we will take the informational component of the color channels of the identified image {R ∈ (0; 255), G ∈ (0; 255), B ∈ (0; 255)} this way: • Reference information vector for sky identification: [128 128 128; ... % “Sky”]. • Reference information vector for building and structure identification: 000 128 064; ... % “Bridge” 128 000 000; ... % “Building” 064 192 000; ... % “Wall” 064 000 064; ... % “Tunnel” 192 000 128; ... % “Archway”. • Reference information vector for identifying columns, pillars, etc: 192 192 128; ... % “Column_Pole” 000 000 064; ... % “TrafficCone”. • Reference information vector for identifying road surface: 128 064 128; ... % “Road” 128 000 192; ... % “LaneMkgsDriv” 192 000 064; ... % “LaneMkgsNonDriv”. • Reference information vector for identifying sidewalks, pavement, and pedestrian paths: 000 000 192; ... % “Sidewalk” 064 192 128; ... % “ParkingBlock” 128 128 192; ... % “RoadShoulder”. • Reference information vector for identifying trees, shrubs, and other significant vegetation areas: 128 128 000; ... % “Tree” 192 192 000; ... % “VegetationMisc”. • Reference information vector for identifying road signs, informational signs, etc.: 192 128 128; ... % “SignSymbol” 128 128 064; ... % “Misc_Text” 000 064 064; ... % “TrafficLight”. • Reference information vector for identifying fences and barriers: 064 064 128; ... % “Fence”. • Reference information vector for identifying vehicles: 064 000 128; ... % “Car” 064 128 192; ... % “SUVPickupTruck” 192 128 192; ... % “Truck_Bus” 192 064 128; ... % “Train” 128 064 064; ... % “OtherMoving”. • Reference information vector for identifying pedestrians, animals, and light means of manual cargo transportation: 064 064 000; ... % “Pedestrian” 192 128 064; ... % “Child” 064 000 192; ... % “CartLuggagePram” 064 128 064; ... % “Animal”. • Reference information vector for identifying light mechanized means of transportation for people and cargo (motorcycles/bicycles): 000 128 192; ... % “Bicyclist” 192 000 192; ... % “MotorcycleScooter”.
From the provided reference information vectors, it can be seen that some classes of the identified and subsequently clustered data contain subclasses, namely: • The ‘Building’ class contains 4 embedded subclasses. • The ‘Pole’ class contains 2 subclasses. • The ‘Road’ contains 3 subclasses. • The ‘Pavement’ contains 3 subclasses. • The ‘Tree’ contains 2 subclasses. • The ‘SignSymbol’ contains 3 subclasses. • The ‘Car’ contains 5 subclasses. • The ‘Pedestrian’ contains 4 subclasses. • The ‘Bicyclist’ contains 2 subclasses.
The color channel reference map for highlighting the classes of identified objects contains a color channel vector for 11 basic classes. Its representation is shown in Table 2. It is critical to define a reference map for color channels.
When training the model, the total number of training images is 421, and the number of test images is 280. As a result of training, the weight of each class can be determined (Table 3, column ‘Class weight’). Frequency characteristics of the occurrence of weights for individual classes on specific frames are also determined in column ‘IoU’ of Table 3. As shown in column ‘IoU’, there are any pedestrians and Bicyclist in the test image were identified.
Through training on the training set, the algorithm based on a deep learning neural network distinguishes the background from the informational content of identification (object—car) (Figure 3). As an example, to showcase the operation of the developed object identification system, a graphical frame depicted in Figure 4 has been loaded. As a result of the developed algorithm, we observe the identification results on the demonstration test frame of objects subjected to further cluster analysis. The legend for the identification algorithm classes is provided in Figure 5. The practical application of the developed identification system implies that the identification results were presented not in a visual (graphic) form but in the form of a numerical data array, allowing these results to be further used in more complex systems. To achieve this, we construct a histogram of the frequency of occurrence of identified classes and subclasses of objects in the identification zone of video surveillance cameras (Figure 6). By comparing deviations of the obtained numerical arrays with reference ones, the developed system for identifying graphic objects can make decisions regarding the detection of specific incidents or determine certain reactions of the system.
As a result of testing the implemented algorithm in the Matlab environment, the accuracy of the graphic information identification system model was proven to be high. The identification of objects from the video data stream of perimeter surveillance cameras demonstrated an average accuracy of 96.38% across all cameras in the test examples, consisting of 12 video fragments with a duration of 68.35 seconds each. These results were achieved due to several factors, including: • Type of neural network and thoughtfully chosen parameters of the neural network training function. • Method for determining the similarity measure of an object to existing classes (distance measure for clustering) through Euclidean distance. • Using the Cambridge (CamVid) labeled video database as a collection of videos with semantic labels of object classes, accompanied by metadata. This database facilitated effective training of [7] the foundation of the identification system—the neural network model. • Corrected definition of the classifier’s grading scales to perform cluster analysis and the final classification of [8] identified graphical objects and others.
As seen from the results of the image identification model, incorporating Haar’s features into the neural network model yields excellent results in the classification and identification of images of various types. [9]