The video surveillance market is one of the most popular areas in information technology. Therefore, the question of creating an effective approach to the construction of video surveillance systems, a method of optimizing subsystems and their correct adaptation, which was solved in this work, is acute. To solve this problem, the task was decomposed into user groups and software and hardware levels, existing tools in the field of application and their analysis were identified. An analysis of new and unpopular tools was carried out to check the expediency of their use. Thus, several main levels of the video surveillance system were created, each of which had separate tools for solving problems. The following levels are defined: equipment selection, solution topology, network optimization, use of data processing hardware resources, and visualization of system operation. Each of these sections carries a global solution to problems, namely, reducing financial costs, reducing the number of equipment, eliminating unprofitability, thereby ensuring the use of a smaller number of environmental resources for specific tasks and saving the time and efficiency of the work of specialists. As a result, a method of optimizing integrated video surveillance systems was developed for two blocks of users: the administrator and the operator. The method is modular, so its further development and filling with new technologies is possible in the future.
Today gives us more sophisticated security tools and methods, most processes become automated
and cyclical, but still require human attention. Therefore, the primary factor of any tool is the
efficiency of its work, while economic profitability, the issue of attracting technical and human
resources, and, of course, ease of use takes a back seat [
Since humanity has evolved, the life of each individual is an incredible number of tasks that are
usually performed in parallel. Urbanization has created a large number of commercial and state
structures, which are aimed at the same processes of improving life [
Modern video surveillance systems have many different tools for solving any problems, starting
with classic object monitoring and ending with the use of analytics and process automation. In
addition to hardware implementations, there are many software solutions, and the most complex
and at the same time effective are software and hardware complexes of integrated video surveillance
systems [
Returning to the issue of efficiency, it is worth clearly understanding what tasks this or that system performs and how quickly the received information is processed and delivered to the addressee or user [15, 16]. A good example of an effective system is video analytics, which allows you to analyze video images and quickly notify about a detected object, instead of a primitive approach to manually view streaming video or analyze archived recordings [17, 18]. The effectiveness of such systems has already been well researched. In the same work, the problems of optimizing such systems are considered, since at the current moment there is no single correct scenario for building a high-quality and efficient system, which would simultaneously be optimized according to various criteria [19–22]. Therefore, it turns out that the choice of users is between budget and at the same time ineffective complexes and, on the contrary, powerful solutions that are simultaneously incredibly unprofitable.
Optimization is the process of determining and applying the most beneficial characteristics of a certain action. The principle of this process allows you to improve work and make it even more effective, simple, and high-quality. As an example, we can turn to history, when farmers worked on plots of land, sowed grain crops with their own hands, looked after and harvested crops. Let's take the efficiency of such work as the "x" coefficient. Over time, farmers began to use livestock to simplify this process, then the speed of the task increased, and the resources - decreased. Now our efficiency has become greater - "3x". When specialized agricultural machinery came into operation, our conditional coefficient became "10x", or even more. So, in this case, the optimization of the process took place, which allowed: • • •
For the same amount of time as before, process a several times larger amount of work; Use fewer resources.
And optimization is not only about the speed of execution of work, it is also such characteristics as: • • • • • • •
Efficiency; Quality; Expenditure; Resource intensity; Profitability;
Benefit.
That is why, when building any process, a general basic plan is built first. After a certain number of integrations, blocks of tasks are defined that can be simplified and improved. And so gradually a simple plan becomes a complex, regulated and unified method.
Over the entire life cycle of video surveillance systems, a large number of optimization processes took place. First there were analog video cameras that recorded data on huge computers. The amount of information that could be recorded was limited by the capacity of the data storage facilities. Over time, hard drives appeared that were smaller in size and had a larger amount of usable volume. In addition to the evolution of data storage systems, new devices appeared - IR video cameras, the principle of operation of which was built on the basis of the TCP/IP protocol stack. These devices were supposed to solve a number of issues of optimization, formation and transmission of video data. Along with the appearance of IP devices, new concepts were formed, such as: codecs, bitrate, data transfer protocols and others. All these tools have become methods of optimization. Codecs provided the transmission of video data with the same quality, but with a smaller size, bitrate allowed to save bandwidth during information transmission, and transmission protocols unified the methods of connection and exchange of information between different nodes of the system.
Despite such a large number of tools that allow to simplify and improve processes in security systems, there is still a large number of problems for which a solution has not been invented. Separately, it is worth highlighting the process of problem formation and its initial formation, because it is derived from the process of creating new principles, tools and solutions. Taking the previously mentioned tools as an example, we will explain how the solution of some problems created new ones.
Since a codec is the work of a mathematical algorithm that detects similar blocks in a video image and decides which information to transmit to the end node and which not, then it is clear that an appropriate executive device is needed for encoding and decoding information. Such a device is the central processor in computers and specialized microprocessors on board cameras. For this method to work, it was necessary to equip them with an even more powerful CPU, which would allow processing such a volume of data. This became especially relevant when models with a larger image expansion (2 MP, 4 MP, 8 MP) began to be released. Decoding such a stream for display or using even the simplest motion detector has become a task for a segment of the most productive CPUs.
On the other hand, such a tool as bitrate does not have obvious large-scale side effects, and yet, in order to implement such a tool, it was necessary to further refine video cameras, make changes to the board structure, and add new executive devices for this algorithm.
Data transfer protocols also created inconvenience for the market, when unification requirements appeared already on the established technological process. From that moment, such protocols became mandatory for all devices, so each vendor had to integrate the SDK protocol and adapt it to their own device. The main problem was the inconsistency of protocol functions for each of the devices, especially for specialized devices with specific functions and modules. Therefore, for the full and effective implementation of such a solution, a conglomerate of various companies had to spend more than 10 years on its adaptation and optimization. Today, this solution looks not only effective, but also attractive to users of flexible and multi-vendor systems.
In summary, one feature can be highlighted that any optimization process is followed by a number of problems that will have to be solved. Therefore, the formation of the optimization process is a particularly difficult task when there are too many tools, each of them has its advantages and disadvantages, and the technically declared characteristics do not always correspond to the presented ones.
One of the most popular solutions among various security systems are video surveillance systems, as such a tool allows you to fully control, monitor and record remote, scattered objects. The main advantages lie in the elementary familiarization with the material, when in order to confirm the alarm status, it is worth analyzing the archive and determining whether this or that event is valid.
But, in addition to video surveillance systems, there are other solutions in the field of security, such as:
SFAS (Security and fire alarm system); PPS (Perimeter protection systems); Software tools for data analysis and reporting systems;
Network firewalls and traffic analysis tools.
The next stage in the evolution of video surveillance systems is the formation of a hardware and software complex with integrations and analytics. The main concept is to combine different security subsystems under one working interface and one software tool. In this way, administration, fixing, monitoring and calibration takes place through one entry point - the software complex. In addition to the fact that the system becomes monolithic and unified, it works within the same information space, which means that interaction between its modules can be configured. This is how the idea of introducing complex scripts with high-level languages was born. Now the execution of any process can be reduced to a minimum by prescribing clear cycles and rules for the operation of subsystems among themselves. For example, when receiving a specific event from the fire sensor, you can automatically turn on the alarm, give a command to turn on the floor sprinklers, automatically open the door for employees to leave, and analyze the video data through which you can check that all people have left the premises. This story can happen automatically, and its scenario can be many times more complicated. In addition to the presented format of automatic execution of procedures, it is important to work with the product interface, that is, how convenient and efficient the operator is to work with this system. Returning to the ephemeral situation, the operator, working with the interface windows, has before his eyes all the information about the operation of the subsystems. When the sensor on the interactive map is activated, the operator understands in which room the reaction occurred, and to confirm the alarm, the nearest camera can be turned on. If the alarm is really confirmed, the operator has the opportunity to send information to all units with a full description of the problem, a photo or video report and other visual elements with the help of one button. Thus, during critical situations, you can ignore the human factor and fully rely on the integrated system.
In addition to high efficiency in performing various processes, such systems are relevant for reducing financial costs. That is why these systems are the most popular and desired, because if such a project is financed, adapted and put into operation, the potential costs will be minimal and the number of people needed for its operation will be minimal. In some cases, the need for separate departments immediately disappears.
building an optimized video surveillance system
There is a large number of IR camera manufacturers in the security market, especially this has become relevant after the development of the Chinese market and the formation of new OEM solutions (existing solutions from companies that have given the opportunity to customize their equipment and software to third parties, which essentially creates a new brand, such as Linux OS distributions). Unifying the work with such devices has become a difficult task, and therefore in 2008 the world leaders of the CCTV market created a forum called ONVIF - Open Network Video Interface Forum, the main goal of which was to standardize all devices under one interaction protocol. This protocol has the same name as ONVIF (Figure 1).
The beginning of the solution to this problem of device disparity was set and solved gradually through the specifications of the profiles. To support this protocol, the software developer had to integrate this protocol as well as the vendors of the video cameras themselves. This story stuck for a long time, until the intensity of the development of the idea was not widely used. Primary profiles (Profile S) implemented common basic capabilities of interaction between cameras and software, namely streaming video transmission and its settings. The following profiles (Profile G and T) expanded the possibilities and allowed users to use: • • •
Codec H.265; Detailed setting;
Generate alarms on motion detectors;
As a result, in 2021, most products on the video surveillance market support this protocol and the main goal of the forum have been achieved. But during this time, there have been many changes that also require standardization, as a result of which a new M profile was announced at the end of 2021. Its specification will expand the possibilities of interaction with devices and gradually transform them into IoT devices. Further development of this profile will allow building smart homes, working with cloud storage and automating processes via MQTT (Message Queue Telemetry Transport).
For the work of neuroanalytical networks, it is necessary to choose the right tools, which will allow to get the most productive system, which would be flexible and optimal from the point of view of the use of resources. For such tasks, we decided to use technologies developed by Intel - OpenVINO.
The OpenVINO toolkit (or Intel Distribution of OpenVINO Toolkit) is an open-source toolkit that allows developers and data analysts to accelerate the development of analytical tools. This kit supports machine vision, optimizes the deployment of deep learning, and allows full use of Intel platforms for the implementation of analysis tasks. This technology also supports work with NVIDIA video cards and their specialized SDK - CUDA. Such a conglomeration of world leaders makes it possible to build powerful analytical systems, the potential of which will be many times greater than that of AMD and analogues.
The first thing to understand when considering this topic is the absolute use of neural network technologies on the side of graphics cards, as they are more productive by their structure. The main advantage of a GPU is that the graphics chip is designed to execute multiple threads at the same time, while the CPU core works with a stream of sequential instructions. If we take modern video cards, then we can say that they are multiprocessor because they consist of several computing clusters. Therefore, for tests of the optimization method, we will use OpenVINO tools and a video card with CUDA technologies.
After conducting research on a processor with support for Intel Quick Sync technology, we determined that the use of this technology is mandatory for operator workstations and individual server solutions.
First, it is worth considering what the above-mentioned technology is. Intel Quick Sync Video is a technology for hardware acceleration of video encoding and decoding, which is directly built into individual processors of the Intel family. In this case, this technology is implemented on an integrated circuit that specializes in specific tasks, i.e. hardware video decoders and video encoders are allocated in the graphics core, which in turn allows for faster and more energy-efficient video processing.
In my opinion, such technology is mandatory, because the results of the conducted tests show that processors of the same line and generation have a 2-fold increase in efficiency (Figure 2).
Data processing is a priority task, and without true and effective visualization, new processes for monitoring the performance of such modules and results are formed. The solution was to use the open technology of the company AxxonSoft - Dashboard. Dashboard is a specialized tool for building a visual display of data that can be flexibly configured and adapted to specific tasks. The main advantage of the solution is the updating of information online, which means that the information operated by the user of the system will be constantly relevant.
To determine the effectiveness of such a tool, we decided to take on the task that Ukravtodor sets before us and test how effectively we can use this tool. The practical application of this module will be described in the next block.
The construction of local systems has its advantages and disadvantages, which primarily concern the investment of funds in the software and hardware complex, and then its further support and development. For certain solutions that have reliable network connections, it will be relevant to use cloud technologies to build a video surveillance system with elements of data analysis. Despite the popular direction of this business, there are few representatives on the market, and indeed there are no high-quality professional solutions at all. As a result, users can receive online video viewing and archive recording without the ability to connect analytics.
We paid attention to the product of the AxxonSoft company - Axxon Datacenter, which is deployed on the Linux OS and builds a complete virtualization system for building dynamic clusters of data processing and storage. A striking difference of this service is the full integration of analytics and reverse work with system devices. For video analysis tasks, it is now not necessary to build your own mini-data centers, but you can use cloud technologies.
The advantages of such a decision are:
Use any analytics without hardware limitations; Absence of problems of constant maintenance and administration of the system; Increased system stability due to the use of a fault-tolerant cluster of virtual machines - nod; Continuous automatic update; Rejection of a physical complex set of subsystems; Data replication, which means that in case of loss of hardware capabilities, video data will be available on another service.
Among the shortcomings, only one critical requirement can be noted - a stable Internet connection, nevertheless, this task is solved by technologies Edge Storage Device.
The "GreenStream" function is a specialized algorithm for displaying video data at the operator's workplace. The principle of operation consists in the automatic selection of the stream from the camera to the server, and then its transmission to client workplaces depending on the resolution that is being played at the moment.
Such a tool is especially relevant for modern IP devices, because they allow broadcasting more than 10 different streams with different parameters at the same time. For example, streams can have not only a different resolution, but also a different frame rate, bitrate, codec type. The "GreenStream" technology was formed as a response to the problem of using high-performance servers specifically for client workplaces that display a large number of streams. Since the video data is first decoded before being displayed on the monitor, it creates a load on the central processor. If one volume of resources is spent on displaying one camera with 1920x1080 resolution, then nine times more resources are needed to display at least 9 cameras at the same time, but provided that the data is decoded with the same settings (resolution, codec, bitrate). When working with streaming video in low quality 640x480, resource consumption will naturally be less. Therefore, as a solution, a scenario was created, according to which when viewing a large number of cameras, a low-quality stream is displayed, because for the human eye, there will be no noticeable deterioration in quality on a layout of 4x4 cameras (16 in total). If the operator is interested in a specific video camera, when it is enlarged, the algorithm automatically switches the channel from a low-quality channel to a high-quality one (Figure 3).
As a result, the remote workplace does not require a powerful display computer, but this is not the only advantage of such a solution. The most important aspect of optimization is reducing the level of network bandwidth, since much lighter and smaller streams are broadcast instead of the main stream.
Several different methods have been invented to solve the problems of optimizing streaming video from cameras, the main goal of which is variability and flexibility. We will consider only 3 main approaches to obtaining a video image in a complex system: • • •
Receiving video data through transport layer network protocols - TCP and UDP; Multicast or multi-channel transmission;
Distribution of flows between recipients.
The simultaneous use of TCP and UDP is effective only if the strengths of these protocols are used. As you know, TCP (Transmission Control Protocol) works with confirmation of transmitted data packets, due to which this method of data exchange is reliable, but due to the long process of obtaining confirmation, it is slow. In turn, UDP (User Datagram Protocol), on the contrary, transmits information without confirmation and therefore it is faster than TCP, but does not have such reliability. Therefore, for maximum confidence in the integrity of the archive, it makes sense to use the TCP protocol for recording video, and for broadcasting video, you can use UDP, since the loss of one or two frames will not be critical for the observer.
Multicast. The use of this technology allows you to significantly reduce the load on communication channels when transferring data to a large number of end devices. To understand the effectiveness of this method, consider two popular methods of data transmission: unicast and multicast.
When using Multicast, the data stream will be broadcast by a specific recipient, due to which the corresponding number of copies of packets is created. If there are 4 such receivers, then there will be 4 copies. Multicast works with one copy of the package but using a specialized group of IGMP (Internet Group Management Protocol) receivers. When sending one packet to a router, a list of users to whom this message will be addressed will also be sent, and already at the level of network devices, these packets are addressed to end nodes (Figure 4).
Multithreading. As mentioned earlier, each camera can generate several different streams of video information with different characteristics. Accordingly, these flows can be configured in such a way that the main requirements are met for specific tasks. For example, to store an archive, you should get a high-quality stream, but you can reduce the number of frames, for the operator, create a stream with the maximum frame rate, but choose a video resolution lower than the main one. In order to reduce the load on the computing power of the server for analytics, the minimum permissible parameters of the video image are set.
The network infrastructure tends to fail, or its bandwidth is not always sufficient to transfer large volumes of information. This is especially relevant for the centralized model, where video cameras directly send packets to the main data processing server cluster, which may be located at a great distance. For such cases, the technology of built-in video camera storage is used. Some devices have the option of additionally connecting SD memory cards to record information directly from the device immediately to the medium. This allows you to build flexible systems without data loss because, if necessary, the operator can remotely view the recorded archive information, thereby not constantly occupying the channel with video transmission to the server/client.
An important aspect for the implementation of this tool is the support of the relevant functions of the protocol that regulates this process between the system and the end device. In addition to watching video, it is possible to create automatic rules for transferring video to the data center for its centralized storage. It is relevant to use such a solution, for example, at night, when the network channel is least loaded, and the system automatically copies data from the device little by little (Figure 5).
Having logical blocks, we have to choose the appropriate tools for each of them, which were considered in the work.
At the first level, it is worth setting up video streams and their parameters:
• • • •
For another group of users, such a module will consist of the following tools: 1. Selection of video cameras
At the second level, the process of setting up the Green Stream technology will take place. The third level is devoted to the selection of optimal hardware resources, namely:
Support for the integration of built-in analytics. 3. Network optimization tools:
• • •
Using specialized GPUs and Open Vino tools; Analytics on board cameras;
To build an optimized video surveillance system, the hardware and software tools listed in Table 1 were used.
AxxonNext software was chosen for the practical task as it supports all available codec integration protocols, Open Vino toolkit and multi-threading. The Wisenet device was chosen as the source of information, because such a camera has a complete list of the above-mentioned functions, which will allow testing all the advantages and disadvantages of each tool. Data processing will be carried out on two servers with different specifications. This approach is due to the testing of various technologies that will be supported on one device and will not be available on another. Such a stand will make it possible to apply methods empirically to the full extent and to record performance indicators of the equipment.
This system supports a large number of integrations and implemented solutions for VMS optimization. 6MP resolution / H.265/H.264 / WiseStream 2 / ONVIF Profile S/G/T Intel Сore I5-8400 / 16 RAM / Asus GT 640 Intel Core i7-9700/ 16 RAM / NVIDIA Quadro RTX 4000
The Windows operating system and a set of codecs were installed on the specified servers. The next stage is the installation of the software, which will conduct empirical testing of the described tools and issues and record their effectiveness.
To implement the tests, we will configure several streams with the following parameters and, accordingly, conduct tests on videos with different indicators of dynamic changes - the complexity of the scene. The following hardware resources will be used for the research: Intel Core I5-8400, Intel Core i7-9700, NVIDIA Quadro RTX 4000.
In the first stage, we will test the change in bitrate depending on the codec. • •
Video stream #1 - Resolution: 1980x1080; Number of frames: 25; Codec: H.265;
Video stream #2 - Resolution: 1980x1080; Number of frames: 25; Codec: Wise Stream 2; The next criterion that we will test is the effect of changing the number of frames on the same video streams, we reduce this parameter to 18 frames per second.
After completing the tests to determine the change in bitrate, we will take the same video archive and create layouts with different numbers of cameras for online display and check the effectiveness of using Intel Quick Sync technology. We will additionally configure a second video stream with similar parameters, but with a resolution of 360x240 to compare the download (Figures 7 and 8).
The next stage will be testing the optimization of the Green Stream tool. In the software, the automatic stream selection function will be enabled.
The final stage of optimization for the operator will be the use of a specialized tool and the creation of interface graphs to display information on the operation of detectors. To test such a task, we decided to take the issue of cameras installed by Ukravtodor and applied neuroanalytics to them. The main task for such devices was not informativeness and a large amount of noise. Using a general trained neural network for automobile vehicles, we configured line crossing detectors for vehicles and specified 2 size criteria - passenger car and truck (sizes were determined empirically in pixels). As a result, we had to trigger two detectors when passing vehicles with different dimensions. Working with a table that showed a list of triggers is not informative, when, in turn, graphs that are updated online allow you to familiarize yourself with the data during the day in a minute. The results of the work are shown in Figure 9.
The performance of such a task was possible with the help of the open Dashboard tool. In order to start working with it, you need to log in to the AxxonNet cloud service and go to the appropriate tab.
The display interface was written using JSON syntax and is shown in Figure 10.
The first was testing the effect of changing camera characteristics on the bitrate. The results are presented in Table 2.
The next study was about video decoding load testing on different hardware capacities, to determine effective workstation platforms (Table 3).
It is necessary to pay attention to the created interface objects, since this technology has made it possible to build an information space for operational familiarization with a useful volume of data. It is impossible to determine its effectiveness in relative numerical terms, but subjective judgment shows that the use of such a tool is mandatory in large-scale systems.
Video surveillance systems are a very popular segment in the IT industry, because, first of all, it allows you to ensure the safety of people and property. Separately, its capabilities are used for business automation, analytics and marketing. But, after analyzing several dozens of built projects, a conclusion was made about their unprofitability and inefficiency.
To solve this problem, the task was decomposed into user groups and software and hardware levels, existing tools in the field of application and their analysis were identified. An analysis of new and unpopular tools was carried out to check the expediency of their use. Thus, several main levels of the video surveillance system were created, each of which had separate tools for solving problems. The following levels are defined: equipment selection, solution topology, network optimization, use of data processing hardware resources, and visualization of system operation. Each of these sections carries a global solution to problems, namely, reducing financial costs, reducing the number of equipment, eliminating unprofitability, thereby ensuring the use of a smaller amount of environmental resources for specific tasks and saving the time and efficiency of the work of specialists.
As a result, a method of optimizing integrated video surveillance systems was developed for two blocks of users: the administrator and the operator. The method is modular, so its further development and filling with new technologies is possible in the future.
Thanks to the group of developers and management staff of the company "AxxonSoft" led by Oleksandr Kurinny for providing information support and the possibility of using the modern analytics algorithms for testing them and collecting results.
[14] Y. Averyanova, et al., UAS cyber security hazards analysis and approach to qualitative assessment, In: S. Shukla, A. Unal, J. Varghese Kureethara, D.K. Mishra, D.S. Han (Eds.), Data science and security, volume 290 of Lecture Notes in Networks and Systems, Springer, Singapore, 2021, pp. 258–265. doi: 10.1007/978-981-16-4486-3_28. [15] M. Purohit, M. Singh, S. Yadav, A. K. Singh, A. Kumar, B. K. Kaushik, Multi-sensor Surveillance System based on Integrated Video Analytics, IEEE Sens. J. (2021) 1. doi: 10.1109/jsen.2021.3131579. [16] V. Tsakanikas, T. Dagiuklas, Video surveillance systems-current status and future trends,
Comput. & Electr. Eng. 70 (2018) 736–753. doi: 10.1016/j.compeleceng.2017.11.011. [17] S. Hwang, J. Lee, W. J. Kim, S. Woo, K. Lee, S. Lee, LiDAR depth completion using colorembedded information via knowledge distillation, IEEE Trans. Intell. Transp. Syst. 23(9) (2022) 14482–14496. [18] A. C. Cob-Parro, C. Losada-Gutiérrez, M. Marrón-Romera, A. Gardel-Vicente, I. Bravo-Muñoz, Smart video surveillance system based on edge computing, Sensors 21.9 (2021) 2958. doi: 10.3390/s21092958. [19] M. Zaliskyi, et al., Heteroskedasticity analysis during operational data processing of radio electronic systems, in: S. Shukla, A. Unal, J. Varghese Kureethara, D.K. Mishra, D.S. Han (Eds.), Data science and security, volume 290 of Lecture Notes in Networks and Systems, Springer, Singapore, 2021, pp. 168–175. doi: 10.1007/978-981-16-4486-3_18. [20] T. Tran et al., Enabling multicast and broadcast in the 5G core for converged fixed and mobile networks, IEEE Transactions on Broadcasting 66(2) (2020) 428–439. doi: 10.1109/TBC.2020.2991548. [21] O. Sushchenko, et al., Airborne sensor for measuring components of terrestrial magnetic field, in: Proceedings of IEEE 41st International Conference on Electronics and Nanotechnology (ELNANO), IEEE, Kyiv, Ukraine, 2022, pp. 687–691. doi: 10.1109/ELNANO54667.2022.9926760. [22] N. S. Kuzmenko, I. V. Ostroumov, K. Marais, An accuracy and availability estimation of aircraft positioning by navigational aids, in: Proceedings of 2018 IEEE 5th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC), IEEE, Kiev, Ukraine, 2018, pp. 36-40. doi: 10.1109/MSNMC.2018.8576276.