It is shown that recently the development of systems for remote information exchange has been gaining momentum. The article shows that there are several standards for encoding video information streams. At the same time, for real-time applications, which include video conferencing communication, technologies with basic profiles are most often used. However, the disadvantages of this direction of video compression are that it has a relatively low compression ratio relative to full-profile implementations of standardized codes. This is a restriction on the use of these codecs in real-time applications with low bandwidth of the telecommunications segment. Therefore, standardized pipelines for processing video information streams need to be improved in the direction of decom, updating the structure of the overall stream and using new compression methods without loss of image quality.
Recently, the development of remote information exchange systems has been gaining momentum. This is
facilitated by the factors of increasing the productivity of information technologies, telecommunication
networks, and the algorithmic implementation of many applied areas of information processing and
analysis. Including artificial intelligence technologies. However, problematic moments arise here
[
Therefore, there is a problem regarding the need under modern conditions to ensure the improvement of the quality characteristics of the provision of video information services in the remote mode using the limited capacities of the telecommunications sector.
The direction of solving such problem is to increase the efectiveness of methods and algorithms for
processing and analyzing multimedia information flows [
Characteristic here is the improvement of multimedia data compression methods. This allows to reduce the number of bits that are transmitted per unit of time between users using telecommunication networks. Therefore, the purpose of the article concerns the research of existing compression technologies and the justification of the direction for their further improvement.
There are several standards for encoding video data and video information streams in particular. At the same time, for real-time applications, which include video conferencing, technologies with basic profiles are most often used. In this case, coding is carried out for individual frames taking into account their classification according to three types: I, P, B. A typical configuration of codecs for individual frames is similar to algorithms in a JPEG pipeline. Frames after pre-processing according to the classification requirements in the frames group are processed independently of each other. This mechanism provides high-speed access to arbitrary frames both in the forward and reverse order of processing. Accordingly, smooth "rewinding" in both directions, audiovisual synchronization are easily realized. Opportunities for editing are also created. Common JPEG operations are now supported at the hardware level by most graphics cards. Independent processing of individual frames allows to apply diferent efects without fear that the mutual influence of adjacent frames will introduce additional distortions into the overall video content. The advantages of this coding approach are that it: provides fast arbitrary access; easily edits the stream of video information; has a relatively simple hardware implementation; is based on a set of transformations that are implemented on the hardware and software scheme.
The disadvantage of this direction of video data compression is that it has a relatively low degree of compression compared to full-profile implementations of standardized codes. Therefore, there is a need to improve the components of standardized codecs.
As a basic standardized configuration, it is proposed to use MPEG-4, H264 and H265 concept implementations. Let’s consider the algorithm for processing three-dimensional streaming video information. The MPEG code structure consists of several hierarchical levels. This allows an increase the quality characteristics of the coding and compression process and reduces the complexity of the video stream analysis process. So there are the following levels (fig. 1).
The first level is its own stream of video information. The third layer of data flow is a layer of individual frames of diferent types. The fourth level is sectional. A section (its usual width is equal to the width of the frame) consists of a certain list of macroblocks with a size of 16 x 16 pixels.
The decoder as a component part includes a block of visualization of three-dimensional objects Animation Framework extension - AFX - what is called data for three-dimensional movement. Here, the background and fragments containing objects are processed separately. Accordingly, the implementation of slice processing is used to take into account static areas in the sequence of frames. Such flexible work with three-dimensional objects allows significantly increase the degree of compression with significantly better image quality.
But such advantages are accompanied by a significant increase in the complexity of the algorithmic implementation and construction of the computational process. This leads to significant delays in the process of compressing video information streams. Accordingly, this is a limitation on the use of the specified codecs in real-time applications with low throughput capabilities of the telecommunications segment.
Let’s consider the study of ways to improve the components of the standardized pipeline for processing video information streams.
The method of coding the information flow without loss is a method in which there is no loss of quality of the information flow compared to the original. An unencoded information stream is mathematically identical to its original. Lossless data stream processing usually provides lower intensity reduction factors than lossy intensity reduction.
Lossy stream encoding methods are methods that sacrifice some quality of the stream in exchange for a reduction in data size. The amount of degradation depends on the coding algorithm used and the quality factor set by the user. These methods are based on eliminating the redundancy of the information flow in accordance with the properties of the human visual system. When applying these methods, the decoded information stream difers from the original information stream. That is, there is a distortion of the information flow and, accordingly, there is a loss of information.
Motion compensation is one of the most important components of standard codecs (fig. 2).
The method of compensatory motion prediction allows to significantly reduce the time redundancy of the video stream. If the next frame contains shifted parts of the previous frame, then in this case it is beneficial to transmit not the entire frame, but only information about the movement and change of the shifted pixel. Given the high spatial correlation, it is enough to transmit one common motion vector for a 16 x 16 pixel macroblock. For each block of the first frame, a search is made for the most similar block in the second and a movement vector is calculated, which indicates the direction of movement of the block from the first frame to the second. The search area must be large enough so that the moving macroblock of the first frame image does not go out of the search area of the second frame.
Most often, the size of the search zone is set 4 times larger than the size of a separate macroblock (64x64). After which they can be localized. For example, during the formation of the P frame, it is necessary to determine the coordinates of the movement in the forward forecast. For this, the macroblock of the current P frame is taken and its new position is searched for in the search area of the previous I or P frame, then the point diferences between the frames are calculated. The position of the macroblock, at which the total value of the modules between the frame diferences of the macroblock is the smallest, is taken as its real movement, after which the coordinates of the motion vector are calculated as the vertical and horizontal displacement of the macroblock relative to its initial position.
For the correct restoration of the video stream, the sequence of frame decoding must difer from the sequence of their transmission. Since within a group of frames, which usually consists of 12 frames, each B frame is restored by the surrounding P frames (at the beginning and end of the group - by I and P), and in turn, each P frame - by the previous P (or I) frame, I frames can be restored independently of the others, they are the reference for all P and B frames of the group.Accordingly, I frames have the lowest degree of compression, and B - the highest. In terms of size, a typical P frame is 1/3 of I, and B - 1/8. Two consecutive B-frames generated by the same algorithm and using the same reference frames are not the same, since they represent diferent moments of video display time. Additional element-by-element encoding of series lengths. It should be taken into account that the lengths of the series carry the basic information about the shapes and sizes of the image objects, which is decisive for the correct recognition of the images. Even small distortions in the series length values lead to partial or complete destruction of the image of objects. Therefore, it is not recommended to use existing methods based on the reduction of psychovisual redundancy to compress the lengths of series.
Based on the analysis, it can be concluded that: analysis of existing coding methods without loss of quality of the information flow showed that they do not provide processing and transmission of video data in real time, and also do not provide a given level of immunity to errors in the communication channel. At the same time, the redundancy of the compressed information flow with respect to the theoretical limit reaches 90% ; the analysis of the structural properties of arrays of series lengths revealed the possibility of an additional increase in the degree of coding of the information flow due to the use of statistical coding methods.
Therefore, the methods of encoding information streams by eliminating statistical redundancy in arrays of series lengths are of interest for research on compliance with the requirements for video information processing, transmission and display subsystems in ITS, namely, processing video data in real time while preserving the original quality of the information stream. The structural diagram of compression in standardized implementations is presented in Fig. 3. Three main subsystems of the coder can be distinguished: the subsystem for eliminating spatial redundancy; subsystem for eliminating time redundancy; flow rate control subsystem. Redundancy is a consequence of certain correlations. The presence of correlation means that a certain element of the information flow is in a certain dependence on neighboring elements in space and time. Redundancy in perception (psychovisual) is related to the peculiarities of human vision. For example, the color sensitivity of human vision is below luminance, and this feature is taken into account in all standard analog color coding systems. For the first subsystem, the block of color space conversion and subsampling (reduction of the expansion of chrominance components) is intended for primary elimination of psychovisual redundancy of the frame.
The block for changing the order of frame processing is necessary for the operation of the motion compensation system. Block of discrete cosine transformation - for obtaining spectral coeficients and eliminating spatial correlation of pixels. Quantization block - to eliminate secondary psychovisual redundancy. Block of conversion of the matrix of DCT coeficients into a vector by zigzag scanning - to increase the eficiency of the Hufman code.
In the process of analyzing the characteristics of the existing video information flow encoding methods, it was found that the intensity of the video flow of standardized implementations largely depends on the quality of the image that is restored on the receiving side. It is substantiated that the intensity of the information flow of the means of data transmission in the intelligence information collection and processing systems is up to 10 Mbit/s, and the coding standard of the video information flow transmits the necessary information flow at speeds from 15 Mbit/s, therefore the imbalance of the intensity of the transmission of information flows is 30%. Therefore, standardized pipelines for processing video information streams need improvement in the direction of decomposing the structure of the general stream and using new compression methods without loss of image quality.