Bit Speed Control Method in Compression of Predicted Frames in Video Sequence

Abstract

The method of bit rate control in compressing videodata is proposed, taking into account achieving the maximum image quality. In the process of finding optimal compression parameters it was decided to use the dichotomy method. It allows you to find the optimal values with a given accuracy, without having to resort to a complete search of options. This leads to the reduction of video group frame processing time, which is especially important in case of media content transmission in real time mode. The simplicity of the method is also one of the necessary requirements for control algorithms, when used in devices with limited computing power. A quality factor was selected as the main compression parameter, with which the quantization matrix is formed. At this stage, the greatest loss of information occurs. By choosing the method of block processing in the frame, it is possible to reduce the bit rate to the necessary limit for transmission over the communication channels.

Keywords

• predicted frames bit rate brightness component quantization quality factor the bit rate control

-

MPEG and H.264 standards require that the video frames processed by separate units - the macroblock. If the controlled encoding parameters are held constant (e.g., the size of the search field motion compensation step quantizer, and the like), the number of code bits of each macroblock will vary from macroblock to macroblock depending on the picture content, leading to variations in the output bit stream rate (measured in bits / frame or bit / s). Typically, encoder with fixed parameters produces more bits for the reference frame in which the rapid motion sealed or removed small parts, for frames with a slow change and without parts he will need fewer bits. Such jumps bit rate can produce a big problem for many transport and storage protocols. For instance, channel with constant speed (channel switching) is unable to transmit data streams at variable speed. Network based on packet switching can support variable bit rate, but the average throughput at any given time is limited by certain factors, depending on the connection speed and congestion.

In these cases, monitoring and bit rate adaptation, produced by the codec for its compliance with the transport speed. Variable coding rate may be adapted for transporting through the channels at a constant rate by the encoder and decoder buffers. However, for such adaptation have to pay amount of buffer memory and decoding delay. The higher the bit rate variation, the greater the required volume of buffer and the decoding delay is longer. In addition, this method cannot cope with arbitrary irregular bit rate, if not prevent unreasonably large amounts of buffers and not to make too long delays. For this it is necessary to implement a feedback mechanism for controlling the encoder output bit rate in order to prevent overflow or underload of buffers.

A common approach to monitoring the bit rate is to change the compression parameters during the encoding time in order to maintain the speed limit in the communication channel. One of the choices of parameters optimal performance compression method is the interval bisection or dichotomy. 2 2.1

Construction of basic bit rate control method during compression of predicted frames in a video sequence

Problem statement

Currently, solutions of the bit rate adapting problems, when transmitting the video data through the channels at a constant rate buffers used in the encoder and decoder. Thus, the higher the bit rate variation, the bigger volume of buffer required and the decoding delay is longer. Furthermore, this method cannot cope with arbitrary irregular bit rate, since the volumes of buffers and delays in the transmission and decoding have limits [ 4, 5 ]. Therefore, a bit rate control technique must be implemented using feedback that at lower bit rate also takes into account the quality of the reconstruction and processing of the video sequence frame time.

Since I-frames are the most critical to image quality deterioration, their compression ratio will be low. Therefore, to achieve the required values of the video sequence at the encoder rate in output is proposed to design the control method for the frames of P-type. Since their requirement for image quality is significantly lower, this allows to use techniques of compression with high ratios. And also, the number of the video sequence in order to support larger, which has a significant impact on the overall bit rate of the video stream.

For the encoder output stream, which corresponds to the capacity of the data channel, the control algorithms used bit rate. The desired output speed is achieved by controlling the quantization parameter. Depending on the application, compression may be used as the constant output rate (CBR - constant bit rate), and a variable (VBR variable bit rate) [ 4 ]. To compress with constant rate, depending on the purpose of the algorithm, may vary the size of the buffer for the encoder and decoder. If necessary, a more detailed output rate control algorithm used by encoder multi-pass encoding. Widespread received two-pass coding scheme: the first pass - preliminary analysis step and the establishment of the required encoding parameters; a second passage coding the previously specified parameters. In this case the selected parameters may be used as a separate block in the frame, and the entire frame. 2.2

Construction of method

During the frame processing occurs in the dimension of its partition blocks m  n . Thus, the whole frame represents a plurality of blocks b1, b2 ,...., bk  . These blocks are encoded separately from each other [43]. We denote by d(t)i and (t)i bit costs and standard error for the block bi current frame t. meaning d(t)i and (t)i depend on making vector

i  , which is used for encoding each block. A vector solution is part of the solution set, which is denoted by the letter  . A vector solution includes k components, i.e. i  ψ1i, ψi2 ,..., ψik  . Thus, the expression for d(t)i and (t)i It can be represented as follows: d(t)i  di  t, i   d(t)i 1i, i2 ,...ik  ; (t)i  i  t, i   (t)i 1i, i2 ,..., ik  .

The article [ 1 ] proposed an algorithm of processing of blocks in P-frames. In accordance with what is proposed blocks brightness component processed by two different methods: using the DCT as an I-type or using the DICM as P-type, and the color difference components will be processed only by using a DICM previous frame.

When compression decisions of P-frames, vector for bit rate control method consists of two components: informativeness measure unit and the parameter quality i  1i, i2 . Accordingly, for each i-th block of the bit rate and the mean square error will be a function of two variables: d(t)i  di  t, i   d(t)i 1i, i2 ,...ik  ; (t)i  i  t, i   (t)i 1i, i2 ,..., ik  .

The standard error for the entire frame is given by:

(1) (2) Similarly, the bit cost per frame are defined as:

k   t,       t, i  .

i1 k d  t,     d  t, i  .

i1 To optimize the compression parameters necessary to find such values of the vector solutions that will meet the following conditions:   t, *   min   t, i ;  i d  t, *   dreq .   (3) Where dreq - the required bit cost per frame; * - optimal vector solutions.

In processing blocks as a control parameter is proposed to use only quality factor, which is used for generating the quantization matrix. Since at this stage conducts a correction of component transformant under psychovisual characteristics of visual perception. Informativeness measure unit determines by transformant forming method and its impact on the bit rate is not significant.

In the formation of matrices of quantization in the JPEG standard [ 7 ] used two approaches. One is that the standard JPEG recommended to include two quantization tables, one for the brightness (Table 1), or the second for color.

Quantization matrix calculation is as follows: the quality factor is given by one value (Quality Factor - QF) is usually in the range from 1 to 25, and calculates the formula of the matrix of values: q(t)i, j  1  (1  i  j)  QF . (4) ue d  t, *   dreq .

The quality factor sets the interval between adjacent quantization levels matrix located at its diagonals.

This paper proposes to manage not only the number of bits to the compressed block, but also its visual quality. It is necessary to find the value of the quality factor QFopt , in which the mean square error is minimal   t, *   min   t, i  , given i the fact that the speed, and therefore the bit cost will not exceed the desired val

To find the optimum ratio QF using known half segment division method (dichotomy) which, relative to other methods is faster, easier and provides the desired accuracy (  ).

QF denote as a parameter  and correspondingly QFopt as a parameter opt . The method of implementing the search for the problem can be illustrated by the following sequence of steps:

Prepare stage To determine such values and that certainly is 1  min 2  max true d  t, 2   dreq  d  t, 1  .

Iteration

Find the average value  in accordance with the method of the dichotomy in the 1  2  to move closer to opt . interval [ 1;2 ]:   

 2  To check compliance   opt You need to calculate: d  t,   .   t,   .

If d  t,    dreq and   t,    req the lower limit of the interval is displaced ( 1 :  ); mal *   .

If d  t,    dreq and   t,    req then these parameters there is no solution and the control unit initiates a change in the value QF or ΔD.

If d  t,    dreq the upper limit of the interval is displaced ( 2 :  ).

The verification of algorithm complete condition.

If 1  2   , the search for solutions is completed and the result is opti1  QFmin If not, it proceeds to the next iteration.

A block diagram of the implementation of the method for finding the optimal parameter control bit rate is shown in Fig.1.

Consider an example of operation of this method.

Assign values and such that the condi2  QFmax tions d  t, 2   dreq  d  t, 1  .

1  2  and calculate values for a givWe spend the first iteration. We find 1    2  en d  t,   and   t,   . We make a comparison d  t,    dreq . If the condition is not satisfied, the bit rate should be increased to improve the image quality.

In this case, assign 2 :  and then examines the range of [ 1; ]. If the condition d  t,    dreq is performed, to review by the standard error:   t,    req .

If the error is less than the required   t,    req assign 1 :  and further finding the optimum value will be in the range [ ;2 ]. Thus, after the first iteration in the example shift will be performed: max   .

In case if the speed and the error exceeds the required value any of the conditions does not satisfy: d  t,    dreq .   t,    req , We see that the optimization of the data compression method is not possible and necessary changes of the original parameters. In this case it is proposed to increase the quality factor in the quantization in increments of 1, if the bit cost will not be reduced to the desired level, the next step will be to increase ΔD threshold.

Thus, the control unit selects the quality factor for each P-frame. Also, when deciding the control unit comprises a memory in which stored optimum parameters QF and ΔD, which further reduces the time of the decision and selection of the desired values in the quantization.

The last step is carried out verification of optimality. We check the condition 1  2   , where the parameter  - shows the set accuracy of calculations. If the required accuracy was achieved, the optimal value is considered   t,   . Solution found. Otherwise the search will continue for implementation of the next iteration.

Thus, the elaborated method does not require complete (or nearly complete) sorting a plurality of solutions such as, for example, by dynamic programming. This reduces processing time and frame transmission, which is necessary when working in real time.

The simplicity of the method also makes it possible to reduce the load on the computing device encoder, so there is the possibility of its use in systems in limited computing capabilities.

In view of this method represent the overall control algorithm to work under compression P frames.

At the initial stage of the control set initial parameters of blocks compression for brightness and chrominance components, respectively: thresholds D(Y)in . D(Ca )in and quality of factors QF(Y)in . QF(Ca )in , then assesses the total bit rate V(t)comp and the mean square error (t) video frame being processed.

Since the brightness component when restoring the image information has a great load than color-difference components Cr, Cb, the threshold value of [ 3,4 ] in the processing of the brightness component blocks D(Y) We will choose less than the processing of the color difference components D(Y)<D(Ca ) . D(Cr )=D(Cb )=D(Ca ) , where Ca - represents one of the components color difference planes: Ca  Cr or Ca  Cb .

Similarly, the quality factors for brightness and chrominance components will also be chosen with different values: QF(Y)  QF(Ca ) .

The initial parameters selected for reasons best image quality, but they are set in the ranges: Dmin  D(Y)in , D(Ca )in  Dmax

QFmin  QF(Y)in , QF(Ca )in  QFmax .

If the standard error   t  will exceed the required value   t   req , the quality of the reconstructed image is less than a predetermined, carry out reduction value of quality factor brightness component QF(Y) .

After that assesses the total bit rate V(t)comp and the mean square error   t  stream [ 5 ]. Further, if necessary, QF(Y) decreases again, until a minimum value QFmin . If QF(Y)=QFmin and the condition for the image quality is not yet achieved   t   req , the decision to change the next parameter - QF(Ca )in for the components of color difference. Reducing procedure QF(Ca )in repeated similarly.

If the parameters of the quality factors achieved the minimum values QF(Y)=QFmin and QF(Ca )  QFmin and root mean square error   t  still higher than the desired value   t   req it occurs threshold increase first for brightness D(Y)in and then for the color difference components   t   req . This increases the I-type blocks in the frame, which in turn will improve the quality of the compressed P frame. At each change thresholds also assesses total bit rate V(t)comp and the mean square error   t  .

Thresholds D(Y) and D(Ca ) can only increase to a predetermined maximum values: D(Y)  Dmax . D(Ca )  Dmax .

Selection is carried out as long as the predetermined image quality at a bit rate desired value is reached:   t   req .

If too much compression possible for this parameter is not allowed to reach a given quality:   t   req In accordance with the method developed by the management decision to change the color subsampling format (for example format of 4: 2: 2 to 4: 4: 4).

This approach will greatly improve the quality of the reconstructed image due to the transmission of the complete information from all color planes model YCrCb, however this can lead to a sharp increase in the final compressed bit-rate frame.

Consider the case when the total bit rate V(t)comp higher than necessary:

V(t)comp  Vreq .

Here, the characteristics of the intensity of the control mechanism of the video stream are utilized in the following order: 1. increase QF(Ca ) for the color difference components; 2. increase QF(Y) for the brightness component;

3. decrease D(Ca ) for the color difference components;

4. decrease D(Y) for brightness component.

The values of the quality factors QF(Y) QF(Ca ) and thresholds D(Y) , D(Ca ) can only increase or decrease to predetermined values:

QF(Ca )  QFmax . D(Y)  Dmin ; D(Ca )  Dmin . D(Ca )  Dmin .

If too much compression possible for this parameter is not allowed to reduce the bit rate to a desired value: V(t)comp  Vreq In accordance with the control method is decided to change the color subsampling format (for example format of 4: 2: 2 to 4: 1: 1), which will greatly reduce the amount of data per color-difference components.

Evaluation of the final bit rate V(t)comp it is performed after change of each parameter, as long as it is less than the desired value V(t)comp  Vreq . If, after all the changes did not reach the set speed during the processing time that has been allotted to one frame compression, it may be decided to permit the passage of the current Pframe. 2.3

Results and discussion

The designed total bit rate control method allowing to make an adjustment of the intensity of the video stream in accordance with the parameters of the telecommunications network.

For comparison, the developed method bit rate control existing source video stream primarily been compressed codec standard with characteristics of transmitted streams: a frame size of 1280x720 pixels; video codec H.264 / AVC variable bit rate (VBR); frame rate - 30 frames / s.

From the data obtained, were taken averaged maximum values of bit rate, as they play a crucial during accelerations and loss (the average maximum value of the intensity of the video stream for MPEG was 1,153 Mbit / s, the average value at which no overload is defined as required Vtr = 733, 6 Kbyte / s). Similar operations are also performed using the developed method and coding control. As a result, the dependencies bit rate were calculated ( V(t)comp ) And peak signal / noise ratio (PSNR) of the threshold ( D ) For various values of the quality factor (QF), when subsampling formats 4: 1: 1, 4: 2: 2 and 4: 4: 4 (Figure 1-4).

As the minimum limit for the peak signal / noise ratio is taken equal to the standard value of 26 dB MPEG.

From the results obtained it can be concluded that the best treatment for the Pframe is a color sub-sampling format 4: 2: 2, wherein the quality factor (quantization step) = 0 and the decision threshold = 33 out of 255 possible is provided: ─ gain in the degree of reduction of the bit rate for a given parameter PSNR on average 36.36% with respect to the MPEG standard; ─ predetermined level of bit rate control process for its higher quality indicators determined parameter PSNR, at the level of 30.18 dB, which is on average 16.07% better MPEG standard. Fig. 2. The dependence of the peak signal / noise ratio (PSNR) from the threshold (D) for different values of the quality factor (QF) Thus, the designed total bit rate control method allowing making an adjustment of the video intensity stream in accordance with the parameters of the telecommunications network. The developed method takes into account the following mechanisms: 1. used differential processing blocks selectable by a predetermined limit, which indicates a measure of the current block information content, its algorithm described in [ 4,5 ]; 2. in the process of compression algorithm produced the following parameters: color sub-sampling, threshold information content, as well as the quality factor. Combining mechanism of the selected parameters is made in such way that the decision was made about the optimality with minimal latency. This allows you to adapt quickly to the throughput of the communication channel and to select the quality of the transmitted image. 3. for processing brightness and chrominance components of a frame in accordance with the method developed by the threshold value and the quality factor, will be chosen to lower brightness as compared with the chrominance components. This leads to the fact that the brightness component of the frame is compressed with better quality than chrominance components. Depending on the conditions of desired velocity or a predetermined quality parameters, quality factors and thresholds will change so to select optimum values for the compression in the shortest possible time span.

References

1. End-user multimedia QoS categories; G. 1010 ( 11 /01), ITU-T STANDARD , INTERNATIONAL TELECOMMUNICATION UNION , GENEVA ; CH, no. G. 1010 ( 11 /01), 29 November 2001 ( 2001-11 -29), pages 1 - 18 , XP017463369

2. Barannik , V.V. , Ryabukha , Yu.N. and Kulitsa , О.S. : The method for improving security of the remote video information resource on the basis of intellectual processing of video frames in the telecommunication systems . In: Telecommunications and Radio Engineering . Vol. 76 . No 9. pp. 785 - 797 . ( 2017 ). doi: 10 .1615/TelecomRadEng.v76. i9 . 40

3. Barannik , V. , Krasnorutskiy , A. , Ryabukha , Y. , Okladnoy , D. : Model intelligent processing of aerial photographs with a dedicated key features interpretation . In: Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET), pp. 736 - 739 . Lviv-Slavske ( 2016 ). doi: 10 .1109/TCSET. 2016 .7452167

4. Barannik V. , Podlesny S. , Krasnorutskyi A. , Musienko A. , Himenko V. : The ensuring the integrity of information streams under the cyberattacks action . In: East-West Design & Test Symposium (EWDTS), pp. 1 - 5 . IEEE Press ( 2016 ) doi: 10.1109/EWDTS. 2016 .7807752

5. Barannik V. , Podlesny S. , Tarasenko D. , Barannik D. , Kulitsa O. : The video stream encoding method in infocommunication systems . In: Advanced Trends in Radioelecrtronics, Telecommunications and Computer Engineering (TCSET), pp. 538 - 541 . Lviv 2018 . doi: 10 .1109/TCSET. 2018 .8336259

6. Gonzales R.C. , Woods R .E.: Digital image processing . Prentice Hall , New Jersey, edition . II, 1072 - 1073 . ( 2002 )

7. Richardson I.: H.264 and MPEG-4 Video Compression: Video Coding for NextGeneration Multimedia . Ian Richardson ( 2005 )

8. Zhang Y. , Negahdaripour S. , Li Q. : Error-resilient coding for underwater video transmission . In: OCEANS MTS/IEEE , pp. 1 - 7 . Monterey, CA ( 2016 )

9. Barannik V.V. , Ryabukha Yu . N., Tverdokhleb V.V. , Barannik D.V. : Methodological basis for constructing a method for compressing of transformants bit representation, based on non-equilibrium positional encoding . In: Advanced Information and Communication Technologies (AICT) , 2017 2nd International Conference, pp. 188 - 192 . ( 2017 ). doi: 10 .1109 / AIACT. 2017 .8020096

10. Baccouch H. , Ageneau P. L. , Tizon N. , Boukhatem N. : Prioritized network coding scheme for multilayer video streaming . In: 14th IEEE Annual Consumer Communications & Networking Conference (CCNC) , pp. 802 - 809 , IEEE Press ( 2017 )

11. Tsai W. J. , Sun Y. C. : Error-resilient video coding using multiple reference frames . In: IEEE International Conference on Image Processing , pp. 1875 - 1879 , IEEE Press ( 2013 ).

12. Chigorin A. , Krivovyaz G. , Velizhev A. , Konushin A. : A method for traffic sign detection in an image with learning from synthetic data . In: 14th International Conference Digital Signal Processing and its Applications , pp. 316 - 335 . IEEE Press ( 2012 )

13. Salomon D. : Data Compression: The Complete Reference . Fourth Edition. SpringerVerlag London Limited , pp. 899 ( 2007 )

14. Vatolin D. , Ratushnyak A. , Smirnov M. , Yukin V. : Methods of data compression. The device archiver, compression of images and videos . DIALOG MIFI , Moskow ( 2013 )

15. Barannik , V. , Krasnoruckiy , A. and Hahanova , A. : The positional structural-weight coding of the binary view of transformants . In: East-West Design & Test Symposium (EWDTS), pp. 1 - 4 . Rostov-on- Don ( 2013 ) doi: 10.1109/EWDTS. 2013 .6673178

16. Ding Z. , Chen H. , Gua Y. , Peng Q. : GPU accelerated interactive space-time video matting . In: Computer Graphics International, pp. 163 - 168 . ( 2010 )

17. Ablamejko S.V. , Lagunovskij D.M. : Obrabotka izobrazhenij: tehnologija, metody, primenenie . Amalfeja, Minsk ( 2000 )

18. Rao K. R. , Hwang J. J. : Techniques and Standards for Image, Video and Audio Coding . Englewood Cliffs, NJ: Prentice-Hall ( 1996 )

19. Shi , Yun Q. : Image and video compression for multimedia engineering: fundamentals, algorithms, and standards . Huifang Sun, NY , CRC Press ( 2008 )

20. Bai X. , Wang J. : Towards temporally-coherent video matting . In: Proceedings of the 5th international conference on Computer vision/computer graphics collaboration techniques , pp. 63 - 74 . MIRAGE' 11 , Springer-Verlag ( 2011 )

21. Musienko A. , Ganjaric J . : Technology of coding of digital aerial photographs taking into account classes of a semantic saturation of blocks in system of air monitoring . In: VII Inter University Conference of Students, PhD Students and Young Scientists ["Engineer of XXI Century , pp. 215 - 220 . Bielsko-Biała, Poland ( 2016 )

22. JPEG 2000 image coding system: Core coding system . ITU-T recommendationt. t. 800 and iso/iec 15444-1. ITU-T and ISO/IEC JTC 1 ( 2000 )

23. In J., Shirani S. , Kossentini F.: Jpeg compliant efficient progressive image coding . Department of Electrical and Computer Engineering,University of British Сolumbia, Vancouver BC V6T 1Z4, Canada

24. Zheng B., S. Gao: A soft-output error control method for wireless video transmission . In: 8th IEEE International Conference on Communication Software and Networks (ICCSN) , pp. 561 - 564 . IEEE Press, Beijing ( 2016 )

25. Kubasov D.V. , Vatolin S. : Review of methods of motion compensation . In: Computer graphics and multimedia , vol. 3 , pp. 33 - 43 . KPI, Kharkiv ( 2010 )

26. Kudryashov B.D. : Information theory . Peter, St. Petersburg ( 2009 )

27. Akimov D. , Shestov A. , Voronov A. , Vatolin D. : Occlusion Refinement for Stereo Video Using Optical Flow . In: International Conference on 3D Imaging , pp. 115 - 138 . ( 2012 )

28. Pratt W. K. , Chen W. H. , Welch L. R. : Slant transform image coding . Proc. Computer Processing in communications. New York: Polytechnic Press, рр. 63 - 84 . ( 1969 )

29. Stankiewicz O. , Wegner K. , Karwowski D. , Stankowski J. , Klimaszewski K . Grajek T.: Encoding mode selection in HEVC with the use of noise reduction . In: International Conference on Systems, Signals and Image Processing (IWSSIP) , pp. 1 - 6 . Poznan ( 2017 )