Mathematical processing of images in real-time vision systems can be conventionally divided into two stages: preprocessing (filtering, contrasting, protection from natural distortions, etc.) and final one (imposition, visualization, solution of the navigation task, etc.). Mentioned tasks can be solved by a lot of known and specially developed methods with various degrees of efficiency. The present paper suggests a mathematical criterion model and algorithm of automatic selection of the most effective method at each stage of the image processing pipeline in relation to the current situation at its input.
N W Lxy L y 1 x1
HW where H , W – a height and width of the image, and Lxy – brightness of the element of the current image with coordinates x and y .
Such objective characteristics as root-mean-square deviation ( ) and entropy ( ) are used for quantitate quality assessment. Root-mean-square deviation is equal to notions - local contrast and accuracy to some extent. Entropy is a measure of quantity of information in the image.
Task to estimate image quality has a multicriterion nature, so an additive generalized criterion F is introduced as follows: p F i fi ,
i1 p where i – weight coefficients, i 1 – a condition of normalization F , f i – partial normalized criteria, p – a number i1 of partial criteria.
Normalized partial indices of the contrast and numbers of informative levels are determined as follows:
Kn (Lmax Lmin) ,
255 Nn
N
, Nmax where Lmax max( Lxy ), Lmin min( Lxy ) – maximum and minimum values of brightness of image elements, N – a number of informative levels being different from null, Nm ax 256 – a maximum number of informative levels in digital images for visualization.
Shannon entropy estimation can be calculated for any half-tone (including television and thermal) image. In this case
estimation of distribution of possibilities of gray shades in the half-tone image is calculated [
N
i ,
HW 2 pi (i i )2 ,
i where Ni – a number of elements having i level.
Calculation of the image entropy is performed according to formula: pi log 2 ( pi ) . (2)
i
For normalization entropy values can be divided into a coefficient being an entropy maximum for such number of levels. For the half-tone image with 256 brightness gradations it is equal to 8. So, the image entropy value can vary from 0 to 1.
Image dispersion is calculated as following: where i i pi , i – a level of quantization.
i
For the half-tone image estimation of dispersion of distribution of possibilities of gray shades is calculated. Experimentally it was determined on a series of different images that root-mean-square deviation varies within the range from ≈0 to ≈100, then the mean value is 50, consequently n :
, 50, 50 (100 ) n , 50 100, 50 0, 100.
For average brightness, values belonging to middle of the range are preferable, and on boundaries of the brightness range its gradations cannot exceed 8. Normalized value of the entropy has the form: n
Main complication of particular index application is selection of weight coefficients taking into consideration influence of
corresponding particular indices on the generalized criterion as a whole. Fishburne criterion is used for selection of initial values
of these coefficients [
For this purpose partial criteria are divided into groups by priorities: (Ln , n ) ; (K n , N n ) and ( n ). Root-mean-square deviation contribution weightiness to the value of the quality function integral index is described by this index meaning: it determines accuracy and to some extent perceives intense noise. Then indices are arranged by descending of influence inside separated priority groups.
Taking into account above-mentioned facts, integral performance index (IPI) of the image brightness component has the form: IPI 0,33Ln 0,27 n 0,20K n 0,13N n 0,07 n.
Correction of coefficients under partial indices is performed by the method of expert evaluations, besides their amount should be equal to one.
Noise power for the whole image is calculated in the block of noise evaluation. For this purpose, image is divided into windows of size 3x3 and value of the neighbor pixel brightness which is located diagonally to the central one is subtracted from the central pixel brightness. Result of subtraction is raised to the square and summed up by all pixels of the image.
Described algorithm allows automatically selecting a method of contrasting and also a mode of interference compensation. However, usage of this algorithm on a video sequence leads to appearance of areas where a sharp jump in brightness occurs (other algorithm of contrasting is chosen), such event negatively influences on perception of video information by an operator. The method described below is suggested to be used for solution of this issue.
following form:
Let’s suppose that is a number of the video frame where replacement of the algorithm occurs, then + = ′ is a number of the following frame where analysis and selection of the enhancement algorithm are performed, then т is a current video frame, besides
≤ т ≤ ′. Let’s designate as an enhancement algorithm on the k –frame and ′ as an enhancement algorithm after t frames after k –frame. Interpolation method is the following: for each т frame, proportion of two algorithms
and ′ is calculated, so, the closer т is to ′, the higher coefficient the result of algorithm ′ is used with, and consequently, the lower coefficient the result of algorithm is used with. Reversed situation can occur similarly when т is closer to boundary . Hence, formula for calculation of the resulting image pixel brightness depending on value т has the , = , ( т) ∗ (1 − т − ) + , ′( т) ∗ ( т −
), where x,y – pixel coordinates, , ( т) – a buffer with a result of the first algorithm for the current frame, , ′( т) – a buffer automatic selection of the following enhancement algorithm. ′ frames correspondingly, – a number of frames between moments when automatic choice of the algorithm happens. with a result of the second algorithm for the same frame, ( т), ′( т) – results of operation of algorithms selected on and
Such formula allows gradually changing applied algorithms without sharp bursts on the resulting image but requires processing of the image by two algorithms that decreases resulting performance. The algorithm based on this method begins operation with obtaining of results of the automatic selection for and ′ frames. Then each frame is processed by two algorithms and
′, processing results are stored in two buffers of images. Each pixel of the resulting image is formed according to formula 4. After achievement of the interval boundary ( т = ′), = ′, and value ′ is obtained from
So, the best element is (A3, B0) M. (4) Number of levels: 250 Average brightness: 155.36 Entropy: 0.815 IPI: 0.827 Noise evaluation: 12.8 Noise-compensation algorithm: without noise compensation Contrasting algorithm: Retinex
Image Processing, Geoinformation Technology and Information Security / D.A. Kolchaev, Е.R. Muratov, М.B. Nikiforov
We can see that the frame with number (k+t) is now processed by the same sequence of algorithms as other frames. It allows avoiding undesirable darkening.
Fig.3 shows a diagram of dependence of IPI on a number of the frame. Numbers of frames are marked in horizontal direction
and values of quality indices are marked in vertical direction, A0-A3 – contrasting algorithms (A0 –without contrasting, A1 –
linear stretch of the histogram [
0,96 0,86 0,76 0,66 0,56 0,46 0,36 0 5 7 A0 A1
On the diagram we can see areas which value of IPI has not the best value for, it is described by delay occurred because the stack is used and evaluation is performed after every t=50 frames.
Application of the suggested algorithm for automatic selection of the enhancement method allows automatically finding the best combination of methods for image enhancement. The algorithm provides a possibility to change both a number of used methods of enhancement and also methods for image evaluation not causing significant changes in the algorithm structure. It allows implementing new enhancement algorithms and also new methods of evaluation. Application of the stack for accumulation of processing results and interpolation method of proportional application of two boundary algorithms allows avoiding errors in choice of the best algorithms.