Increasing the Image Sharpness with Linear Operator for Social Internet-Services

Abstract

In the paper it has been propounded and experimentally researched the linear operators that can be used to sharpen digital images or video frames distorted by a micro-motion of fixation chamber. It has been assumed that the cause of the problem, which leads to deterioration of sharpness, is a lowfrequency interference, which is exemplified by a random non-recursive filter in the form of a discrete convolution. It has been experimentally proven that the proposed stabilizer filters allow for significant visual enhancement of distorted images, with a peak-to-peak signal-to-noise ratio for the improved images higher than if using similar sharpening filters implemented in Adobe PhotoShop CS6. The corresponding masks of the proposed operators and the examples of application are given in the paper.

Keywords

• Image processing
• image sharpness
• digital stabilization
• B-spline
• linear filter operators

1. Topicality and problem statement

An important factor for the positive perception of a digital image, or video stream, as a sequence of frames is how realistic it is. At the same time the sharpness of an image os one of the constituents that determine the realness of the image. The main causes for distortion that lead to deterioration of clarity are the limited resolution capabilities of the forming system, defocus, the presence of the distorting medium (such as the atmosphere), the movement of the camera towards the object which is being recorded [1]. Further we shall consider the processing of images obtained under the conditions of the micro-motion of the fixation chamber. This defect is most common in the case of a photography without a tripod or if shooting from a platform that may be a subject to some mechanical impact, such as microvibration (aerophotography, etc.). Unlike othercases, the consequences of micro-motion can be eliminated or at least significantly offset, not only by hardware but also by mathematical treatment procedures. Therefore, we believe the topical task is to find appropriate procedures for image sharpening. These should have low computational complexity and upon implementation in software products they should provide real-time processing.

In assumption of isoplanatic system of observation for intensity distribution I(x) the image of the object, that is being formed in the x plane of its registration, is being used an expression of the following type [2]:

I  x   O y H  x  y dy , ( 1 ) where O(y) - the distribution of the intensity of reflection from the object of light irradiation in the image plane y; H(x) - distribution of intensity in the image of an axial source point (impulse response or else – scattering of the function system point). Expression ( 1 ) provides an image that is recorded in the form of a convolution of a true image and the impulse response of the registration system. Thus [1], the intensity value of the original image is "smeared" at each of the points of registration in accordance with the type of function H(x).

To fix the problems caused by the micro-movement of the fixation chamber, they use image stabilizers, which can be divided into three general types [3; 4]: mechanical, electronic and digital. In this paper we research namely digital stabilizers. They are not used to against the cause of the blurry images in photo and video material. On the contrary they are intended to eliminate the consequences - that is, they are used for the post-fixation mathematical processing of digital data.

There is a number of methods used to reproduce blurred images [1; 2; 5; 7-9]. However, among the effective procedures of digital image sharpening (digital stabilization) that satisfy the requirements of actual processing in real time, preference should be given to those that achieve the target processing function at a minimum of computational operations. In fact, these are linear operators obtained in the form of a discrete convolution of the color components of the raster and masks of filtersstabilizers.

Assuming that the distortion of the original image has been caused by the micromotion of the locking chamber, as e.g. by the vibration of the aircraft during aerial photography [6], then, according to the above-mentioned and based on ( 1 ), the following representation can considered reasonable for a digital image pi, j  L pi, j   iiiiriri jjjjrjrj iii, jj j pii, jj , i   k2i , k2i , j   k j , k j , 2 2 ( 2 ) where pi,j - color raster component (red, green or blue); L(i,j) - linear low-pass image filter operator; (i,j) - the pixel index of the raster; ki,kj - image frame sizes; pii, jj - the  color component of the raster of the perfect undistorted image; iii, jj j - low pass filter mask element; 2ri 1  2rj 1 - the size of the low pass filter mask.

Considering a digital image, specified by the raster with any of the constituents P   pi, j ;i  0, H 1, j  0,W 1 . As an experiment, we will simulate an accidental lowfrequency interference, such as the operator ( 2 ), for some digital images. To obtain a linear operator L pij  from ( 2 ), we shall define the low pass filter mask as follows. Assumingly pii - one-dimensional sequence of a function (for distinctness), and  pii - a sequence obtained after smoothing. If pi  pi  2 pi  4 pi ,

2 pi  pi1  2 pi  pi1 ; 4 pi  2 pi1  22 pi  2 pi1 ,  0,05;0,5 , ( 3 ) where then

pi  pi2    4 pi1  1 2  6 pi    4 pi1  pi2 , hence, from the condition of additionality (as for the low pass filter) with the coefficients pi , i  , we get:    0,    4  0, 1 2  6  0,    0,    0, 25,   0,333   0,5, 0;0, 25 .

( 4 )

With the direct multiplication it is not difficult to obtain a low-pass filter mask of size 5x5 to determine the operator ( 2 ), namely (taking into account the symmetry:  2   42     42   42      2  62   41 2  6  

  2  62   41 2  6 1 2  62    ,     where  and  are defined as evenly distributed random realizations that satisfy conditions ( 3 ) and ( 4 ).

Linear operator for the digital stabilization of images

Let’s introduce linear operators C  pij  of digital image’s stabilization, as follows pi, j  C  pij  , and the quality of stabilization will be considered acceptable if considering ( 2 ) true is pi, j  pi, j , i, j  ( 5 ) for a random mask  .

Linear operators, such as those considered in [6], can be used to stabilize the image:

irl jrl Cl  pij     iiirl jj jrl l iii, jj j pii, jj , i, j  , ( 6 ) Where l=0,1,2,3,4; r0=2; r1=r2=3; r3=r4=4; 0  0,000233473 0,00522272 3   9,20847E-07 0,000233473  2,14132E-06 0,000542915

0,012144825 0,028241375 -0,249914233  -1,8949E-05 -0,004804375 -0,107472272 -0,249914233 2,211546853   

We shall notice that the coefficient of masks l , l  1, 4 can be determined by taking into account the symmetry of the corresponding matrices.

3. Experimental studies

Operators ( 6 ) provide visual enhancement of the perception of digital images distorted by the micro-motion of the fixation chamber, in particular data from the cameras of the target load of aircraft. However, in order to avoid subjectivism in assessing the quality of perception improvement, we present the results of experimental studies that have been conducted using the introduced operators.

Let n, m - raster sizes, N  n  m - the number of pixels of the raster. The image aberration in each pixel is determined as follows:

i, j  pi, j  pi, j , i  1, n , j  1, m , then the average error of reproduction for each component equals constant error variance –   N1 in1 jm1i, j

; 2  1 n m

  i, j   2 .

N 1i1 j1

To check the fulfillment of a condition ( 5 ) when analyzing reproduced images widely used is peak-to-peak signal-to-noise ratio - PSNR, which is defined as follows:

The total PSNR for the image is determined by averaging the PSNR of each of the color components. The interpretation of PSNR is quite simple: the greater the value of the statistics is, the greater is the correspondence between the two images.

There has been conducted an experiment for some high-quality digital image. It was aimed at comparison of the performance of the described operators and sharpeners presented in the Adobe PhotoShop CS6 digital image processing environment. In particular, we compared the two filters – “Unsharp Mask” and “Sharpen More filter options” presented in the “Filter” menu and in the “Unsharp Mask” submenu. The use of such filters does not require additional adjustments, so, probably, the filters themselves are operators similar in design ( 6 ). Unfortunately, the description of the mentioned filters is not freely available, so the comparative analysis was performed as follows.

Step 1. Generate evenly distributed  ,  and require their correspondence with conditions ( 3 ) and ( 4 ).

Step 2. To test the work of each of the five operators ( 6 ) and filters “Unsharp Mask” and “Sharpen More filter options” presented in Adobe PhotoShop CS6, we simulate operator distortion ( 2 ) according to the generated  ,  and  mask.

Step 3. Define a PSNR, comparing it to the original image, for each image reproduction result.

Step 4. Repeat the experiment 24 times.

The number of repetitions of the experiment (step 4) is not large, which (unlike the previous experiment) is due to the inability to automate the work with series of images in Adobe PhotoShop CS6. However, as can be seen from the results of the experiment Table 1, even this number of repetitions clearly demonstrates the advantage in the use of filters ( 6 ). It is worth paying attention to the following pattern. PSNR value according to the results of the use of «Unsharp Mask» filter heavily correlates with PSNR values after the use of C0  pi, j  and C2  pi, j  operators, but the filters, researched in this paper, keep the advantage. The same is true for «Sharpen More filter options» filter and operator C4  pi, j  .

As it can be seen from the results presented in the table, the introduced operators have advantages in comparison to the well-known «Unsharp Mask» and «Sharpen More filter options». №

We shall remark, that the maximum PSNR when using the operators ( 6 ) is obtained by increasing  ordinary index l. Thus for C2  pij  maximum PSNR at approximately   0,125 , at the same time for C4  pij  - at   0, 45 (Fig.1). smaller values of  ) and 4 : the abscissa axis -  ; the axis of ordinates – PSNR.

Thus, the presented and researched linear operators ( 6 ) can be recommended for the automated processing of digital images distorted by interference, such as the micro-movement of the fixation chamber, which is possible for aerial photography from the aircraft.

4. Conclusions

As a result of shooting with digital cameras aimed for different purposes the obtained images can become distorted by the effects of low-frequency hindrance caused by the vibration of the carrier during photo or video shooting. To eliminate the effects of such interference we offer linear filter operators to process the obtained digital images (Fig. 2). In particular, we have presented and experimentally substantiated the linear digital image stabilization operators for the use in case of the random nature of lowfrequency interference. Fig. 2. An example of a digital image: А) and С) – images distorted by a random interference; В) and D) - images after stabilization.

Further research may be aimed at obtaining similar operators with a larger filter mask window width and at the extension of this approach to processing digital video streams in real time for social Internet-services.

References

1. Gruzman I. : Digital image processing in information systems : Textbook / Grusman I., Kyrychuk V. , Kosyh V. , Peretaygin G. , Spector A . - Novosibirsk: Publishig House of the NTU , 2000 . - 168 p.

2. The latest image processing techniques . / Ed. A. Potapov . - Moscow: FIZMATLIT, 2008 . - 496 p

3. Gonzalez R. : Digital image processing / Gonzalez R., Woods R . - Moscow: Technosphere, 2006 . - 1072 p.

4. P. Prystavka “ The use of combined filters based on polynomial splines in raster's images processing ” // Bulletin of NAU. - 2008 . - # 4. - P. 104 - 107 .

5. Zohair Al-Ameen , Alaa Muttar, Ghofran Al-Badrani, " Improving the Sharpness of Digital Image Using an Amended Unsharp Mask Filter" , International Journal of Image, Graphics and Signal Processing(IJIGSP) , Vol. 11 , No. 3 , pp. 1 - 9 , 2019 .

6. A. Chyrkov “Suspicious Object Search in Video Stream from Aircraft Camera by Using Histogram Analysis” , Problems of Creation, Testing and Usage of Information Systems: Scientific Publications of ZhMI , Zhytomyr 13 P. 126 - 135 ( 2016 ).

7. J. Clark , C. Wadhwani , K. Abramovitch , D. Rice and M. Kattadiyil , "Effect of image sharpening on radiographic image quality" , The Journal of Prosthetic Dentistry , vol. 120 , no. 6 , pp. 927 - 933 , 2018 .

8. Roman Odarchenko , Serhii Dakov, Olexandr Oksiuk and Larisa Dakova SoftwareControlled Network SDN Reliability Calculation 5th International Scientific-Practical Conference Problems of Infocommunications Science and Technology , PIC S and T 2018 - Conference Proceedings, pp 99 - 103 .

9. J. Ye , Z. Shen , P. Behrani , F. Ding and Y. Shi , "Detecting USM image sharpening by using CNN" , Signal Processing: Image Communication , vol. 68 , pp. 258 - 264 , 2018 .

10. F. Ding , G. Zhu, W. Dong and Y. Shi , "An efficient weak sharpening detection method for image forensics" , Journal of Visual Communication and Image Representation , vol. 50 , pp. 93 - 99 , 2018

11. Prystavka P. , Cholyshkina O. : Components of information support for automated processing of digital images based on local operators . Actual problems of automation and information technology: Coll. Sciences. wash . - D .: View of Dnepropetrovsk University - 2010 . -T. 14 . -S. 27 - 36 .