Increasing the Image Sharpness with Linear Operator for Social Internet-Services Prystavka Pylyp 1 [0000-0002-0360-2459], Cholyshkina Olha 2 [0000-0002-0681-0413] and Zhengbing Hu 2 [0000-0002-6140-3351] 1 National Aviation University, Komarova 1, Kyiv, Ukraine 2 Interregional Academy of Personal Management, Frometivska 2, Kyiv, Ukraine 3 Central China Normal University, Wuhan, China greenhelga5@gmail.com Abstract. In the paper it has been propounded and experimentally researched the linear operators that can be used to sharpen digital images or video frames distorted by a micro-motion of fixation chamber. It has been assumed that the cause of the problem, which leads to deterioration of sharpness, is a low- frequency interference, which is exemplified by a random non-recursive filter in the form of a discrete convolution. It has been experimentally proven that the proposed stabilizer filters allow for significant visual enhancement of distorted images, with a peak-to-peak signal-to-noise ratio for the improved images high- er than if using similar sharpening filters implemented in Adobe PhotoShop CS6. The corresponding masks of the proposed operators and the examples of application are given in the paper. Keywords: Image processing, image sharpness, digital stabilization, B-spline, linear filter operators. 1. Topicality and problem statement An important factor for the positive perception of a digital image, or video stream, as a sequence of frames is how realistic it is. At the same time the sharpness of an image os one of the constituents that determine the realness of the image. The main causes for distortion that lead to deterioration of clarity are the limited resolution capabilities of the forming system, defocus, the presence of the distorting medium (such as the atmosphere), the movement of the camera towards the object which is being recorded [1]. Further we shall consider the processing of images obtained under the conditions of the micro-motion of the fixation chamber. This defect is most common in the case of a photography without a tripod or if shooting from a platform that may be a subject to some mechanical impact, such as microvibration (aerophotography, etc.). Unlike othercases, the consequences of micro-motion can be eliminated or at least signifi- cantly offset, not only by hardware but also by mathematical treatment procedures. Therefore, we believe the topical task is to find appropriate procedures for image sharpening. These should have low computational complexity and upon implementa- Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib- ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in Global Information Networks. tion in software products they should provide real-time processing. In assumption of isoplanatic system of observation for intensity distribution I(x) the image of the object, that is being formed in the x plane of its registration, is being used an expression of the following type [2]: I  x    O  y  H  x  y  dy , (1) where O(y) - the distribution of the intensity of reflection from the object of light irradiation in the image plane y; H(x) - distribution of intensity in the image of an axial source point (impulse response or else – scattering of the function system point). Expression (1) provides an image that is recorded in the form of a convolution of a true image and the impulse response of the registration system. Thus [1], the intensity value of the original image is "smeared" at each of the points of registration in ac- cordance with the type of function H(x). To fix the problems caused by the micro-movement of the fixation chamber, they use image stabilizers, which can be divided into three general types [3; 4]: mechani- cal, electronic and digital. In this paper we research namely digital stabilizers. They are not used to against the cause of the blurry images in photo and video material. On the contrary they are intended to eliminate the consequences - that is, they are used for the post-fixation mathematical processing of digital data. There is a number of methods used to reproduce blurred images [1; 2; 5; 7-9]. However, among the effective procedures of digital image sharpening (digital stabili- zation) that satisfy the requirements of actual processing in real time, preference should be given to those that achieve the target processing function at a minimum of computational operations. In fact, these are linear operators obtained in the form of a discrete convolution of the color components of the raster and masks of filters- stabilizers. Assuming that the distortion of the original image has been caused by the micro- motion of the locking chamber, as e.g. by the vibration of the aircraft during aerial photography [6], then, according to the above-mentioned and based on (1), the fol- lowing representation can considered reasonable for a digital image i  ri j  rj   iiir jj j r iii, jj  j pii, jj , i   k2 , k2 , j   2 , 2 , (2) kj kj pi, j  L pi, j   i i i j where pi,j - color raster component (red, green or blue); L(i,j) - linear low-pass image filter operator; (i,j) - the pixel index of the raster; ki,kj - image frame sizes; pii, jj - the color component of the raster of the perfect undistorted image;  ii i , jj  j - low pass filter mask element;  2r  1   2r  1 - the size of the low pass filter mask. i j Considering a digital image, specified by the raster with any of the constituents P   pi , j ; i  0, H  1, j  0, W  1 . As an experiment, we will simulate an accidental low- frequency interference, such as the operator (2), for some digital images. To obtain a linear operator L  pij  from (2), we shall define the low pass filter mask as follows. Assumingly  pi  - one-dimensional sequence of a function (for distinctness), and i  pi i - a sequence obtained after smoothing. If pi  pi  2 pi  4 pi , where 2 pi  pi1  2 pi  pi1 ; 4 pi  2 pi1  22 pi  2 pi1 , (3)   0,05;0,5 , then pi  pi 2     4  pi1  1  2  6  pi     4  pi1  pi 2 , hence, from the condition of additionality (as for the low pass filter) with the coeffi- cients pi , i  , we get:    0,    0,    0;0, 25 . (4)    4  0,    0, 25, 1  2  6  0,   0,333     0,5  ,   With the direct multiplication it is not difficult to obtain a low-pass filter mask of size 5x5 to determine the operator (2), namely (taking into account the symmetry:  2   42   2  6 2       42     4  2    4 1  2  6  ,     2  62    4 1  2  6  1  2  6 2      where  and  are defined as evenly distributed random realizations that satisfy conditions (3) and (4). Linear operator for the digital stabilization of images Let’s introduce linear operators C  p ij  of digital image’s stabilization, as follows   , and the quality of stabilization will be considered acceptable if consider- pi, j  C p ij ing (2) true is pi, j  pi, j , i, j  (5) for a random mask  . Linear operators, such as those considered in [6], can be used to stabilize the im- age: i  rl j  rl   Cl pij    l   ii i , jj  j pii , jj , ii i  rl jj  j  rl i, j  , (6) Where l=0,1,2,3,4; r0=2; r1=r2=3; r3=r4=4;  1 8 74 8 1    8 64 592 64 8  0 1  ;    74 592 5476 592 74  3136    8 64 592 64 8   1 74 1   8 8  3,75457E-09 8,93587E-07 5,40282E-06 -7,38748E-05     8,93587E-07 0,000212674 0,001285871 -0,01758221 ; 1     5,40282E-06 0,001285871 0,007774658 -0,106305883    -7,38748E-05 -0,01758221 -0,106305883 1,45356119       1,24562E-08 2,96456E-06 1,59314E-05 -0,000149424     2,96456E-06 0,000705566 0,003791678 -0,035562919 ;  2     1,59314E-05 0,003791678 0,020376288 -0,191113331    -0,000149424 -0,035562919 -0,191113331 1,792490633       1,6236E-10 4,1165E-08 9,20847E-07 2,14132E-06 -1,8949E-05     4,1165E-08 1,0437E-05 0,000233473 0,000542915 -0,004804375   9,20847E-07 0,000233473 0,00522272 0,012144825 -0,107472272 ;     3   2,14132E-06 0,000542915 0,012144825 0,028241375 -0,249914233   -1,8949E-05 -0,004804375 -0,107472272 -0,249914233 2,211546853       5,26177E-10 1,32248E-07 2,7676E-06 4,92362E-06 -3,85865E-05     1,32248E-07 3,32391E-05 0,000695605 0,001237494 -0,009698272   2,7676E-06 0,000695605 0,014557157 0,025897446 -0,202958992      4 .  4,92362E-06 0,001237494 0,025897446 0,046072025 -0,361067725   -3,85865E-05 -0,009698272 -0,202958992 -0,361067725 2,829697678      We shall notice that the coefficient of masks   l  , l  1, 4 can be determined by taking into account the symmetry of the corresponding matrices. 3. Experimental studies Operators (6) provide visual enhancement of the perception of digital images distorted by the micro-motion of the fixation chamber, in particular data from the cameras of the target load of aircraft. However, in order to avoid subjectivism in assessing the quality of perception improvement, we present the results of experimental studies that have been conducted using the introduced operators. Let n, m - raster sizes, N  n  m - the number of pixels of the raster. The image aberration in each pixel is determined as follows: i, j  pi, j  pi, j , i  1, n , j  1, m , then the average error of reproduction for each component equals 1 n m    i , j ; N i 1 j 1 constant error variance – 1 n m    i , j   .  2 2  N  1 i 1 j 1 To check the fulfillment of a condition (5) when analyzing reproduced images widely used is peak-to-peak signal-to-noise ratio - PSNR, which is defined as follows: 2552 1 2552 PSNR  10  lg  10   ln 2 . 2 ln10  The total PSNR for the image is determined by averaging the PSNR of each of the color components. The interpretation of PSNR is quite simple: the greater the value of the statistics is, the greater is the correspondence between the two images. There has been conducted an experiment for some high-quality digital image. It was aimed at comparison of the performance of the described operators and sharpen- ers presented in the Adobe PhotoShop CS6 digital image processing environment. In particular, we compared the two filters – “Unsharp Mask” and “Sharpen More filter options” presented in the “Filter” menu and in the “Unsharp Mask” submenu. The use of such filters does not require additional adjustments, so, probably, the filters them- selves are operators similar in design (6). Unfortunately, the description of the men- tioned filters is not freely available, so the comparative analysis was performed as follows. Step 1. Generate evenly distributed  ,  and require their correspondence with conditions (3) and (4). Step 2. To test the work of each of the five operators (6) and filters “Unsharp Mask” and “Sharpen More filter options” presented in Adobe PhotoShop CS6, we simulate operator distortion (2) according to the generated  ,  and  mask. Step 3. Define a PSNR, comparing it to the original image, for each image repro- duction result. Step 4. Repeat the experiment 24 times. The number of repetitions of the experiment (step 4) is not large, which (unlike the previous experiment) is due to the inability to automate the work with series of imag- es in Adobe PhotoShop CS6. However, as can be seen from the results of the experi- ment Table 1, even this number of repetitions clearly demonstrates the advantage in the use of filters (6). It is worth paying attention to the following pattern. PSNR value according to the results of the use of «Unsharp Mask» filter heavily correlates with PSNR values after the use of C0  pi, j  and C2  pi, j  operators, but the filters, re- searched in this paper, keep the advantage. The same is true for «Sharpen More filter options» filter and operator C4  pi, j  . As it can be seen from the results presented in the table, the introduced operators have advantages in comparison to the well-known «Unsharp Mask» and «Sharpen More filter options». Table 1. PSNR values after filter comparison experiment with Adobe PhotoShop CS6 №   0 1 2 3 4 «Unsharp «Sharpen Mask» More filter options» 1 0,06003 0,00097 39,87 46,42 38,71 32,92 28,77 36,85 28,27 2 0,07743 0,00657 40,36 47,50 39,18 33,24 29,00 37,17 28,43 3 0,08533 0,0009 43,54 51,20 41,88 34,71 30,06 39,73 29,50 4 0,09678 0,01427 40,37 47,68 39,24 33,34 29,08 37,11 28,45 5 0,1226 0,00797 47,77 50,61 45,44 36,49 31,28 42,67 30,58 6 0,1527 0,01274 51,43 46,63 49,14 38,34 32,50 46,10 31,68 7 0,17326 0,02473 50,95 48,90 48,50 37,89 32,17 44,15 31,17 8 0,17365 0,04012 41,40 48,00 40,44 34,45 29,93 37,75 28,99 9 0,22971 0,00407 37,43 36,20 37,76 38,91 36,62 38,42 36,63 10 0,24989 0,00879 36,91 35,77 37,23 38,64 36,98 37,84 37,06 11 0,28518 0,01956 36,47 35,37 36,80 38,60 37,80 37,38 37,92 12 0,29498 0,03736 39,30 37,30 39,92 43,63 39,48 41,13 37,98 13 0,29504 0,03268 38,23 36,59 38,74 41,73 39,36 39,71 38,49 14 0,31059 0,07654 43,04 42,51 43,30 38,79 33,14 40,32 31,24 15 0,36104 0,01354 32,33 32,04 32,44 33,05 33,21 32,44 32,87 16 0,36322 0,02906 33,71 33,15 33,90 35,06 35,95 34,11 35,88 17 0,37001 0,0009 31,11 31,01 31,17 31,44 31,33 31,02 30,49 18 0,37427 0,06638 38,35 36,55 38,97 44,31 42,85 39,87 38,84 19 0,39046 0,08296 39,84 37,71 40,65 46,78 40,30 41,06 35,99 20 0,39988 0,02496 32,03 31,75 32,15 32,77 33,07 32,15 32,77 21 0,4038 0,05305 34,55 33,76 34,82 36,69 38,73 35,21 38,69 22 0,48595 0,10096 35,94 34,75 36,35 39,81 44,01 36,83 38,91 23 0,4881 0,0434 30,89 30,73 30,97 31,40 31,66 30,91 31,19 24 0,49757 0,01782 29,32 29,39 29,35 29,35 29,11 29,08 27,91 Average: 38,55 39,23 38,21 36,76 34,43 37,46 33,33 For a more thorough check of the quality of the restortation we shall introduce the low-frequency noise operators of image distortion and we shall carry out an experi- ment of simulation modeling according to the following steps. Step 1. Generate evenly distributed  ,  and require their correspondence with conditions (3) and (4). Step 2. To check how each out of five operators (6) functions we model distortion by the operator (2), according to the generated  ,  and  mask. Step 3. Define a PSNR, comparing it to the original image, for each image repro- duction result. Step 4. Repeat the experiment 400 times. The results of the experiment are presented in Table 2. For the sake of clarity, the results of the experiments were sorted by the value of  , and the value of PSNR was averaged within the change intervals of  , (the last column indicates the number of values that were averaged). Table 2. The value of the avareged PSNR after the experiment was conducted.  max 0 1 2 3 4 N [0,05;0,1) 0.0235 38,73849 44,77572 37,71818 32,35721 28,35858 41 [0,1;0,15) 0.0340 43,98148 49,35406 42,37825 35,04397 30,28478 48 [0,15;0,2) 0.0474 45,7946 46,9347 44,81747 37,56944 32,17712 53 [0,2;0,25) 0.0589 42,83499 41,16095 43,1387 39,88584 34,79463 38 [0,25;0,3) 0.0637 40,7402 38,50291 41,46732 41,71586 36,89772 37 [0,3;0,35) 0.0829 38,37078 36,73771 38,96636 40,56728 37,20889 38 [0,35;0,4) 0.0973 36,43578 35,18418 36,89537 39,81749 37,9436 43 [0,4;0,45) 0.1002 34,04021 33,29069 34,31427 36,23808 36,87641 44 [0,45;0,5) 0.1201 32,73673 32,21276 32,93596 34,41253 35,57243 57 [0,05;0,5] 39,1156 39,7277 38,9605 37,211 37,8732 400 We shall remark, that the maximum PSNR when using the operators (6) is ob-   tained by increasing  ordinary index l. Thus for C2 pij maximum PSNR at approx-   - at   0, 45 (Fig.1). imately   0,125 , at the same time for C4 p ij Fig. 1. The values of PSNR when using operators with masks 1 (lager values respond for the smaller values of  ) and  4 : the abscissa axis -  ; the axis of ordinates – PSNR. Thus, the presented and researched linear operators (6) can be recommended for the automated processing of digital images distorted by interference, such as the mi- cro-movement of the fixation chamber, which is possible for aerial photography from the aircraft. 4. Conclusions As a result of shooting with digital cameras aimed for different purposes the obtained images can become distorted by the effects of low-frequency hindrance caused by the vibration of the carrier during photo or video shooting. To eliminate the effects of such interference we offer linear filter operators to process the obtained digital images (Fig. 2). In particular, we have presented and experimentally substantiated the linear digital image stabilization operators for the use in case of the random nature of low- frequency interference. а) в) c) d) Fig. 2. An example of a digital image: А) and С) – images distorted by a random interfer- ence; В) and D) - images after stabilization. Further research may be aimed at obtaining similar operators with a larger filter mask window width and at the extension of this approach to processing digital video streams in real time for social Internet-services. References 1. Gruzman I.: Digital image processing in information systems: Textbook / Grusman I., Kyrychuk V., Kosyh V., Peretaygin G., Spector A. – Novosibirsk: Publishig House of the NTU, 2000. – 168 p. 2. The latest image processing techniques. / Ed. A. Potapov. - Moscow: FIZMATLIT, 2008. - 496 p 3. Gonzalez R.: Digital image processing / Gonzalez R., Woods R. - Moscow: Technosphere, 2006. - 1072 p. 4. P. Prystavka “The use of combined filters based on polynomial splines in raster’s images processing” // Bulletin of NAU. – 2008. - #4. – P. 104-107. 5. Zohair Al-Ameen, Alaa Muttar, Ghofran Al-Badrani, " Improving the Sharpness of Digi- tal Image Using an Amended Unsharp Mask Filter", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.11, No.3, pp. 1-9, 2019. 6. A. Chyrkov “Suspicious Object Search in Video Stream from Aircraft Camera by Using Histogram Analysis”, Problems of Creation, Testing and Usage of Information Systems: Scientific Publications of ZhMI, Zhytomyr 13 P. 126-135 (2016). 7. J. Clark, C. Wadhwani, K. Abramovitch, D. Rice and M. Kattadiyil, "Effect of image sharpening on radiographic image quality", The Journal of Prosthetic Dentistry, vol. 120, no. 6, pp. 927-933, 2018. 8. Roman Odarchenko, Serhii Dakov, Olexandr Oksiuk and Larisa Dakova Software- Controlled Network SDN Reliability Calculation 5th International Scientific-Practical Conference Problems of Infocommunications Science and Technology, PIC S and T 2018 - Conference Proceedings, pp 99-103. 9. J. Ye, Z. Shen, P. Behrani, F. Ding and Y. Shi, "Detecting USM image sharpening by us- ing CNN", Signal Processing: Image Communication, vol. 68, pp. 258-264, 2018. 10. F. Ding, G. Zhu, W. Dong and Y. Shi, "An efficient weak sharpening detection method for image forensics", Journal of Visual Communication and Image Representation, vol. 50, pp. 93-99, 2018 11. Prystavka P., Cholyshkina O.: Components of information support for automated pro- cessing of digital images based on local operators. Actual problems of automation and in- formation technology: Coll. Sciences. wash. - D .: View of Dnepropetrovsk University – 2010. –T.14. –S.27–36.