=Paper=
{{Paper
|id=Vol-2588/paper15
|storemode=property
|title=Increasing the Image Sharpness with Linear Operator for Social Internet-Services
|pdfUrl=https://ceur-ws.org/Vol-2588/paper15.pdf
|volume=Vol-2588
|authors=Prystavka Pylyp,Cholyshkina Olha,Zhengbing Hu
|dblpUrl=https://dblp.org/rec/conf/cmigin/PylypCH19
}}
==Increasing the Image Sharpness with Linear Operator for Social Internet-Services==
https://ceur-ws.org/Vol-2588/paper15.pdf
Increasing the Image Sharpness with Linear Operator for
Social Internet-Services
Prystavka Pylyp 1 [0000-0002-0360-2459], Cholyshkina Olha 2 [0000-0002-0681-0413]
and Zhengbing Hu 2 [0000-0002-6140-3351]
1
National Aviation University, Komarova 1, Kyiv, Ukraine
2
Interregional Academy of Personal Management, Frometivska 2, Kyiv, Ukraine
3
Central China Normal University, Wuhan, China
greenhelga5@gmail.com
Abstract. In the paper it has been propounded and experimentally researched
the linear operators that can be used to sharpen digital images or video frames
distorted by a micro-motion of fixation chamber. It has been assumed that the
cause of the problem, which leads to deterioration of sharpness, is a low-
frequency interference, which is exemplified by a random non-recursive filter in
the form of a discrete convolution. It has been experimentally proven that the
proposed stabilizer filters allow for significant visual enhancement of distorted
images, with a peak-to-peak signal-to-noise ratio for the improved images high-
er than if using similar sharpening filters implemented in Adobe PhotoShop
CS6. The corresponding masks of the proposed operators and the examples of
application are given in the paper.
Keywords: Image processing, image sharpness, digital stabilization, B-spline,
linear filter operators.
1. Topicality and problem statement
An important factor for the positive perception of a digital image, or video stream, as
a sequence of frames is how realistic it is. At the same time the sharpness of an image
os one of the constituents that determine the realness of the image. The main causes
for distortion that lead to deterioration of clarity are the limited resolution capabilities
of the forming system, defocus, the presence of the distorting medium (such as the
atmosphere), the movement of the camera towards the object which is being recorded
[1]. Further we shall consider the processing of images obtained under the conditions
of the micro-motion of the fixation chamber. This defect is most common in the case
of a photography without a tripod or if shooting from a platform that may be a subject
to some mechanical impact, such as microvibration (aerophotography, etc.). Unlike
othercases, the consequences of micro-motion can be eliminated or at least signifi-
cantly offset, not only by hardware but also by mathematical treatment procedures.
Therefore, we believe the topical task is to find appropriate procedures for image
sharpening. These should have low computational complexity and upon implementa-
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib-
ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in
Global Information Networks.
tion in software products they should provide real-time processing.
In assumption of isoplanatic system of observation for intensity distribution I(x)
the image of the object, that is being formed in the x plane of its registration, is being
used an expression of the following type [2]:
I x O y H x y dy , (1)
where O(y) - the distribution of the intensity of reflection from the object of light
irradiation in the image plane y; H(x) - distribution of intensity in the image of an
axial source point (impulse response or else – scattering of the function system point).
Expression (1) provides an image that is recorded in the form of a convolution of a
true image and the impulse response of the registration system. Thus [1], the intensity
value of the original image is "smeared" at each of the points of registration in ac-
cordance with the type of function H(x).
To fix the problems caused by the micro-movement of the fixation chamber, they
use image stabilizers, which can be divided into three general types [3; 4]: mechani-
cal, electronic and digital. In this paper we research namely digital stabilizers. They
are not used to against the cause of the blurry images in photo and video material. On
the contrary they are intended to eliminate the consequences - that is, they are used
for the post-fixation mathematical processing of digital data.
There is a number of methods used to reproduce blurred images [1; 2; 5; 7-9].
However, among the effective procedures of digital image sharpening (digital stabili-
zation) that satisfy the requirements of actual processing in real time, preference
should be given to those that achieve the target processing function at a minimum of
computational operations. In fact, these are linear operators obtained in the form of a
discrete convolution of the color components of the raster and masks of filters-
stabilizers.
Assuming that the distortion of the original image has been caused by the micro-
motion of the locking chamber, as e.g. by the vibration of the aircraft during aerial
photography [6], then, according to the above-mentioned and based on (1), the fol-
lowing representation can considered reasonable for a digital image
i ri j rj
iiir jj j r iii, jj j pii, jj , i k2 , k2 , j 2 , 2 , (2)
kj kj
pi, j L pi, j
i i
i j
where pi,j - color raster component (red, green or blue); L(i,j) - linear low-pass image
filter operator; (i,j) - the pixel index of the raster; ki,kj - image frame sizes; pii, jj - the
color component of the raster of the perfect undistorted image; ii i , jj j - low pass
filter mask element; 2r 1 2r 1 - the size of the low pass filter mask.
i j
Considering a digital image, specified by the raster with any of the constituents
P pi , j ; i 0, H 1, j 0, W 1 . As an experiment, we will simulate an accidental low-
frequency interference, such as the operator (2), for some digital images. To obtain a
linear operator L pij from (2), we shall define the low pass filter mask as follows.
Assumingly pi - one-dimensional sequence of a function (for distinctness), and
i
pi i - a sequence obtained after smoothing. If
pi pi 2 pi 4 pi ,
where
2 pi pi1 2 pi pi1 ;
4 pi 2 pi1 22 pi 2 pi1 , (3)
0,05;0,5 ,
then
pi pi 2 4 pi1 1 2 6 pi 4 pi1 pi 2 ,
hence, from the condition of additionality (as for the low pass filter) with the coeffi-
cients pi , i , we get:
0, 0,
0;0, 25 . (4)
4 0, 0, 25,
1 2 6 0, 0,333 0,5 ,
With the direct multiplication it is not difficult to obtain a low-pass filter mask of
size 5x5 to determine the operator (2), namely (taking into account the symmetry:
2 42 2 6 2
42
4 2
4 1 2 6 ,
2 62 4 1 2 6 1 2 6 2
where and are defined as evenly distributed random realizations that satisfy
conditions (3) and (4).
Linear operator for the digital stabilization of images
Let’s introduce linear operators C p ij of digital image’s stabilization, as follows
, and the quality of stabilization will be considered acceptable if consider-
pi, j C p ij
ing (2) true is
pi, j pi, j , i, j (5)
for a random mask .
Linear operators, such as those considered in [6], can be used to stabilize the im-
age:
i rl j rl
Cl pij l
ii i , jj j pii , jj ,
ii i rl jj j rl
i, j , (6)
Where l=0,1,2,3,4; r0=2; r1=r2=3; r3=r4=4;
1 8 74 8 1
8 64 592 64 8
0 1 ;
74 592 5476 592 74
3136
8 64 592 64 8
1 74 1
8 8
3,75457E-09 8,93587E-07 5,40282E-06 -7,38748E-05
8,93587E-07 0,000212674 0,001285871 -0,01758221 ;
1
5,40282E-06 0,001285871 0,007774658 -0,106305883
-7,38748E-05 -0,01758221 -0,106305883 1,45356119
1,24562E-08 2,96456E-06 1,59314E-05 -0,000149424
2,96456E-06 0,000705566 0,003791678 -0,035562919 ;
2
1,59314E-05 0,003791678 0,020376288 -0,191113331
-0,000149424 -0,035562919 -0,191113331 1,792490633
1,6236E-10 4,1165E-08 9,20847E-07 2,14132E-06 -1,8949E-05
4,1165E-08 1,0437E-05 0,000233473 0,000542915 -0,004804375
9,20847E-07 0,000233473 0,00522272 0,012144825 -0,107472272 ;
3
2,14132E-06 0,000542915 0,012144825 0,028241375 -0,249914233
-1,8949E-05 -0,004804375 -0,107472272 -0,249914233 2,211546853
5,26177E-10 1,32248E-07 2,7676E-06 4,92362E-06 -3,85865E-05
1,32248E-07 3,32391E-05 0,000695605 0,001237494 -0,009698272
2,7676E-06 0,000695605 0,014557157 0,025897446 -0,202958992
4
.
4,92362E-06 0,001237494 0,025897446 0,046072025 -0,361067725
-3,85865E-05 -0,009698272 -0,202958992 -0,361067725 2,829697678
We shall notice that the coefficient of masks l , l 1, 4 can be determined by
taking into account the symmetry of the corresponding matrices.
3. Experimental studies
Operators (6) provide visual enhancement of the perception of digital images distorted
by the micro-motion of the fixation chamber, in particular data from the cameras of
the target load of aircraft. However, in order to avoid subjectivism in assessing the
quality of perception improvement, we present the results of experimental studies that
have been conducted using the introduced operators.
Let n, m - raster sizes, N n m - the number of pixels of the raster. The image
aberration in each pixel is determined as follows:
i, j pi, j pi, j , i 1, n , j 1, m ,
then the average error of reproduction for each component equals
1 n m
i , j ;
N i 1 j 1
constant error variance –
1 n m
i , j .
2
2
N 1 i 1 j 1
To check the fulfillment of a condition (5) when analyzing reproduced images
widely used is peak-to-peak signal-to-noise ratio - PSNR, which is defined as follows:
2552 1 2552
PSNR 10 lg 10 ln 2 .
2 ln10
The total PSNR for the image is determined by averaging the PSNR of each of the
color components. The interpretation of PSNR is quite simple: the greater the value of
the statistics is, the greater is the correspondence between the two images.
There has been conducted an experiment for some high-quality digital image. It
was aimed at comparison of the performance of the described operators and sharpen-
ers presented in the Adobe PhotoShop CS6 digital image processing environment. In
particular, we compared the two filters – “Unsharp Mask” and “Sharpen More filter
options” presented in the “Filter” menu and in the “Unsharp Mask” submenu. The use
of such filters does not require additional adjustments, so, probably, the filters them-
selves are operators similar in design (6). Unfortunately, the description of the men-
tioned filters is not freely available, so the comparative analysis was performed as
follows.
Step 1. Generate evenly distributed , and require their correspondence with
conditions (3) and (4).
Step 2. To test the work of each of the five operators (6) and filters “Unsharp
Mask” and “Sharpen More filter options” presented in Adobe PhotoShop CS6, we
simulate operator distortion (2) according to the generated , and mask.
Step 3. Define a PSNR, comparing it to the original image, for each image repro-
duction result.
Step 4. Repeat the experiment 24 times.
The number of repetitions of the experiment (step 4) is not large, which (unlike the
previous experiment) is due to the inability to automate the work with series of imag-
es in Adobe PhotoShop CS6. However, as can be seen from the results of the experi-
ment Table 1, even this number of repetitions clearly demonstrates the advantage in
the use of filters (6). It is worth paying attention to the following pattern. PSNR value
according to the results of the use of «Unsharp Mask» filter heavily correlates with
PSNR values after the use of C0 pi, j and C2 pi, j operators, but the filters, re-
searched in this paper, keep the advantage. The same is true for «Sharpen More filter
options» filter and operator C4 pi, j .
As it can be seen from the results presented in the table, the introduced operators
have advantages in comparison to the well-known «Unsharp Mask» and «Sharpen
More filter options».
Table 1. PSNR values after filter comparison experiment with Adobe PhotoShop CS6
№ 0 1 2 3 4 «Unsharp «Sharpen
Mask» More filter
options»
1 0,06003 0,00097 39,87 46,42 38,71 32,92 28,77 36,85 28,27
2 0,07743 0,00657 40,36 47,50 39,18 33,24 29,00 37,17 28,43
3 0,08533 0,0009 43,54 51,20 41,88 34,71 30,06 39,73 29,50
4 0,09678 0,01427 40,37 47,68 39,24 33,34 29,08 37,11 28,45
5 0,1226 0,00797 47,77 50,61 45,44 36,49 31,28 42,67 30,58
6 0,1527 0,01274 51,43 46,63 49,14 38,34 32,50 46,10 31,68
7 0,17326 0,02473 50,95 48,90 48,50 37,89 32,17 44,15 31,17
8 0,17365 0,04012 41,40 48,00 40,44 34,45 29,93 37,75 28,99
9 0,22971 0,00407 37,43 36,20 37,76 38,91 36,62 38,42 36,63
10 0,24989 0,00879 36,91 35,77 37,23 38,64 36,98 37,84 37,06
11 0,28518 0,01956 36,47 35,37 36,80 38,60 37,80 37,38 37,92
12 0,29498 0,03736 39,30 37,30 39,92 43,63 39,48 41,13 37,98
13 0,29504 0,03268 38,23 36,59 38,74 41,73 39,36 39,71 38,49
14 0,31059 0,07654 43,04 42,51 43,30 38,79 33,14 40,32 31,24
15 0,36104 0,01354 32,33 32,04 32,44 33,05 33,21 32,44 32,87
16 0,36322 0,02906 33,71 33,15 33,90 35,06 35,95 34,11 35,88
17 0,37001 0,0009 31,11 31,01 31,17 31,44 31,33 31,02 30,49
18 0,37427 0,06638 38,35 36,55 38,97 44,31 42,85 39,87 38,84
19 0,39046 0,08296 39,84 37,71 40,65 46,78 40,30 41,06 35,99
20 0,39988 0,02496 32,03 31,75 32,15 32,77 33,07 32,15 32,77
21 0,4038 0,05305 34,55 33,76 34,82 36,69 38,73 35,21 38,69
22 0,48595 0,10096 35,94 34,75 36,35 39,81 44,01 36,83 38,91
23 0,4881 0,0434 30,89 30,73 30,97 31,40 31,66 30,91 31,19
24 0,49757 0,01782 29,32 29,39 29,35 29,35 29,11 29,08 27,91
Average: 38,55 39,23 38,21 36,76 34,43 37,46 33,33
For a more thorough check of the quality of the restortation we shall introduce the
low-frequency noise operators of image distortion and we shall carry out an experi-
ment of simulation modeling according to the following steps.
Step 1. Generate evenly distributed , and require their correspondence with
conditions (3) and (4).
Step 2. To check how each out of five operators (6) functions we model distortion
by the operator (2), according to the generated , and mask.
Step 3. Define a PSNR, comparing it to the original image, for each image repro-
duction result.
Step 4. Repeat the experiment 400 times.
The results of the experiment are presented in Table 2. For the sake of clarity, the
results of the experiments were sorted by the value of , and the value of PSNR was
averaged within the change intervals of , (the last column indicates the number of
values that were averaged).
Table 2. The value of the avareged PSNR after the experiment was conducted.
max 0 1 2 3 4 N
[0,05;0,1) 0.0235 38,73849 44,77572 37,71818 32,35721 28,35858 41
[0,1;0,15) 0.0340 43,98148 49,35406 42,37825 35,04397 30,28478 48
[0,15;0,2) 0.0474 45,7946 46,9347 44,81747 37,56944 32,17712 53
[0,2;0,25) 0.0589 42,83499 41,16095 43,1387 39,88584 34,79463 38
[0,25;0,3) 0.0637 40,7402 38,50291 41,46732 41,71586 36,89772 37
[0,3;0,35) 0.0829 38,37078 36,73771 38,96636 40,56728 37,20889 38
[0,35;0,4) 0.0973 36,43578 35,18418 36,89537 39,81749 37,9436 43
[0,4;0,45) 0.1002 34,04021 33,29069 34,31427 36,23808 36,87641 44
[0,45;0,5) 0.1201 32,73673 32,21276 32,93596 34,41253 35,57243 57
[0,05;0,5] 39,1156 39,7277 38,9605 37,211 37,8732 400
We shall remark, that the maximum PSNR when using the operators (6) is ob-
tained by increasing ordinary index l. Thus for C2 pij maximum PSNR at approx-
- at 0, 45 (Fig.1).
imately 0,125 , at the same time for C4 p ij
Fig. 1. The values of PSNR when using operators with masks 1 (lager values respond for the
smaller values of ) and 4 : the abscissa axis - ; the axis of ordinates – PSNR.
Thus, the presented and researched linear operators (6) can be recommended for
the automated processing of digital images distorted by interference, such as the mi-
cro-movement of the fixation chamber, which is possible for aerial photography from
the aircraft.
4. Conclusions
As a result of shooting with digital cameras aimed for different purposes the obtained
images can become distorted by the effects of low-frequency hindrance caused by the
vibration of the carrier during photo or video shooting. To eliminate the effects of
such interference we offer linear filter operators to process the obtained digital images
(Fig. 2). In particular, we have presented and experimentally substantiated the linear
digital image stabilization operators for the use in case of the random nature of low-
frequency interference.
а) в)
c) d)
Fig. 2. An example of a digital image: А) and С) – images distorted by a random interfer-
ence; В) and D) - images after stabilization.
Further research may be aimed at obtaining similar operators with a larger filter
mask window width and at the extension of this approach to processing digital video
streams in real time for social Internet-services.
References
1. Gruzman I.: Digital image processing in information systems: Textbook / Grusman I.,
Kyrychuk V., Kosyh V., Peretaygin G., Spector A. – Novosibirsk: Publishig House of the
NTU, 2000. – 168 p.
2. The latest image processing techniques. / Ed. A. Potapov. - Moscow: FIZMATLIT, 2008.
- 496 p
3. Gonzalez R.: Digital image processing / Gonzalez R., Woods R. - Moscow: Technosphere,
2006. - 1072 p.
4. P. Prystavka “The use of combined filters based on polynomial splines in raster’s images
processing” // Bulletin of NAU. – 2008. - #4. – P. 104-107.
5. Zohair Al-Ameen, Alaa Muttar, Ghofran Al-Badrani, " Improving the Sharpness of Digi-
tal Image Using an Amended Unsharp Mask Filter", International Journal of Image,
Graphics and Signal Processing(IJIGSP), Vol.11, No.3, pp. 1-9, 2019.
6. A. Chyrkov “Suspicious Object Search in Video Stream from Aircraft Camera by Using
Histogram Analysis”, Problems of Creation, Testing and Usage of Information Systems:
Scientific Publications of ZhMI, Zhytomyr 13 P. 126-135 (2016).
7. J. Clark, C. Wadhwani, K. Abramovitch, D. Rice and M. Kattadiyil, "Effect of image
sharpening on radiographic image quality", The Journal of Prosthetic Dentistry, vol.
120, no. 6, pp. 927-933, 2018.
8. Roman Odarchenko, Serhii Dakov, Olexandr Oksiuk and Larisa Dakova Software-
Controlled Network SDN Reliability Calculation 5th International Scientific-Practical
Conference Problems of Infocommunications Science and Technology, PIC S and T 2018
- Conference Proceedings, pp 99-103.
9. J. Ye, Z. Shen, P. Behrani, F. Ding and Y. Shi, "Detecting USM image sharpening by us-
ing CNN", Signal Processing: Image Communication, vol. 68, pp. 258-264, 2018.
10. F. Ding, G. Zhu, W. Dong and Y. Shi, "An efficient weak sharpening detection method for
image forensics", Journal of Visual Communication and Image Representation, vol. 50, pp.
93-99, 2018
11. Prystavka P., Cholyshkina O.: Components of information support for automated pro-
cessing of digital images based on local operators. Actual problems of automation and in-
formation technology: Coll. Sciences. wash. - D .: View of Dnepropetrovsk University –
2010. –T.14. –S.27–36.