=Paper= {{Paper |id=Vol-3156/paper35 |storemode=property |title=On Image Complexity in Viewpoint of Image Processing Performance |pdfUrl=https://ceur-ws.org/Vol-3156/paper35.pdf |volume=Vol-3156 |authors=Vladimir Lukin,Sergey Krivenko,Fangfang Li,Sergey Abramov,Victor Makarichev |dblpUrl=https://dblp.org/rec/conf/intelitsis/LukinKLAM22 }} ==On Image Complexity in Viewpoint of Image Processing Performance== https://ceur-ws.org/Vol-3156/paper35.pdf

On Image Complexity in Viewpoint of Image Processing
Performance
Vladimir Lukinа, Sergey Krivenkoа, Fangfang Liа, Sergey Abramovа and Viktor Makarichevа
a
National Aerospace University, 17 Chkalova Street, Kharkiv, 61070, Ukraine

Abstract
It is well known that image processing efficiency considerably depends on image properties.
Among operations of image processing, we mean quality assessment, noise characteristic
estimation, lossless and lossy compression, denoising, etc. In many papers, such terms as
“image complexity”, “rich image content”, “highly textural image” are used. Their meaning is
intuitively clear and described verbally but there is a limited amount of quantitative estimation
and analysis. In this paper, we show that there is a very high correlation of performance of
different operations of image processing like characteristics of lossless and lossy compression,
blind variance estimation and denoising and so on. We also recall what quantitative parameters
indirectly describe image complexity for different operations of image processing.

Keywords 1
Image complexity, performance, image processing operations, rank correlation.

1. Introduction
Images have become an essential part of our life [1-3]. They are applied in numerous areas like
entertainment and advertising [4, 5], ecological monitoring, agriculture [6], forestry [7], etc. Images are
subject to many different operations at stages of their acquisition, transferring, pre-processing,
enhancement, and so on [1]. This can be quality estimation [8], blind evaluation of noise type and
characteristics [9], denoising and enhancement [4], lossless and lossy compression [10], object
detection and classification [11], etc.
Certainly, not all these operations are employed in an image processing chain for a given image and
a given application. Some of them are not needed or can be skipped. Meanwhile, practically each
scientist/researcher dealing with image processing knows that there are images processing of which is
relatively easy and successful and there are images processing of which is problematic (difficult, does
not bring an obvious success). The latter usually happens for images that have a small percentage of
pixels belonging to homogeneous regions, contain a lot of small-sized objects, edges, textures. For
example, designers of nonlinear filters (denoising techniques) for grayscale and color images know [12]
that it is considerably more difficult to get good results for the test image Baboon than for the test
images Lena and Peppers. This is one of reasons why it is usually recommended to test performance of
an image processing method for a set of images containing images with different properties. This was
also the reason for creating image databases like LIVE (http://live.ece.utexas.edu/research /quality/),
TID2008 (http://www.ponomarenko.info/tid2008.htm) and TID2013 (http://www.
ponomarenko.info/tid2013.htm), Kodak (http://r0k.us/graphics/kodak/), etc. Nowadays, there is a
tendency of creating medical (for example, SICAS Medical Image Repository, https://www.smir.ch)
and remote sensing (for instance, ESA “Sentinel-2”, https://sentinel.esa.int/web/sentinel/missions/

IntelITSIS’2022: 3rd International Workshop on Intelligent Information Technologies and Systems of Information Security, March 23–25,
2022, Khmelnytskyi, Ukraine
EMAIL: lukin@ai.kharkov.com (V. Lukin); krivenkos@ieee.org (S. Krivenko); liff_niat@yahoo.com (Fangfang Li); s.abramov@khai.edu
(S. Abramov); victor.makarichev@gmail.com (V. Makarichev)
ORCID: 0000-0002-1443-9685 (V. Lukin); 0000-0001-6027-5442 (S. Krivenko); 0000-0002-7392-586X (Fangfang Li); 0000-0002-8295-
9439 (S. Abramov); 0000-0003-1481-9132 (V. Makarichev)
©️ 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
sentinel-2) image databases for different purposes, in particular, for training modern neural networks
(deep learning) for which a huge volume of training and verification data is needed. There are also
texture image database USC-SIPI (https://sipi.usc.edu/database/), European Space Agency database of
color images (https://www.esa.int/ESA_Multimedia/Images) and many others. The existence of such
databases offers such opportunities as statistical analysis of indicators of image processing efficiency
[13], detection of outlying results, anomalies and “strange” images [14], thorough comparisons of
methods’ performance [15], scatter-plot obtaining and regression (curve fitting) [16, 17], etc.
As the result of such analysis for a particular application, images are often classified to simple
structure and complex structure (textural) ones where there are no objective criteria for such a
classification (or the criteria are mostly intuitive or expressed verbally). Because of this, the goal of this
paper is to generalize the problem, to show some criteria characterizing image complexity, and to
demonstrate inter-connection between such criteria and image processing efficiency.

2. Lossless and lossy compression
Probably, the most known characteristic (criteria) of image complexity is its entropy that directly
influences image compression ratio for lossless compression (coding) techniques [18, 19]. Let us give
one example. Consider 25 color test images used in the databasesTID2008 and TID2013. Twenty four
of them are natural scene images and one (#25) is artificial. Small copies of these images are given in
Fig. 1. As one can see, images are of different complexity where the test image #13 is the most complex
since it is practically impossible to find homogeneous (non-textural) fragments for it. Meanwhile, there
are also quite simple structure images like, e.g., the test images ## 3 or 20 with rather large
homogeneous regions.
These properties appear themselves in entropy E calculated for all 25 test images. Its values are
given in Fig. 2,a. They are in the limits from about 6 for the test image # 2 till about 7.5 for the test
images ## 13, 14, and 19. Alongside, Fig. 2 shows the values of compression ratio (CR) attained by
ZIP. Comparison of data in Figures 2,a and 2,b clearly shows that a larger CR is usually obtained for
images having smaller entropy values and vice versa. The Spearman rank order correlation coefficient
(SROCC) is equal to -0.62 and its quite large absolute value shows that there is a rather strict rank
correlation between E and CR.
Let us also present some data for grayscale images. In the paper [14], a set containing 61 images has
been analyzed. This set included standard test images as Baboon, Lena and others, a lot of texture
images from the USC-SIPI database, and some others. Entropy values have been determined and they
are shown by dots of different colors in Fig. 3. As one can see, E values are from 0.5 till 7.9, whist CR
for lossless compression CRlossless varies from 100.7 to 1. Correlation factors for these two indicators
(parameters) are the following: -0.74 for Pearson correlation, -0.88 for Spearman rank correlation, and
-0.76 for Kendall rank correlation. In other words, a larger CRlossless usually relates to a simpler structure
image with smaller entropy. In [14], it has been proposed to divide images into three groups: 1) complex
structure images having E>7; 2) middle complexity images for which 6≤E≤7; 3) simple structure images
with E<6. Fig. 4 shows examples for all three groups. The entropy values are equal to 5.82, 6.80, and
7.36, respectively.
It is also interesting that among simple structure images (E<6) there are three images that have very
small E (about 3 and less). Analysis shows that these are artificially created images or scanned copies
of such images (see Fig. 5) that have large homogeneous regions (or a large percentage of pixels that
belong to such regions). Further analysis will show that performance characteristics of image processing
operations can differ a lot from performance characteristics for natural scene images. This indicates that
it is necessary to use such artificial images with care in analysis of different operations of image
processing (denoising, compression, classification) since conclusions and tendencies obtained based on
the corresponding studies can be wrong or, at least, be aside of tendencies and conclusions for most
typical images met in practice. In other words, if there is no possibility to carry out studies concerning
image processing except of using artificially created images, it is worth making such images as close to
natural as possible.
In lossy image compression [10, 14, 16], there is a much wider variety of performance
characteristics. First, CR can be varied where this is done in different ways in different coders. CR can
be controlled and varied by quality factor, scaling factor, quantization step, bits per pixel (BPP) and so
on. Second, since distortions (losses) are introduced, they can be characterized using various criteria.
These can be traditional criteria as mean square error (MSE) or peak signal-to-noise ratio (PSNR)
strictly connected with MSE; meanwhile, visual quality metrics (see [14, 26] and the metric PSNR-
HVS-M, http://www.ponomarenko. info/psnrhvsm.htm) are also widely used since visual inspection is
often an ultimate goal of compressed images.

1 2 3 4 5

6 7 8 9 10

11 12 13 14 15

16 17 18 19 20

21 22 23 24 25

Figure 1: Small copies of color images in TID2008

a b
Figure 2: Entropy (a) and CR (b) values for 25 test color images in TID2008

Figure 3: Entropy values for 61 test grayscale images

a b c
Figure 4: Examples of simple (a), middle (b) and complex (c) structure grayscale images

Visual quality metrics take into account some peculiarities of human vision system (HVS). For
example, the metric PSNR-HVS-M (see PSNR-HVS-M official website http://www.ponomarenko.
info/psnrhvsm.htm) takes into account two properties: a larger sensitivity to distortions in low spatial
frequencies and a masking effect of texture. PSNR-HVS-M is expressed in dB and it has smaller values
for more degraded images. Due to aforementioned properties, PSNR-HVS-M is usually larger than
PSNR. This takes place until distortions due to lossy compression become too large and spatially
correlated, then masking effect disappears and PSNR and PSNR-HVS-M become practically equal or
even PSNR-HVS-M occurs to be smaller than PSNR. Such an example is shown in Fig. 5,a for the
coder SPIHT (http://www.spiht.com/). Recall that since compression parameters can be varied we have
to analyze the so-called rate/distortion curves that can be presented in different ways. For SPIHT, this
is dependence of a metric on BPP whilst, e.g., for the coder BPG (better portable graphics,
https://bellard.org/bpg/) these are dependences of a metric on parameter Q that controls compression
(Q falls into integer values from 1 to 51 and a larger Q results in worse quality). Examples of
rate/distortion curves are given in Fig. 5,b. Note that usually the rate/distortion curves are smooth
monotonous functions (Fig. 5,b). Meanwhile, they can be “not very smooth” like in Fig. 5,a and even
not monotonous as this happens for “strange images” – see examples in Fig. 5,c and 5,d.
Behavior of the rate/distortion curves for particular images has common features but is individual.
The examples in Fig. 6 clearly show this for two coders: SPIHT and AGU. As one can see, there are
sufficient differences in PSNR values for the same BPP for SPIHT and QS for AGU.
a b

c d
Figure 5: PSNR and PSNR-HVS-M on BPP for SPIHT, the test image Goldhill (a), PSNR and PSNR-HVS-M
on Q for BPG, the test image Goldhill (b), PSNR on quantization step (QS) for the coder AGU
(http://www.ponomarenko.info/agu.htm) for “strange images” (c), PSNR-HVS-M on QS for AGU for
“strange images” (d)

a b
Figure 6: Examples of rate/distortion curves for particular images for different coders: PSNR on BPP
for SPIHT (a) and PSNR on QS for AGU (b)

Although some curves intersect, it is quite probable that if a metric value for an image1 is larger
than for an image2 for BPP1, it will be also larger for BPP2. Similar holds for rate/distortion curves for
the coder AGU – the curves for more complex structure images go “below” the curves for simpler
structure images.
It is easy to show that entropy that characterizes image complexity is highly correlated with
performance of lossy image compression. But it is also possible to show that other performance
parameters are also highly correlated. We have analyzed CR (compression ratio for ZIP highly
correlated with entropy) and parameters PSNR40, PSNR35, PSNR30, PSNRHVSM40,
PSNRHVSM35, PSNRHVSM30 where, e.g., PSNR40 means PSNR provided by AGU coder for
QS=40. It occurs that there is correlation close to unity (both Pearson and Spearman) for such pairs of
parameters as PSNR40 and PSNR35, PSNR30 and PSNR40, PSNR35 and PSNR30, PSNRHVSM40
and PSNRHVSM35, PSNRHVSM30 and PSNRHVSM40, PSNRHVSM35 and PSNRHVSM30. There
is also a rather high correlation (about 0.6) for such pairs of parameters as PSNR30 and PSNRHVSM30,
PSNR35 and PSNRHVSM35. This means that such parameters as, e.g., PSNR30 or PSNR35 are able
to characterize image complexity. Larger values of these parameters relate to simpler structure images.
Correlation of these parameters with CR is not high – SROCC is of the order 0.13.
Note that there is no necessity to carry out compression and decompression of an image compressed
by AGU with QS=30 to determine PSNR30 and compare it to a threshold to classify this image. There
is a very fast method that allows predicting PSNR for a given QS by estimating MSE in a limited
number (e.g., 500) of 8x8 blocks for which direct DCT is carried out.
All data and conclusions that are given above relate to images that are practically noise-free. If
images to be compressed are noisy, lossless compression is practically useless since it produces very
small values of CR whilst lossy compression has specific features that will be considered later.
Two things are worth noting here. First, analysis of entropy or CR for ZIP can be useful in other
applications. For example, analysis of CR for electrocardiograms compressed by ZIP has allowed to
assess ECG quality. Analysis of entropy can indicate that images have been pre-compressed. For
example, Sentinel sites offer images that have been slightly compressed by JPEG2000. This leads to
entropy values that are slightly smaller than for uncompressed images. For example, for visualized
three-channel images in Fig. 7 the entropy values are equal to 4.15, 5.67, 4.46, and 5.72, respectively.
As one can see, entropy values are larger for the images SS2 and SS4 which are obviously more textural.
These images are compressed worse (CR for them is smaller than for the images SS1 and SS3 for the
same quality characterized by PSNR of PSNR-HVS-M) and they are classified worse [18] irrespectively
are the compression techniques based on DCT used or based on atomic functions employed [18].

3. Noise characteristic estimation, image filtering and lossy compression of
noisy images
In many practical situations, noise type and/or characteristics are unknown a priori and this makes
problems for image processing since many techniques exploit this information for providing high
efficiency (good examples are well known local statistic Lee and sigma filters that have versions for
additive and multiplicative noise). Thus, noise type and characteristics have to be estimated once for a
certain type of images with stable noise characteristics or each time if these characteristics can vary. If
the number of such images or its components (like in hyperspectral data) is large, it is difficult to carry
out such estimation “manually” (in interactive way) using special tools of image analysis. Then, one
has to apply automatic or blind methods for estimation of noise characteristics [9].
Let us quickly recall what characteristics of the noise can be estimated. If the noise is white, these
are statistical characteristics like noise variance for additive noise, relative variance if noise is pure
multiplicative, and two or several parameters if noise is signal-dependent. If noise is not white, then
spatial spectral or correlation characteristics have to be estimated. The most complicated situation is,
probably, for the noise with non-stationary characteristics where variations of spatial and spectral
properties are possible.
The main problem in blind estimation of noise characteristics (BENC) is to separate noise from
signal/noise (image/noise) mixture. Pre-filtering and further analysis of difference data is often
inefficient since filtering without exploiting noise characteristics is far away from perfect and, thus,
difference signal, in addition to the noise, contains errors due imperfect pre-filtering. Most existing
approaches try to detect (extract) image regions (blocks) that are, most probably, homogeneous and for
which it is possible to obtain local estimates of noise characteristics that are quite close to the true ones.
For this purpose, different methods and mathematical tools are used. Fractal theory can be employed
to discriminate texture informative and noise informative regions. Robust estimation and robust curve
fitting techniques are used to minimize the influence of abnormal (highly erroneous) local estimates of
noise characteristics obtained in heterogeneous (locally active) blocks falling into texture regions and/or
edge/detail neighborhoods [21].

a b

c d
Figure 7: Three-channel Sentinel-2 images SS1 (a), SS2 (b), SS3 (c), and SS4 (d)

Here we do not want to go into theory and practice of BENC deeply. Our desire is to demonstrate
that image complexity and noise characteristic estimates are highly correlated. For this purpose, let us
consider the most typical case of zero mean additive white Gaussian noise (AWGN). Some data are
presented for the so-called minimal inter-quantile method of noise variance estimation in grayscale
images in [21]. It is shown that estimation bias and variance are sufficiently larger for the highly textural
image Baboon compared to the middle complexity images Barbara and Goldhill.
A more complex case has been analyzed by us where it has been supposed that noise is independent
in color (RGB) components and has equal variances. A modification of the method [21] has been
applied. Instead of obtaining local estimates in blocks in spatial domain, estimates of noise standard
deviation have been obtained in 8x8 blocks using robust processing of discrete cosine transform
coefficients. An example of the obtained estimates of the noise standard deviation is presented in Fig.
8. Note that it is usually desired to get the standard deviation estimates that differ from the true values
by no more than 10%. Because of this, two horizontal lines show the desired margins of estimates.

Figure 8: Estimates of the noise standard deviation for R, G, and B color components for noise variance
equal to 65 for images of TID2008 (Fig. 1)

As one can see, there are some estimates below the lower margin, e.g., for red and green components
of the test image # 6. These estimates are smaller than desired due to the noise saturation effects
observed for very intensive (almost white) and very dark (almost black) regions of images.
To give a better explanation of basic principle of operation for many BENC methods, consider the
histograms of local estimate distribution for three cases. Fig. 9,a shows a typical distribution obtained
for the red component of the test image # 3 from TID2008 (Fig. 1) for the noise variance equal to 65.
The local estimates of the noise variance have been obtained in 5x5 blocks in spatial domain. As one
can see, there is a mode of local estimates (approximately equal to the true value) that correspond to
normal local estimates and a right hand heavy tail that relates to abnormal local estimates. Thus, the
task is to find the distribution mode quite accurately. However, there are cases when there are two
modes where one mode corresponds to erroneous local estimates (Fig. 9,b) that result from clipping. It
is also possible that there are very many abnormal local estimates that form their own mode that has no
relation to the true value of the noise variance.
One observation is that the estimates of noise standard deviation are the largest for textural images
## 5, 8, 13, 18. This means that despite of the efforts to minimize the image content influence on noise
characteristic estimation it still exists.
Fig. 10 shows the noise variance estimates for inter-quantile method operating in spatial domain.
Again most estimates are within the required limits (20% with respect to the true value). However, there
are two estimates (for the image # 20, green and red components) that are smaller than required due to
clipping effects and a few estimates that are larger than required due to high complexity of image
content.
We have calculated SROCC between variance estimates and E (calculated for noise-free
components of color images). The SROCC values are equal to 0.53, 0.28, and 0.18 for red, green, and
blue components, respectively. Therefore, the estimates have a certain correlation with entropy.
One problem is that the estimated SROCC is for variance estimates and entropy for noise-free
images where the latter ones are absent. Hence, we have also calculated E for noisy images and
determined SROCC for E and the estimates in Fig. 10. They are equal to 0.31, 0.07, and 0.03,
respectively. Thus, for noisy images their entropy cannot serve as a feature characterizing their
complexity and some other parameters (features) should be employed. (note that E values for noisy
images are from 6.25 for the image # 2 till 7.62 for the image # 25).
200

150

100

0
0 50 100 150 200 250 300
a
300

200

100

0
0 50 100 150 200 250 300

b
60

0
0 50 100 150 200 250 300

c
Figure 9: Examples of histograms of local estimates in 5x5 blocks for noise variance equal to 65 for the
standard case (red component, image # 3); image with clipping effect (green component, image # 20),
and highly textural image (green component, image # 13)

300
Blue component
Green component
250 Red component

200
 ^n 2

150

100

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Test image index

Figure 10: Blind estimates of AWGN variance for interquantile method [21] operating in spatial domain
for components of color images from TID2008 for noise variance equal to 65

Meanwhile, as an auxiliary information, the interquantile method [21] is able to produce an estimate
of percentage p of homogeneous image blocks. Such estimates for images in TID2008 corrupted by
AWGN with the same variance values in all components are represented in Fig. 11. As one can see, this
percentage is the largest for simple structure images ## 3, 15, and 23 that really contain quite large
quasi-homogeneous regions. On the contrary, the percentage estimates are the smallest for highly
textural images ## 5, 8, 18. For the image # 13 which is textural as well, the percentage estimate is
larger since the local estimate histogram has a very specific appearance (Fig. 10,c).

Figure 11: Estimates of percentage of homogeneous blocks in components of color images in TID2008
for noise variance equal to 65 in all components

Let us now analyze how image complexity influences visual quality of noisy images and efficiency
of their denoising. It has been mentioned earlier that noise can be fully or partially masked by textures
and then image quality is perceived better than when noise is visually seen. Because of masking effects,
PSNR-HVS-M is larger than PSNR for a given image if the masking effect takes place. Fig. 12 shows
PSNR-HVS-M values (thick lines of the corresponding color) for component noisy images of TID2008
for the case of AWGN with variance equal to 65 (PSNR is about 30 dB). As one can see, all PSNR-
HVS-M values are larger than 30 dB where the largest values take place for the most textural (complex
structure) images ## 1, 5, 6, 13, 14, 25, 18. Meanwhile, the smallest values are observed for the simple
structure images ## 3, 15, 16, 20, and 23. The SROCC for PSNR-HVS-M and E calculated for noisy
images is equal to 0.55, 0.59, and 0.57, respectively. This shows that the correlation is quite high but
not large enough to accurately predict performance characteristics.

PSNR − HVS − M k (n), dB
41

32 0 5 10 15 20 25
n
Figure 12: Values of PSNR-HVS-M for original (noisy) and filtered component images (TID2008), noise
variance is equal to 65
Concerning image denoising, there are numerous existing methods. Our goal is not to study and
compare them, but to analyze general tendencies of denoising efficiency. In this sense, it is worth
considering potential and attained (practical) efficiency of image filtering. In the sense of potential
efficiency, very interesting results have been obtained by P. Milanfar et al in [22-25]. We have
determined potential efficiency for grayscale images corrupted by AWGN processed by non-local
filters. Some data have been got for component images of TID2008 (see data in Table 1 for AWGN
variance equal to 65). Potential (lower bound) efficiency is characterized by MSElb, the smaller the
better. As one can see, MSElb for different color components are practically the same. This is explained
by the fact that color components of the same image are highly correlated and, thus, are practically of
the same complexity. The smallest MSElb are observed for simpler structure images, e.g., images ## 10
and 23. In turn, the largest values are observed for the most complex structure images ## 13, 1, and 5.
It is interesting that MSElb is only 1.5 times smaller than noise variance, i.e. even potentially the noise
removal efficiency is very low.

Table 1
Lower bound MSE and output MSE for the DCT-based filter for components of color images in TID2008
Image index R component G component B component
in TID2008 MSElb MSEDCT MSElb MSEDCT MSElb MSEDCT
1 28.9 36.8 29.9 36.5 28.9 36.8
5 27.5 30.4 28.0 30.9 26.4 30.8
6 23.7 31.2 23.5 31.2 22.4 30.8
8 23.2 32.5 23.3 32.1 22.6 32.0
10 7.1 17.6 7.5 18.1 7.3 17.8
11 16.4 26.4 16.2 26.3 15.3 25.8
13 41.0 46.6 42.6 46.2 37.2 46.3
14 20.7 31.0 20.0 30.6 19.5 30.7
18 21.4 28.7 20.4 28.1 17.4 28.3
23 4.6 12.9 4.7 12.7 4.8 12.6
25 14.7 19.4 14.0 18.2 13.5 18.3

Among possible methods of image denoising, consider the DCT-based filter. It is one of the best
transform-based filters which is only a little worse compared to the BM3D filter [15]. The MSE values
obtained for the DCT-based filter (denoted as MSEDCT) are given in Table 1. As one can see, MSEDCT
are almost the same as MSElb for complex structure images. Meanwhile, there are sufficient differences
between MSEDCT and MSElb (they differ by several times) for simple structure images.
However, it does not mean that efficiency of denoising for simple structure images is low.
Comparison of PSNR-HVS-M before and after filtering (Fig. 12) shows that the largest differences in
visual quality take place just for simple structure images whilst filtering can be practically useless for
the complex structure images (see data for the image # 13). Thus, image complexity plays the key role
in image filtering efficiency. Additional materials concerning noise variance estimation and filtering
efficiency can be found in [26].
Keeping this in mind, we have started to develop the direction of studies dealing with prediction of
image denoising efficiency and decision undertaking on expedience of filtering use for a given image.
It contains a thorough analysis of filtering efficiency for quite many test images, a set of modern filters
and several quality metrics. The outcomes of this study are very interesting. First, it is shown that
filtering efficiency can be predicted for many modern filters including transform-based, nonlocal and
other ones, i.e. filters based on different principles. Second, it is demonstrated that filtering efficiency
for many modern filters is of the same order and close to potential efficiency for texture images. Third,
improvement of many metrics due to filtering can be quite accurately predicted even if one or two input
parameters are used.
Here it is worth recalling the main principle and assumptions put into basis of filtering efficiency
prediction. The following has been assumed: 1) there is a parameter that adequately characterizes
filtering efficiency (e.g., improvement of PSNR or PSNR-HVS-M due to filtering); 2) there exist one
or several parameters that adequately characterize image and noise properties; 3) there is a rather strict
dependence between input parameter(s) and a considered parameter characterizing filtering efficiency
where this dependence is established in advance (before filtering) and it is quite simple; 4) input
parameter(s) can be calculated easily and quickly (faster than image denoising) to make possible
decision on is it worth filtering a given image or it is reasonable to skip denoising and save time and
resources.
It has been shown that all these assumptions are valid and efficient prediction is possible. Many
variants have been proposed. One of the main input parameters is Pkσ (or its variants for signal-
dependent noise) where k is a factor and σ is AWGN standard deviation supposed to be known in
advance. The parameter Pkσ is determined as a mean of local estimates Pkσ(n), n=1,…,N in N 8x8 pixel
blocks. A local estimate Pkσ(n) is calculated as Nkσ(n)/63 for the n-th 8x8 pixel block where Nkσ(n) is
the number of DCT coefficients that have absolute values smaller than kσ (the DCT coefficient that
corresponds to the block mean is excluded from analysis, k is usually set equal to 2 although other
variants are possible, e.g., P0.5σ can be analyzed).
The parameter Pkσ jointly characterizes image complexity and noise intensity. Its small values (for
example, P0.5σ about 0.15) correspond to complex structure images and/or low intensity noise. On the
contrary, large values (for example, P0.5σ about 0.35) relate to simple structure images and/or high
intensity noise for which the filtering efficiency is usually high, i.e. large improvements of PSNR and
PSNR-HVS-M are observed. The difference in these improvements for images of different complexity
can be up to 10 dB.
The use of more than one input parameter improves prediction making it more accurate. This can
be illustrated by the plot in Fig. 13 where points show the estimated parameters for images used in
analysis and the regression plain is presented (fitted). Note that it is not a problem to calculate noisy
image PSNR if noise variance is known a priori or pre-estimated with high accuracy whist σ has to be
known for calculation of P0.5σ as well. Also note that more complicated but more accurate prediction
approaches have been proposed recently based on trained neural networks [26]. They are applicable to
images corrupted by signal-dependent spatially correlated noise (speckle) typical for synthetic aperture
radar images produced by Sentinel-1. An advantage of this approach is that, based on prediction carried
out for the Lee filter with different scanning window sizes, the optimal size is recommended.
One idea that follows from the results obtained in [26] is that image quality and complexity can be
assessed using neural network training approach.

Figure 13: Illustration of prediction of improvement of PSNR have two input parameters, P0.5σ and
PSNR for noisy image
The last application considered here is lossy compression of noisy images. The known peculiarity
is the noise filtering effect [27]. One more peculiarity is that performance can be analyzed with respect
to noisy image (subject to compression) and noise-free (ideal) image. Dependences of traditional type,
e.g., when a metric is determined for the compressed image and image subject to compression, behave
as they are thought conventionally, i.e. the compressed image quality decreases if CR increases. Here
it is worth mentioning the recently obtained dependences for the BPG coder, an example is given in
Fig. 14,a. As one can see, the dependences are almost linear in the limits from Q≈15 till Q≈36 where
PSNR is approximately equal to input PSNR (about 26.7 dB for the considered case of AWGN with
σ=12) where Q is the parameter that controls compression in the BPG coder.
Meanwhile, the dependences of PSNRtc calculated for the compressed noisy and noise-free (true)
images behave in a specific manner (Fig. 14,b). There can be maxima of these curves observed for such
Q that PSNRnc≈PSNRinp=10lg(2552/σ2). Then, if σ2 is known, it is easy to determine Q.

b
Figure 14: Examples of dependences PSNRnc(Q) (a) and PSNRtc(Q) (b) for three test grayscale images
corrupted by AWGN with σ2=144
The argument for which maximum takes place is the so-called optimal operation point (OOP). If it
exists (and this happens for simple structure images corrupted by rather intensive noise), it is worth
compressing a noisy image in OOP or its neighborhood (we say neighborhood because noise standard
deviation can be known or pre-estimated not accurately). An example of image compression in OOP is
given in Fig. 15. Noise filtering effect is seen in homogeneous image regions.
In turn, if OOP does not exist, it is worth using a slightly smaller Q than for OOP (e.g., Q=33 for
the case considered in Fig. 14 for the test image Diego which is highly textural). Thus, again the image
processing parameters should be adapted to image complexity.

Figure 15: Original (noisy) image (left) with σ2=100 and this image compressed in OOP (right)

The procedure of OOP existence for the BPG coder has not been developed and tested yet.
However, it has been developed and tested for the DCT-based coder AGU [25]. Not surprisingly, the
parameter P2σ again allows prediction (see the scatter-plot in Fig. 16). This time it allows predicting the
difference of a metric for potential OOP and for noisy image. The polynomial of the fifth order is fitted
very well and it shows that for P2σ>0.81 the OOP exists with high probability. Recall that this happens
if an image has a quite simple structure and/or noise is intensive.

Figure 16: The scatter-plot of difference PSNR vs P2σ for the AGU coder for images corrupted by AWGN

We hope that the methodology for prediction OOP existence for the BPG coder will be developed
soon. Meanwhile, not only OOP existence can be predicted (it is supposed that OOP according to a
given metric exists if the predicted metric improvement is positive). The metric value itself can be
predicted as well.
The presented materials show that for the case of noisy images not entropy but other parameters
relate to image complexity. An image can be considered simple if efficiency of its processing is high
(metric improvement due to filtering is large, OOP exists for lossy compression). Obviously, noise
properties play the role too.

4. Conclusions
In this paper, we have tried to provide a systematic analysis of image complexity and its influence
on image processing performance. It has been shown that, for noise-free images, the entropy can be a
good characteristic of image complexity that strictly affects lossless compression and has a sufficient
impact on characteristics of lossy compression.
If noise is present, then entropy becomes almost useless to characterize image complexity. Other
statistical parameters that jointly describe image properties and noise intensity can be used as indicators
of image complexity. The image complexity sufficiently influences potential and practical efficiency
of noise suppression in terms of conventional and visual quality metrics. It also has impact on existence
of optimal operation point in lossy compression of images corrupted by different types of noise for
various compression techniques.
Meanwhile, the task of designing a unified metric for characterizing the image complexity is not
solved yet. Maybe, the use of neural networks able to unite different indicators in nonlinear manner can
help.

5. References
[1] E. Provenzi (Ed.), Special issue: Color Image Processing, 2018. doi: 10.3390/books978-3-03842-
958-6.
[2] D. K. Pillai, New Computational Models for Image Remote Sensing and Big Data, in: P.
Swarnalatha, P. Sevugan, (Eds.), Big Data Analytics for Satellite Image Processing and Remote
Sensing, IGI Global, 2018, pp. 1–21.
[3] D. Shen, G. Wu, and H.-I. Suk, Deep learning in medical image analysis, Annual review of
biomedical engineering 19 (2017), 221-248. doi: 10.1146/annurev-bioeng-071516-044442
[4] A. Pizurica, L. Platisa, T. Ruzic, B. Cornelis, A. Dooms, M. Martens, H. Dubois, B. Devolder, M.
De Mey, I. Ingrid, Digital Image Processing of The Ghent Altarpiece: Supporting the painting's
study and conservation treatment, IEEE Signal Processing Magazine 32 (2015) 112-122. doi:
10.1109/MSP.2015.2411753.
[5] E.V. Bataeva, Y.A. Polyakova, Motive-analysis of TV-advertising visual content,
Sotsiologicheskiy Zhurnal, 24 (2018) 66–89.
[6] N. Kussul, M. Lavreniuk, A. Shelestov, S. Skakun, Crop inventory at regional scale in Ukraine:
Developing in season and end of season crop maps with multi-temporal optical and SAR satellite
imagery, European Journal of Remote Sensing 51 (2018) 627-636. doi: 10.1080/22797254.
2018.1454265.
[7] A.L. Mitchell, A. Rosenqvist, B. Mora, Current remote sensing approaches to monitoring forest
degradation in support of countries measurement, reporting and verification (MRV) systems for
REDD+, Carbon Balance Manage 12 (2017). doi: 10.1186/s13021-017-0078-9.
[8] K. Okarma, Current Trends and Advances in Image Quality Assessment, Elektronika Ir
Elektrotechnika, 25 (2019) 77-84. doi: 10.5755/j01.eie.25.3.23681.
[9] Y. Zhang, G. Wang, J. Xu, Parameter estimation of signal-dependent random noise in CMOS/CCD
image sensor based on numerical characteristic of mixed Poisson noise samples. Sensors 18
(2018). doi: 10.3390/s18072276.
[10] I. Blanes, E. Magli, J. Serra-Sagrista, A Tutorial on Image Compression for Optical Space Imaging
Systems, IEEE Geoscience and Remote Sensing Magazine 2 (2014) 8-26. doi:
10.1109/MGRS.2014.2352465.
[11] M. Radosavljevic, B. Brkljac, P. Lugonja, V. Crnojevic, Z. Trpovski, Z. Xiong, D. Vukobratovic,
Lossy Compression of Multispectral Satellite Images with Application to Crop Thematic Mapping:
A HEVC Comparative Study, Remote Sens 12 (2020) 1590. doi: 10.3390 /rs12101590.
[12] I. Irum, M. Sharif, M. Raza, S. Mohsin, A Nonlinear Hybrid Filter for Salt & Pepper Noise
Removal from Color Images, Journal of Applied Research and Technology 13 (2015) 79-85. doi:
10.1016/S1665-6423(15)30015-8.
[13] M. S. Al-Ani, Khattab, M. A. Alheeti, Precision Statistical Analysis of Images Based on Brightness
Distribution, Advances in Science, Technology and Engineering Systems Journal 2 (2017), 99-
104. doi: 10.25046/aj020413.
[14] F. Li, V. Lukin, K. Okarma, Y. Fu, J. Duan, Intelligent lossy compression method of providing a
desired visual quality for images of different complexity, in: Proceedings of 2021 International
Conference on Applied Mathematics, Modeling and Computer Simulation, AMMCS 2021,
Wuhan, China, 2021 500-505. doi: 10.3233/ATDE220050.
[15] Image and video denoising by sparse 3D transform-domain collaborative filtering. Block-matching
and 3D filtering (BM3D) algorithm and its extensions, 2020. URL:
https://webpages.tuni.fi/foi/GCF-BM3D/.
[16] V. Makarichev, I. Vasilyeva, V. Lukin, B. Vozel, A. Shelestov, N. Kussul, Discrete Atomic
Transform-Based Lossy Compression of Three-Channel Remote Sensing Images with Quality
Control, Remote Sens. 14 (2022) 125. doi: 10.3390/rs14010125.
[17] S. Jiao, Y. Gao, J. Feng, T. Lei, X. Yuan, Does deep learning always outperform simple linear
regression in optical imaging? 2020. URL: https://arxiv.org/ftp/arxiv/
papers/1911/1911.00353.pdf.
[18] Z. Chen, S. Gu, G. Lu, D. Xu, Exploiting Intra-slice and Inter-slice Redundancy for Learning-
based Lossless Volumetric Image Compression, IEEE Transactions on Image Processing. 31
(2022). doi: 10.1109/TIP.2022.3140608.
[19] K. Sayood, Introduction to data compression, 4th ed.; Morgan Kaufman, 2017.
[20] M. Behrisch et al., Quality metrics for information visualization, in Computer Graphics Forum, 37
(2018) 625-662: Wiley Online Library.
[21] V. Abramova, A Blind Method for Additive Noise Variance Evaluation Based on Homogeneous
Region Detection Using the Fourth Central Moment Analysis, Telecommunications and Radio
engineering, 74 (2015) 1651-1669. doi: 10.1615/TelecomRadEng.v74.i18.50.
[22] P. Chatterjee, P. Milanfar, Is Denoising Dead?, IEEE Transactions on Image Processing 19 (2010)
895-911. doi: 10.1109/TIP.2009.2037087.
[23] P Milanfar, A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and
Theoretical, IEEE Signal Processing Magazine 30 (2013) 106-128. doi:
10.1109/MSP.2011.2179329.
[24] Y. Romano, M. Elad, P. Milanfar, The little engine that could: Regularization by denoising, SIAM
Journal on Imaging Sciences, 10 (2017) 1804-1844. doi: 10.1137/16M1102884.
[25] H. Talebi, X. Zhu, P. Milanfar, How to SAIF-ly Boost Denoising Performance, IEEE Transactions
on Image Processing, 22 (2013) 1470-1485. doi: 10.1109/TIP.2012.2231691.
[26] O. Rubel, V. Lukin, A. Rubel, K. Egiazarian, Selection of Lee Filter Window Size Based on
Despeckling Efficiency Prediction for Sentinel SAR Images, Remote Sens. 13 (2021) 1887. doi:
10.3390/rs13101887.
[27] A. N. Zemliachenko, S. K. Abramov, V. V. Lukin, B. Vozel, K. Chehdi, Prediction of optimal
operation point existence and parameters in lossy compression of noisy images, in: Proceedings
SPIE 9244, Image and Signal Processing for Remote Sensing XX, 2014, 92440H. doi:
10.1117/12.2065947.