=Paper=
{{Paper
|id=Vol-2210/paper13
|storemode=property
|title=The analysis of image characteristics on the base of energy features of the wavelet transform
|pdfUrl=https://ceur-ws.org/Vol-2210/paper13.pdf
|volume=Vol-2210
|authors=Stella Lyasheva,Mikhail Medvedev,Mikhail Shleymovich,Vladimir Mokshin
}}
==The analysis of image characteristics on the base of energy features of the wavelet transform==
https://ceur-ws.org/Vol-2210/paper13.pdf
The analysis of image characteristics on the base of energy
features of the wavelet transform
S A Lyasheva1, M V Medvedev1, M P Shleymovich1 and V V Mokshin1
1
Kazan National Research Technical University -KAI, K. Marks 10, Kazan, Russia, 420111
Abstract. The article shows the relevance of creating models and methods that provide
effective solutions to image processing and analyzing problems in computer vision systems.
We consider models of an average level of image representation. They are constructed on the
basis of their characteristic features (contours, regions and points of interest). To construct such
models, we suggest using the procedure of forming energy features based on the wavelet
transform. As a result, the original image will be transformed to a view where different points
will have different weights. That characterize their contribution to the overall energy of the
image. It is also possible to provide a selection of tuning coefficients. It allows to take into
account the interrelations between the wavelet coefficients of various scales. After receiving
the weight images, they can be processed to form the required characteristics. For example, to
obtain contours, you can perform binarization of a weight image with a certain threshold. To
get singular points, you can define a specified number of the most significant weights in
different areas of the weight image. For texture analysis, you can use statistical characteristics
calculated by the histogram of the scale.
1. Introduction
Computer vision software and hardware based on models and methods of image processing and
analyzing are actively developing. For example, they are used in the navigation and control systems
for pilotless aerial vehicles, systems for remote sensing of the Earth, access control systems for
protected systems, industrial control systems, etc.
Generally, in computer vision systems, you need to provide registration, transformation,
description, and image analysis. The effectiveness of these problems solutions is associated with the
applicable models and methods optimized for the specific conditions of systems functioning.
The description of models and methods of image processing and analyzing is based on the
paradigm of D. Marr. He proposed a three-level model of human perception of real-world objects.
According to this paradigm, we single out low, medium and high levels of representation of images
and their processing. [1].
At a low level, functional, probabilistic and hierarchical models are used. Functional models
describe images in the form of some functions. For example, the description of the image as a function
of spatial coordinates. In the case of a binary or halftone image, the values of the function are scalar.
In the case of color or multispectral image, they have vector quantities. Probabilistic models describe
images in the form of realizations of random processes and use probability density functions and
statistical moments (mathematical expectation, variance, etc.). Hierarchical models represent images
in the form of image sets of different scales. An example of a hierarchical model is the Gaussian
image pyramid.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018)
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
To represent images at the middle level there used the description of their characteristic features —
contours, regions, points of interest. The construction of middle-level models is carried out in two
stages. At the first stage, the segmentation of the image is performed. On the second stage - a
description in the form of a set of attributes characterizing the selected segments.
At a high level of representation of images, models of explicit and implicit use of knowledge are
applied. An example of a model of implicit use of knowledge is a model based on the use of template
images. The knowledge on objects in that model is contained in their sample images. The model of
explicit use of knowledge is a set of rules for interpreting images.
Note that low-level models are used to build middle-level models, and models of medium and low
levels are used to build high-level models. In any case, it is necessary to determine what features of the
images are used to describe them. Usually, the signs of color, texture, shape and structure are used.
They allow you to describe images in terms of their color content, spatial distribution of colors or
brightness, the characteristics of regions, the presence of certain objects and their relative location.
2. Models of image characteristic
As already mentioned, mid-level models of image representation are based on their characteristic
features – contours, regions and points of interest.
Contour models are based on the selection and analysis of boundaries between areas in the image.
There are many methods for delimiting boundaries. For example, the methods based on morphological
operators and derivative operators of Roberts, Prewitt, Sobel, Laplace and others [2,3].
Region models describe the color or texture content of image areas. They are built on the basis of
colour and texture features: a colour histogram [4], a colour connection vector [5], a correlogram of
colours [6], colour moments [7], a dominant colour descriptor [8], statistical textural features [2], local
binary patterns [9], spectral features [10], Tamura’s features [1] and others.
Point models describe points of interest (point features, singular points, key points). These models
are based on detectors and descriptors of singular points. The first are designed to search for special
points on images, and the latter - to describe them. Currently, many detectors and descriptors are used
in practice, e.g., the detector of Moravets, the Haris detector, SUSAN, SIFT, SURF, FAST, BRIEF,
ORB, GLOH, FREAK, BRISK [12, 13].
Although there are many models of image characteristics, we can see new methods of their
construction emerging. Some of the approaches to constructing effective models are based on applying
the wavelet transform [14,15].
3. The wavelet transform of images
In general terms the wavelet transform is represented as follows:
xu
1
Wf (u, s)
f ( x) s
ψ* dx ,
s
D/2
(1)
where Wf – the transform result; f – initial function; ψ* – complex conjugation of the shifted and
scaled function ψ that has zero mean value, the center at the zero point, and the unit norm; D – signal
dimension; u – D-dimensional vector of the shift parameters; s – scale parameter [16].
For digital images, they often use discrete orthogonal multiple-scale wavelet transforms. The
transforms are based on the representation of a discrete function f(x) describing the original signal as a
sum of approximating fa(x) and detailing fd(x) components:
f (x) f a (x) f d (x) . (2)
The process of transforming the function f(x) can also be represented in the iterative form, which
determines it at different levels of the expansion:
f (x) f aJ (x) , (3)
f aj (x) f aj 1 (x) f dj 1 (x) , (4)
where J – number of decomposition levels; j = J, …, j0 + 1. The result of the discrete wavelet
transform has the form of a set of approximation a j0 ,l and detail d j ,l coefficients [17].
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 97
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
Wavelet transformations of single-channel images (e.g., halftone images) are usually performed in
two stages: first converting for lines, then for columns (or vice versa). The results of the
transformation at the j- level are grouped into a matrix of approximating coefficients LL j ,m,n m,n 0 2 j 1
2 j 1
and matrixes of detailed horizontal LH j ,m,n m,n0 , vertical HL j ,m,n m,n 0 , and diagonal HH j ,m,n m,n 0
2 j 1
2 j 1
coefficients. For multi-channel images (for example, color images), each channel is transformed
individually.
4. Energy features of images
The equation for an orthonormal wavelet transform is:
N 1 N 1 2 j0 1 2 j0 1
k 0 l 0
f k2,l LL
m 0 n 0
j0 , m , n
2
(5)
J 1 2 j 1 2 j 1 J 1 2 j 1 2 j 1 J 1 2 j 1 2 j 1
j j0 m 0 n 0
LH 2j ,m,n
j j0 m 0 n 0
HL2j ,m,n
j j0 m 0 n 0
HH 2j ,m,n ,
where f k ,l – brightness of the image point; LL j ,m,n , LH j ,m,n , HL j ,m,n , HH j ,m,n – wavelet
coefficients.
The sums on the right side of (5) show the contribution of the coefficients of different levels to the
total energy of the image at different scales. This contribution makes it possible to get energy
estimates for each point of the image. For example, using the orthonormal wavelet transform, when
the number of rows and columns of the image is N and is divisible by two, the energy estimates can be
obtained as follows:
1. Convert to level j0;
2. Assign value:
w 2j0 1,m,n LL2j0 ,m,n ; (6)
3. Calculate energy estimates:
w 2j ,m,n 0.25w 2j 1,m,n LH 2j ,m / 2,n / 2 HL2j ,m / 2,n / 2 HH 2j ,m / 2,n / 2 . (7)
j+1 j+1
sequentially for j = j0, …, J – 1, where J = log2N, m = 0, 1, …, 2 – 1, n = 0, 1, …, 2 –1
The calculated values preserve the energy equality:
N 1 N 1 N 1 N 1
w ,
k 0 l 0
f k2,l
k 0 l 0
2
k ,l (8)
. Each of the set of values w or w
N 1 N 1
where wk2,l wJ2 1,k ,l 2
k, l k, l 0 k ,l k ,l 0 can serve as the weight of the
corresponding pixel, characterizing its contribution to the total energy of the image.
The energy equation (5) is valid only for orthonormal wavelet transforms. For a more general case,
the described procedure for calculating energy estimates can be modified as follows:
1. Convert to level j0;
2. Assign value:
w 2j0 1,m,n K j0 1 LL2j0 ,m,n ; (9)
3. Calculate energy estimates:
w 2j ,m,n K j w 2j 1,m,n K j LH 2j ,m / 2,n / 2 HL2j ,m / 2,n / 2 HH 2j ,m / 2,n / 2 . (10)
j+1 j+1
sequentially for j = j0, …, J – 1, where J = log2N, m = 0, 1, …, 2 – 1, n = 0, 1, …, 2 –1
The introduction of tuning coefficients K j and K j makes it possible to provide an optimal
N 1
calculation of a set of values wk ,l k ,l 0 in accordance with the problem being solved. In this case,
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 98
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
these values can be considered as weights of points. Thus, you can build an image in which each point
will be associated with its weight - weight image.
5. The application of the energy features model for image analysis
Basing on the energy features model, we can construct a description of the images for detecting and
analyzing their characteristic features.
As a result of the described procedure, the original image is transformed so that different points
have different weights. As already mentioned, the weights characterize the points’ contribution to the
overall energy of the image. In this case, you can select the tuning factors so that the weight points at
the boundary points will be larger than for the inners. It is possible, because while transferring from
one area of the image to another the boundary points are in the places of brightness difference. To
estimate the magnitude of the difference, we use the expression:
f [(LH 2 HL2 HH 2 ) / 3]1 / 2 , (11)
where Δf – amount of difference in the point of the image; LH, HL, HH – detailing wavelet
coefficients at the point of the image. Thus, the energy of the difference in the point is proportional to
the magnitude LH 2 HL2 HH 2 . Besides, the wavelet transform allows us to evaluate the
significance of points at various scales. Since the image points of different scales are interrelated, we
can get integral characteristics that take into account the significance of points on all considered
scales. This reasoning can be used as a basis for the procedure of detecting and analyzing contours in
an image. If we consider the distribution of the image point’s weights, we can get a description of the
texture that characterizes its different regions. Except contours and regions, the analysis of singular
points is often used. In this case, by special points we mean the points with the largest weights in the
neighborhoods of the given dimensions.
To illustrate the approach to detecting and analyzing the characteristic features of the image, we
took a photo of the Kazan Kremlin territory in Figure 1 and its weight models in Figure 2 and Figure
3, obtained using different sets of tuning coefficients. The values of the tuning coefficients for the
weight models in Figure 2 are shown in Table 1, and the values of the tuning coefficients for the
weight models in Figure 3 are given in Table 2. As the source image and weight images have
dimensions of 256 × 256 pixels, the tuning coefficients in Table 1 and Table 2 are shown for eight
levels of decomposition from j0 = 0 to J – 1 =7. Here the value of the coefficient K 1 is assumed to be
1. Table 1 demonstrates that moving from level to level, the influence of the previous levels is
significantly reduced, and the influence of the detailing coefficients of this level increases. The tuning
coefficients given in Table 2 make it possible to reduce the influence of the previous levels weights on
the current level. At the same time, they let increase significantly the influence of the detailed wavelet
coefficients of this level for increasing the importance of the boundary points.
Table 1. The tuning coefficients for the weight models in the figure 2.
Level 0 Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 Level 7
K 0 K 0 K1 K1 K 2 K 2 K 3 K 3 K 4 K 4 K 5 K 5 K 6 K 6 K 7 K 7
Figure 2 (а) 1 1 0.67 1 0.44 1 0.29 1 0.19 1 0.13 1 0.09 1 0.06 1
Figure 2 (b) 1 2 0.67 2 0.44 2 0.29 2 0.19 2 0.13 2 0.09 2 0.06 2
Figure 2 (c) 1 3 0.67 3 0.44 3 0.29 3 0.19 3 0.13 3 0.09 3 0.06 3
Figure 2 (d) 1 4 0.67 4 0.44 4 0.29 4 0.19 4 0.13 4 0.09 4 0.06 4
Table 2. The tuning coefficients for the weight models in the figure 3.
Level 0 Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 Level 7
K 0 K 0 K1 K1 K 2 K 2 K 3 K 3 K 4 K 4 K 5 K 5 K 6 K 6 K 7 K 7
Figure 3 (а) 1 1 0.91 2.00 0.83 4.00 0.75 8.00 0.68 16.0 0.62 32.0 0.56 64.0 0,51 128
Figure 3 (b) 1 1 0.91 1.67 0.83 2.78 0.75 4.63 0.68 7.72 0.62 12.9 0.56 21.4 0.51 35.7
Figure 3 (c) 1 1 0.91 1.43 0.83 2.04 0.75 2.92 0.68 4.16 0.62 5.95 0.56 8.49 0.51 12.1
Figure 3 (d) 1 1 0.91 1.25 0.83 1.56 0.75 1.95 0.68 2.44 0.62 3.05 0.56 3.81 0.51 4.77
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 99
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
Figure 1. Original image.
(а) (b)
(c) (d)
Figure 2. Weight images without approximating wavelet coefficients.
After receiving the weight images, we can process them to form characteristics describing
contours, texture or singular points in the original images. For example, to get contours, you can
binarize a weight image with a specified or calculated threshold value. Figure 4 shows the results of
binarization of images in Figure 2 using the same threshold. To get singular points, you can determine
the specified number of the most significant weights in different areas of the weight image.
(а) (b)
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 100
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
(c) (d)
Figure 3. Weight images with approximating wavelet coefficients.
(а) (b)
(c) (d)
Figure 4. Binary images.
In our case, the singular points will also coincide with certain points of the binary image. For
texture analysis, you can use such statistical characteristics as mathematical expectation, variance, and
asymmetry coefficient, coefficient of excess, smoothness, homogeneity and entropy. They can be
calculated from the weight values histogram. Note that compared to the formation in the processing of
original images, a higher noise resistance is provided in the formation of texture characteristics based
on the energy characteristics model. [18, 19].
6. Conclusion
The described approach to detecting and analysing the characteristic features of images can serve as a
basis for constructing systems of objects detection and recognition in various systems based on
methods and means of computer vision, including onboard systems of pilotless aerial vehicles – to
detect and recognise the objects on images, process control systems, intelligent transport systems, etc.
[20–22].
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 101
Image Processing and Earth Remote Sensing
S A Lyasheva, M V Medvedev, M P Shleymovich and V V Mokshin
7. References
[1] Potapov A S 2007 Pattern Recognition and Machine Perception: A General Approach Based on
the Principle of Minimum Description Length (St. Petersburg: Politekhnika) p 548
[2] Gonzalez R C and Woods R E 2007 Digital Image Processing (Prentice Hall) p 976
[3] Belim S V and Kutlunin P E 2015 Boundary extraction in images using a clustering algorithm
Computer Optics 39(1) 119-124 DOI: 10.18287/0134-2452-2015-39-1-119-124
[4] Long F, Zhang H and Feng D 2003 Fundamentals of content-based image retrieval Multimedia
Information Retrieval and Management – Technological Fundamentals and Applications 1-26
[5] Pass G and Zabih R 1996 Histogram refinement for content-based image retrieval Proceedings
3rd IEEE Workshop on Applications of Computer Vision 96-102
[6] Huang J, Kumar S R, Mitra M, Zhu W-J and Zabih R 1999 Spatial Color Indexing and
Applications International Journal of Computer Vision 35(3) 245-268
[7] Stricker M and Orengo M 1995 Similarity of Color Images Proceedings of the SPIE Conference
2420 381-392
[8] Deng Y, Manjunath B S, Kenney Ch, Moore M S and Shin H 2001 An efficient color
representation for image retrieval IEEE Transactions on image processing 10(1) 140-147
[9] Pietikinen M, Hadid A, Zhao G and Ahonen T 2011 Computer Vision Using Local Binary
Patterns (Springer) p 224
[10] Tuceryan M and Jain A K 1998 Texture analysis The Handbook of Pattern Recognition and
Computer Vision 207-248
[11] Tamura H, Mori S and Yamawaki T 1978 Texture features corresponding to visual perception
IEEE Transactions on Systems, Man, and Cybernetics SMC-8(6) 460-473
[12] Awad A I and Hassaballah M 2016 Image Feature Detectors and Descriptors (Springer) p 438
[13] Fan B, Wang Z and Wu F 2015 Local Image Descriptor: Modern Approaches (Springer) p 108
[14] Tang Y Y 2009 Wavelet Theory and Its Application to Pattern Recognition (World Scientific
Publishing Company) p 482
[15] Spitsyn V G, Bolotova Y A, Phan N H and Bui T T 2016 Using a Haar wavelet transform,
principal component analysis and neural networks for ocr in the presence of impulse noise
Computer Optics 40(2) 249-257 DOI: 10.18287/2412-6179-2016-40-2-249-257
[16] Addison P S 2002 The Illustrated Wavelet Transform Handbook: Introductory Theory and
Applications in Science, Engineering, Medicine and Finance (Institute of Physics Publishing) p
368
[17] Mallat S 2009 A Wavelet Tour of Signal Processing: The sparse way (Academic Press) p 832
[18] Shleymovich M P and Lyasheva S A 2016 The energy characteristics of images based on the
wavelet transform The II International Conference on Information Technology and
Nanotechnology ITNT-2016 334-341
[19] Lyasheva S A and Shleymovich M P 2016 Image analysis based on the wavelet transform
energy features New Technologies, Materials And Equipment of the Russian Aerospace Industry
147-153
[20] Lyasheva S A, Medvedev M V and Shleimovich M P 2014 Terrain object recognition in
unmanned aerial vehicle control system Russian Aeronautics 57(3) 303-306
[21] Shleymovich M P, Medvedev M V and Lyasheva S A 2017 Image analysis in unmanned aerial
vehicle on-board system for objects detection and recognition with the help of energy
characteristics based on wavelet transform Proceedings of SPIE 10342
[22] Shleymovich M P, Medvedev M V and Lyasheva S A 2016 Object detection in the images in
industrial process control systems based on salient points of wavelet transform analysis
Proceedings 2nd International Conference on Industrial Engineering, Applications and
Manufacturing 7911633
Acknowledgments
The research was carried out within the framework of the state task No 2.1724.2017/4.6.
IV International Conference on "Information Technology and Nanotechnology" (ITNT-2018) 102