The article shows the relevance of creating models and methods that provide effective solutions to image processing and analyzing problems in computer vision systems. We consider models of an average level of image representation. They are constructed on the basis of their characteristic features (contours, regions and points of interest). To construct such models, we suggest using the procedure of forming energy features based on the wavelet transform. As a result, the original image will be transformed to a view where different points will have different weights. That characterize their contribution to the overall energy of the image. It is also possible to provide a selection of tuning coefficients. It allows to take into account the interrelations between the wavelet coefficients of various scales. After receiving the weight images, they can be processed to form the required characteristics. For example, to obtain contours, you can perform binarization of a weight image with a certain threshold. To get singular points, you can define a specified number of the most significant weights in different areas of the weight image. For texture analysis, you can use statistical characteristics calculated by the histogram of the scale.
1. Introduction Computer vision software and hardware based on models and methods of image processing and analyzing are actively developing. For example, they are used in the navigation and control systems for pilotless aerial vehicles, systems for remote sensing of the Earth, access control systems for protected systems, industrial control systems, etc.
Generally, in computer vision systems, you need to provide registration, transformation, description, and image analysis. The effectiveness of these problems solutions is associated with the applicable models and methods optimized for the specific conditions of systems functioning.
The description of models and methods of image processing and analyzing is based on the
paradigm of D. Marr. He proposed a three-level model of human perception of real-world objects.
According to this paradigm, we single out low, medium and high levels of representation of images
and their processing. [
At a low level, functional, probabilistic and hierarchical models are used. Functional models describe images in the form of some functions. For example, the description of the image as a function of spatial coordinates. In the case of a binary or halftone image, the values of the function are scalar. In the case of color or multispectral image, they have vector quantities. Probabilistic models describe images in the form of realizations of random processes and use probability density functions and statistical moments (mathematical expectation, variance, etc.). Hierarchical models represent images in the form of image sets of different scales. An example of a hierarchical model is the Gaussian image pyramid.
To represent images at the middle level there used the description of their characteristic features — contours, regions, points of interest. The construction of middle-level models is carried out in two stages. At the first stage, the segmentation of the image is performed. On the second stage - a description in the form of a set of attributes characterizing the selected segments.
At a high level of representation of images, models of explicit and implicit use of knowledge are applied. An example of a model of implicit use of knowledge is a model based on the use of template images. The knowledge on objects in that model is contained in their sample images. The model of explicit use of knowledge is a set of rules for interpreting images.
Note that low-level models are used to build middle-level models, and models of medium and low levels are used to build high-level models. In any case, it is necessary to determine what features of the images are used to describe them. Usually, the signs of color, texture, shape and structure are used. They allow you to describe images in terms of their color content, spatial distribution of colors or brightness, the characteristics of regions, the presence of certain objects and their relative location. 2. Models of image characteristic As already mentioned, mid-level models of image representation are based on their characteristic features – contours, regions and points of interest.
Contour models are based on the selection and analysis of boundaries between areas in the image.
There are many methods for delimiting boundaries. For example, the methods based on morphological
operators and derivative operators of Roberts, Prewitt, Sobel, Laplace and others [
Region models describe the color or texture content of image areas. They are built on the basis of
colour and texture features: a colour histogram [
Point models describe points of interest (point features, singular points, key points). These models
are based on detectors and descriptors of singular points. The first are designed to search for special
points on images, and the latter - to describe them. Currently, many detectors and descriptors are used
in practice, e.g., the detector of Moravets, the Haris detector, SUSAN, SIFT, SURF, FAST, BRIEF,
ORB, GLOH, FREAK, BRISK [
Although there are many models of image characteristics, we can see new methods of their
construction emerging. Some of the approaches to constructing effective models are based on applying
the wavelet transform [
1 Wf (u, s) f (x) s D / 2
x u ψ * dx ,
s
where Wf – the transform result; f – initial function; ψ* – complex conjugation of the shifted and
scaled function ψ that has zero mean value, the center at the zero point, and the unit norm; D – signal
dimension; u – D-dimensional vector of the shift parameters; s – scale parameter [
For digital images, they often use discrete orthogonal multiple-scale wavelet transforms. The transforms are based on the representation of a discrete function f(x) describing the original signal as a sum of approximating fa(x) and detailing fd(x) components:
f (x) f a (x) f d (x) . (2)
The process of transforming the function f(x) can also be represented in the iterative form, which
determines it at different levels of the expansion:
f (x) f aJ (x) ,
(1)
(3)
f aj (x) f aj1 (x) f dj1 (x) , (4)
where J – number of decomposition levels; j = J, …, j0 + 1. The result of the discrete wavelet
transform has the form of a set of approximation a j0 ,l and detail d j,l coefficients [
Wavelet transformations of single-channel images (e.g., halftone images) are usually performed in two stages: first converting for lines, then for columns (or vice versa). The results of the transformation at the j- level are grouped into a matrix of approximating coefficients LL j,m,n m,n0 2 j 1 and matrixes of detailed horizontal LH j,m,n m,n0 2 j 1 , and diagonal HH j,m,n m,n0 2 j 1 , vertical HL j,m,n m,n0 2 j 1 coefficients. For multi-channel images (for example, color images), each channel is transformed individually. 4. Energy features of images The equation for an orthonormal wavelet transform is:
N 1 N 1 2 j0 1 2 j0 1 f k2,l LL2j0 ,m,n k 0 l0 m0 n0
J 1 2 j 12 j 1 J 1 2 j 12 j 1 J 1 2 j 12 j 1 LH 2j,m,n HL2j,m,n HH 2j,m,n ,
j j0 m0 n0 j j0 m0 n0 j j0 m0 n0 where f k,l – brightness of the image point; LL j,m,n , LH j,m,n , HL j,m,n , HH j,m,n – wavelet coefficients.
The sums on the right side of (5) show the contribution of the coefficients of different levels to the total energy of the image at different scales. This contribution makes it possible to get energy estimates for each point of the image. For example, using the orthonormal wavelet transform, when the number of rows and columns of the image is N and is divisible by two, the energy estimates can be obtained as follows: 1. Convert to level j0; 2. Assign value: w j0 1,m,n LL2j0 ,m,n ;
2 3. Calculate energy estimates: w j,m,n 0.25w 2j1,m,n LH 2j,m / 2,n / 2 HL2j,m / 2,n / 2 HH 2j,m / 2,n / 2 .
2 sequentially for j = j0, …, J – 1, where J = log2N, m = 0, 1, …, 2j+1 – 1, n = 0, 1, …, 2j+1 – 1 The calculated values preserve the energy equality:
N 1 N 1 N 1 N 1 f k2,l wk2,l , (8) k0 l0 k 0 l0 where wk2,l wJ21,k,l . Each of the set of values wk2, l kN, l10 or wk,l kN,l10 can serve as the weight of the corresponding pixel, characterizing its contribution to the total energy of the image.
The energy equation (5) is valid only for orthonormal wavelet transforms. For a more general case, the described procedure for calculating energy estimates can be modified as follows: 1. Convert to level j0; 2. Assign value: w j0 1,m,n K j0 1LL2j0 ,m,n ;
2 3. Calculate energy estimates:
w 2j,m,n K j w 2j1,m,n K j LH 2j,m / 2,n / 2 HL2j,m / 2,n / 2 HH 2j,m / 2,n / 2 . sequentially for j = j0, …, J – 1, where J = log2N, m = 0, 1, …, 2j+1 – 1, n = 0, 1, …, 2j+1 – 1
The introduction of tuning coefficients K j and K j makes it possible to provide an optimal calculation of a set of values wk,l k,l0 in accordance with the problem being solved. In this case, N 1 (5) (6) (7) (9) (10) these values can be considered as weights of points. Thus, you can build an image in which each point will be associated with its weight - weight image. 5. The application of the energy features model for image analysis Basing on the energy features model, we can construct a description of the images for detecting and analyzing their characteristic features.
As a result of the described procedure, the original image is transformed so that different points have different weights. As already mentioned, the weights characterize the points’ contribution to the overall energy of the image. In this case, you can select the tuning factors so that the weight points at the boundary points will be larger than for the inners. It is possible, because while transferring from one area of the image to another the boundary points are in the places of brightness difference. To estimate the magnitude of the difference, we use the expression: the magnitude LH 2 HL2 HH 2 . Besides, the wavelet transform allows us to evaluate the significance of points at various scales. Since the image points of different scales are interrelated, we can get integral characteristics that take into account the significance of points on all considered scales. This reasoning can be used as a basis for the procedure of detecting and analyzing contours in an image. If we consider the distribution of the image point’s weights, we can get a description of the texture that characterizes its different regions. Except contours and regions, the analysis of singular points is often used. In this case, by special points we mean the points with the largest weights in the neighborhoods of the given dimensions.
To illustrate the approach to detecting and analyzing the characteristic features of the image, we took a photo of the Kazan Kremlin territory in Figure 1 and its weight models in Figure 2 and Figure 3, obtained using different sets of tuning coefficients. The values of the tuning coefficients for the weight models in Figure 2 are shown in Table 1, and the values of the tuning coefficients for the weight models in Figure 3 are given in Table 2. As the source image and weight images have dimensions of 256 × 256 pixels, the tuning coefficients in Table 1 and Table 2 are shown for eight levels of decomposition from j0 = 0 to J – 1 =7. Here the value of the coefficient K 1 is assumed to be 1. Table 1 demonstrates that moving from level to level, the influence of the previous levels is significantly reduced, and the influence of the detailing coefficients of this level increases. The tuning coefficients given in Table 2 make it possible to reduce the influence of the previous levels weights on the current level. At the same time, they let increase significantly the influence of the detailed wavelet coefficients of this level for increasing the importance of the boundary points.
Figure 2 (а) Figure 2 (b) Figure 2 (c) Figure 2 (d) Figure 3 (а) Figure 3 (b) Figure 3 (c) Figure 3 (d) (c) (d)
Figure 2. Weight images without approximating wavelet coefficients.
After receiving the weight images, we can process them to form characteristics describing contours, texture or singular points in the original images. For example, to get contours, you can binarize a weight image with a specified or calculated threshold value. Figure 4 shows the results of binarization of images in Figure 2 using the same threshold. To get singular points, you can determine the specified number of the most significant weights in different areas of the weight image.
(c) (d) Figure 3. Weight images with approximating wavelet coefficients.
In our case, the singular points will also coincide with certain points of the binary image. For
texture analysis, you can use such statistical characteristics as mathematical expectation, variance, and
asymmetry coefficient, coefficient of excess, smoothness, homogeneity and entropy. They can be
calculated from the weight values histogram. Note that compared to the formation in the processing of
original images, a higher noise resistance is provided in the formation of texture characteristics based
on the energy characteristics model. [