The paper proposes a spline model of a parametrized digital image based on two momentary texture features, namely, the locally averaged illumination intensity and the root mean square. The model is an estimate of the density function of the distribution of these characteristics and is an alternative to the model of a mixture of normal distributions. The problem of digital image segmentation is not new - there are numerous diverse and effective approaches to solving it, both in purely mathematical methods and through the use of hardwaresoftware approaches. However, the segmentation of oversized realistic images or video data (e.g., from aerial or space observations) in real-time or near-real-time mode gains particular relevance. By its nature, the segmentation (clustering) problem relates to unsupervised pattern recognition, generally solved based on the model of feature distribution density defining certain characteristics of digital images. In general, the problem of pattern recognition can be broken down into the following main stages [1]:
1. 2. 3. 4.
Feature formation is one of the most critical stages in constructing any information processing system. The quality of the chosen feature system and algorithms for their computation depends on the specific requirements of the applied system for processing and analyzing digital images. Therefore, when prioritizing the requirement for segmentation speed, it is necessary to pose a demand for both the high adequacy of the density function model on one hand and the efficiency of obtaining and learning it on the other hand [2, 3].
Hence, the relevant research direction of this work will be to define a model of realistic digital images that would address the segmentation task in a mode close to real-time within modern computerized systems
To further the analysis, let's introduce the concept of texture. Texture refers to fragments of an image that can be characterized by different properties. Texture can be homogeneous when the pattern or structure repeats with the same intensity in all directions and at all points. For example, the surface of sand on a beach or a field can have a homogeneous texture. On the other hand, texture can be heterogeneous when the pattern or structure varies at different points or directions. For instance, an image of a forest with different types of trees, shrubs, and other plants can have a heterogeneous texture. Let's consider methods for forming textural features related to the segmentation and classification of digital images, commonly encountered in tasks of processing aerial photography, aerial, and space monitoring data.
The simplest and often-used segmentation method is based on dividing the image by a brightness threshold [4, 5]. For this, the intensities of each image element are used. If the intensity of a certain selected threshold value increases, the element is classified as an object component; otherwise, it's considered a background element. However, such methods may be ineffective for complex image structures like aerial photographs.
The application of segmentation methods based on textural characteristics holds great potential in various image processing areas. Statistical methods, such as local dispersion analysis [6, 7], histogram-based analysis [8], and using local binary patterns for feature extraction and region merging [9], efficiently highlight textural areas of images by computing statistical characteristics. They can be effective in recognizing different types of textures but may require substantial computational resources for calculating a large number of characteristics.
Structural methods, like the multi-layer adaptive graph cut method [10], excel in segmenting textures with complex structures by identifying primitive structures. However, they may be less effective for irregular textures or textures without known standards.
Spectral methods, like deep learning-based methods and conditional random fields [11, 12], ensure high segmentation accuracy by processing deep features and modeling context using conditional random fields. Additionally, wavelet transform methods also prove effective for segmenting textural images. Wavelet transformation [13, 14] enables the extraction of textural details of various scales in an image using multi-scale analysis.
The Gaussian Mixture Model (GMM) method for constructing digital image models is another prevalent approach to data segmentation [15]. This method involves creating a complex image model using a combination of several Gaussian functions with different parameters. The use of the Expectation-Maximization (EM) algorithm adapts GMM to a dataset by considering the multimodal data distribution as a combination of individual unimodal distributions. GMM assumes that data in a certain cluster is generated by a specific Gaussian distribution. The method allows approximating complex structures and details in an image, facilitating their further analysis. However, constructing a complex model requires significant computational resources, which can complicate processing a large amount of data.
Therefore, the analysis of research and sources shows that a computationally simple approach to texture analysis is possible based on the construction of textural features using the distribution of statistical characteristics, given the simplicity of estimating (modeling) the corresponding probability density function. According to studies [16], two-dimensional splines based on B-splines meet similar conditions, closely resembling interpolations on average. This approach may have advantages in terms of computational efficiency and interpretability. However, its accuracy may be limited compared to some deep learning models capable of solving more complex texture classification tasks [17–21].
Therefore, the goal of this work is to formalize and justify the possibility of applying the mentioned splines to the task of constructing a digital image model based on moment characteristics for automated processing systems of digital images, including aerial observation data.
Let's consider a certain digital image (DI), represented by the illumination intensity of each pixel
= , , = 0, − 1; = 0, − 1 , (
If we consider a realistic DI, such as an aerial image, then any element of intensity (
Let = , , , , … , , ; = 1, be a set of textures that exhibit similar characteristics within a certain neighborhood determined by the size of a sliding window (2 + 1) × (2 + 1), = 1,2, …, where − texture; ,! − characteristic of the -th texture; − number of textures; " − number of characteristics of the texture. In fact, each of the features of a separate texture ,! is defined as some random function of the pixel intensities in the local (2 + 1) × (2 + 1) area around any ( , )-th pixel of DI: ,! = #! $,% , & = − , + , ' = − ,
+ , ( = 1, ", = 0, − 1, = 0, − 1.
Therefore, we're discussing the parameterization of DI (2) based on texture characteristics. In such a case, a DI model can be proposed by estimating the density distribution of parameters in the texture characteristic space. Regarding each separate g-th group of "similar" textures, due to the randomness of characteristics, it's possible to assume the existence of some parameter distribution density function
* ( , , , , … , , ), where the natural assumption is that the texture group distribution present in DI is a mixture . *(Θ) = ,
- * ( , , , , … , , ), / ,
For instance, assuming a normal distribution of densities (3), the task of evaluating model (4) boils
down to reproducing a mixture of normal distributions in the texture characteristic space. Solving
this latter task would allow for image segmentation (clustering), for example, based on dividing the
mixture followed by a naive Bayesian pixel classification (
Further on, let's examine the case when the number of texture characteristics " = 2 is only two. For example, these could be parameters characterizing the average color (illumination) of a group of pixels and their variability. Publications [1, 4, 6] have shown that moment characteristics of DI are invariant to scale and rotation with respect to the size of the local window, so for the parameterization that defines the properties of textures, let's limit the choice to estimating the local mean and standard deviation of pixel intensities in the area:
6 7 6 7
1 (2 + 1) = 3, − 3 − 1, = 3, − 3 − 1.
Therefore, as a result of calculating the characteristics (5) and (6), we obtain a random twodimensional sample (5, 9 ) in the texture feature space ; ,< = (5 !, 9!); ( = 1, = , where = is the number of sliding windows within which mean and standard deviation estimates are calculated. It's worth noting that the order of forming the sample Ω ,< is not critical in this presentation: the calculation can be done for a sliding window of size (2 + 1) × (2 + 1) with a shift of 1 pixel or any other shift of the central element of the area relative to the pixels of DI. Moreover, for the following exposition, the magnitude of the positive integer is also not crucial.
With the sample Ω ,<, we will proceed to construct a two-dimensional variational series divided into classes. For this, we'll define the division ?@A,@B of the plane of realizations of characteristics (5) and (6) according to the following algorithm of actions.
Algorithm for constructing the partition CDE,DF.
Step 1. The central point of the (G, H)-th partition element ?@A,@B is taken as the variant row (5 I, 9I) where 5 I = 5 JIK + (G + 0,5)ℎJ, 9O = 9JIK + (H + 0,5)ℎP,
G = 0, NJ − 1, H = 0, NP − 1,
5 JWX − 5 JIK ; 5 JIK = minT5 !U ; 5 JWX = maxT5 !U ; ℎJ =
! ! NJ 9JIK = minT9!U ; 9JWX = maxT9!U ; ℎP = 9JWX − 9JIK ;
! ! NP where NJ, NP − the number of partition elements (classes) along the respective axes.
Step 2. The frequency [I,O of the variant row is calculated as the number of sample points Ω ,< falling within the limits of the (G, H)-th partition element ?@A,@B, whereby the indices G, H
G = \ 5 ! ℎJ ] ,
9 H = \ ! ] ,
ℎP aI,O = [=I,O , where ^∙` is the integer part.
Step 3. The randomness aI,O of the variants (5 I , 9I) is determined by the relation bA8 bB8 , , a I,O = 1.
I/c O/c
Let us consider the estimation of the feature density function of 5 and 9 as a parameterised image model and evaluate it on the basis of the sample ; ,< = (5 !, 9!); ( = 1, = and the two-dimensional variation series built on its basis
5 I, 9O, aI,O; G = 0, NJ − 1, H = 0, NP − 1 *I̅,O = 1 e
Pf 6c,g@ B
* (5, 9 )'5'9 ℎJℎP Jh8c,g@ A Pf 8c,g@ B is the average value of the probability density function of the features (5, 9 ) in the specified area and the following connection with the randomness of the option takes place
aI,O = *I̅,OℎJℎP = = i j: 5 I − 0,5ℎJ ≤ m(j) < 5 I + 0,5ℎJ, 9O − 0,5ℎP ≤ o(j) < 9O + 0,5ℎP
Thus, up to the constant ℎJ ⋅ ℎP, the randomness is an estimate of the average value of the density function *(5, 9 ). Then, as a model for density approximation, we propose a model in the form of a two-dimensional polynomial spline based on q-splines that is close to the interpolation average. That is, the array of values a̅ = a̅I,O, G ∈ s, H ∈ s will be matched with the following local polynomial spline [18]: t ,c (a, 5 , 9) = , , q ,@A(5 − GℎJ)q ,@B(9 − HℎP)aI,O
I∈u O∈u where (with the notation accuracy up to the split step) [12]
0, 5 ∉ ^− 3ℎ⁄2 ; 3ℎ⁄2`, ⎧ (3 + 25 ⁄ℎ) ⁄8 , 5 ∈ ^− 3ℎ⁄2 ; − ℎ⁄2`, q ,@(5 ) = ⎨3⁄4 − (25 ⁄ℎ) ⁄4 , 5 ∈ ^− ℎ⁄2 ; ℎ⁄2`,
⎩ (3 − 25 ⁄ℎ) ⁄8 , 5 ∈ ^ℎ⁄2 ; 3ℎ⁄2`. or in expanded form
1 t ,c (a, 5 , 9) = 64 (1 − ) (1 − ) aI8 ,O8 + (1 − ) (6 − 2 )aI8 ,O + +(1 − ) (1 + ) aI8 ,O6 + (6 − 2 )(1 − ) aI,O8 + (6 − 2 )(6 − 2 )aI,O + +(6 − 2 )(1 + ) aI,O6 + (1 + ) (1 − ) aI6 ,O8 + (1 + ) (6 − 2 )aI6 ,O + +(1 + ) (1 + ) aI6 ,O6 I,O − an arbitrary error.
In particular, for ℎJ → 0, ℎP → 0 for ∀a(5, 9 ) ∈ , is true ^10`:
ℎJ a(5 , 9) − t ,c (a, 5 , 9)С ≤ 6
aJ(5 , 9)С + ℎ6P aP(5 , 9)С + + ℎJℎP 36 aJ()P(5 , 9) + ‖a(5 , 9)‖ + (ℎ)
С = max G.H ,
G,H
ℎ = maxTℎJ, ℎPU.
= @A (5 − Gℎ J), || ≤ 1 ; = @B (9 − HoP), || ≤ 1.
By means of the explicit analytical representation (8), the models (7) can obtain an asymptotically precise estimation of the approximation error of the density distribution *(5, 9 ) indirectly through the estimation of the function a(5, 9 ), the values of which are the relative frequencies of the respective two-dimensional variational series:
aI,O = a̅I,O + I,O, ā I,O =
e ℎJℎP (Jh8c,g ) @A 1 e
Pf 6c,g @B
a (5, 9 )'5'9 , Pf 8c,g @B where where where
Analyzing the local peculiarities of the constructed model of the two-dimensional normalized density a(5, 9 ), for instance, allows solving the segmentation problem. With this approach, individual segments correspond to clusters determined by local maxima (modes) of the estimate (7).
As an application example of model (7), consider an image representing several textures: a grass lawn, a runway, airplanes (Figure 1). We'll illustrate how these textures look locally and what (7) (8) (9) distribution they have. We'll consider the two-dimensional density functions depending on the mean and standard deviation for each of the textures.
The lawn is a simple texture (Figure 2a). By “simple”, we mean relatively low variability of light intensity within a local window. The estimation of the normalized density function of the “lawn” (Figure 3) based on model (7) is unimodal and has a mean 44 ≤ 5 ≤ 84 and standard deviation 9 ≤ 14.
Similarly, the runway is a simple texture (Figure 2b). The distribution density of the average intensity 134 ≤ 5 ≤ 170 and variability 9 ≤ 12 is also unimodal (Figure 4).
Complex textures are characterized by high variability in light intensity due to the presence within a local window of pixels from various 'simple' textures, for example, the 'airplane' texture (Figure 2c). The normalized density function (Figure 5) is multimodal, indicating the presence of pixels from the “grass lawn”, “runway” and airplane.
Let us consider (Figures 6 and 7) the density function of the entire original digital image parameterised by (5, 9 ) (Figure 1). There are three local groups of pixels defined by the proposed model (7). According to a natural analysis, the first group of pixels belongs to the lawn, the second to the runway, and the third to the airplanes (Figures 6a and 7a). Visually, it's not difficult to identify local maxima and intuitively divide the model into separate segments (Figures 6b and 7b), which determine the corresponding segmentation of textures in the original image. = 4: а) spline model; b) 1 – = 8: а) spline model; b) 1 –
In the study, a spline model (7) for parameterized digital imagery (DI) based on two momentary characteristics of textures was proposed, namely: the locally averaged illumination intensity (5) and the root mean square (6). This model represents an estimation of the probability density function of the mentioned characteristics and serves as an alternative to a mixture model of normal distributions. Thanks to the analytical presentation (8), the spline model demonstrates low computational complexity, substantiated acceptable approximation errors, and can be recommended for implementation in automated DI processing systems operating in near real-time regimes.
Future research involves developing a DI segmentation method based on the introduced spline model and procedures for detecting low-probability textures that provide enhanced informativeness in the analysis of aerial observation data. Additionally, the results of this work are expected to be used in developing neural network models and exploring the hyperparameters of deep learning algorithms.
[2] J. S. Al-Azzeh, M. Al Hadidi, R.S. Odarchenko, S. Gnatyuk, Z. Shevchuk, Z. Hu, Analysis of selfsimilar traffic models in computer networks, International Review on Modelling and Simulations 10(5) (2017) 328–336. doi: 10.15866/iremos.v10i5.12009. [3] O. Popov, A. Iatsyshyn, V. Kovach, V. Artemchuk, D. Taraduda, V. Sobyna, et al., Physical features of pollutants spread in the air during the emergency at NPPs, Nuclear and Radiation Safety 4 (84) (2019) 88–98. doi: 10.32918/NRS.2019.4(84).11. [4] R. D. da Silva, R. Minetto, W. R. Schwartz, H. Pedrini, Satellite Image Segmentation Using Wavelet Transforms Based on Color and Texture Features, ISVC 2 (2008) 113–122. doi: 10.1007/978-3-540-89646-3_12. [5] T. Tran et al., Enabling multicast and broadcast in the 5G core for converged fixed and mobile networks, IEEE Transactions on Broadcasting 66(2) (2020) 428–439. doi: 10.1109/TBC.2020.2991548. [6] P. Prystavka, K. Dukhnovska, O. Kovtun, O. Cholyshkina, A. Zhultynska, Devising information technology for determining the redundant information content of a digital image, EasternEuropean Journal of Enterprise Technologies 6 (2021) 59–70. doi: 10.15587/17294061.2021.248698. [7] A. Chyrkov, P. Prystavka, Method for suspicious object search in video from an aircraft camera based on histogram analysis, Science-based technologies 2(38) (2018) 210–219. doi: 10.18372/2310-5461.38.12827. [8] H. H. Permuter, J. M. Francos, I. Jermyn, A study of Gaussian mixture models of color and texture features for image classification and segmentation, Pattern Recognition 39 (2006) 695– 706. doi: 10.1016/j.patcog.2005.10.028. [9] G. Sun, Y. Hao, X. Chen, J. Ren, A. Zhang, B. Huang, Y. Zhang, X. Jia, Dynamic Post-Earthquake Image Segmentation with an Adaptive Spectral-Spatial Descriptor. Remote Sensing 9 (2017) 899. doi: 10.3390/rs9090899. [10] J. Liu, K. He, Multi-scale foreground extraction on graph cut, MATEC Web of Conferences 277 (2019). doi: 10.1051/matecconf/201927702031. [11] X. Song, L. Wu, G. Liu, Unsupervised color texture segmentation based on multi-scale regionlevel Markov random field models, Computer Optics 43(2) (2019) 264–269. doi: 10.18287/24126179-2019-43-2-264-269. [12] S. Ghaffarian, A Kernel-Based Post-Process for Image Segmentation Using GVF Snake, Journal of Automation and Control Engineering 2(3) (2014) 277–281. doi: 10.12720/joace.2.3.277-281. [13] P. Ganesan, V. Rajini, R. G. Salai, Segmentation and Denoising of Noisy Satellite Images based on Modified Fuzzy C-Means Clustering and Discrete Wavelet Transform for Information Retrieval, Int. J Eng. Technol. 5 (2013). [14] C. Wang, W. Pedrycz, J. Yang, M. Zhou, Z. Li, Wavelet Frame-Based Fuzzy C-Means Clustering for Segmenting Images on Graphs, IEEE Trans. Cybern. 50 (2020) 3938–3949. doi: 10.1109/tcyb.2019.2921779. [15] L. P. Osco, et al., A review on deep learning in UAV remote sensing, International Journal of Applied Earth Observation and Geoinformation 102 (2021) 102456. doi: 10.1016/j.jag.2021.102456. [16] A. Ligun, A. Shumeiko, Asymptotic Methods of Curve Reconstruction, NASU Institute of
Mathematics, 1997. [17] M. Unser, Splines: A Perfect Fit for Signal and Image Processing, IEEE Signal Processing
Magazine 16(6) (1999) 22–38. doi: 10.1109/79.799930. [18] P. Prystavka, Polynomial Splines in Data Processing, Dnipropetrovsk, 2004. [19] L. Shapiro, G. Stockman, Computer Vision, Seattle Washington, 2001. [20] P. Prystavka, M. Ryabiy, A model of realistic images based on two-dimensional splines close to interpolation on average, Science-intensive technologies 3(5) (2012) 67–71. [21] P. Prystavka, A. Rogatyuk, Mathematical foundations of foreign object recognition in the video from unmanned aircraft, Proceedings of the National Aviation University 3(64) (2015) 133–139. doi: 10.18372/2306-1472.64.9048.