Texture classification using oriented complex networks considers the functional connections between topological elements and simulates the complex textures more accurately. In contrast to the classical spatial texture analysis, we ofer a novel function of weights in complex networks and a classification method that takes into account the scaling and color of textures. For this, three complex networks represented R, G and B components are built, which provide invariance of color aerial photographs obtained at diferent times. Comparison of the classification results using the proposed multiscale complex networks and conventional texture analysis based on a statistical approach is given. Also we extended this approach on color aerial photographs using multilayer structure of complex network.
Texture analysis is a fundamental problem in computer vision tasks. Due to the fact that this
is a constant research field since the 1960s, many diferent approaches have been developed
that does not prevent from looking for new views on an old problem [
Extraction of texture features is usually performed at the initial stages of visual data processing that makes this procedure important from the point of view of future results. Conventional methods of texture analysis are grouped into four categories, such as statistical, spectral, structural and model-based methods. Recently, alternative approaches based on learning techniques have been developed. These approaches include bag-of-visual-words, cellular automata and complex networks. The use of complex networks (CNs) is one of the interesting and rapidly developing areas, when a texture is interpreted as a graph with certain topological properties and binary connections. This interpretation makes it possible to simulate the physical properties of natural textures.
The structure of this paper is as follows. In Section 2, the evolution of texture analysis methods is briefly reviewed. Section 3 describes the main propositions of CNs. The proposed method based on multiscale CNs is presented in Section 4. Further, the dataset and experimental results are reported in Section 5. Section 6 concludes the paper.
Investigations in texture analysis began in the 1960s and continue to this day, because texture, as
an inherent property of objects in the real world, is a determining factor in both the classification
and modeling. Statistical texture analysis is one of the earliest methods based on estimating
ordinal statistical moments, calculating gray-level co-occurrence matrices, and applying local
binary patterns (LBPs). It should be noted that the use of LBPs proposed in the 1990s is a fast
way to classify textures with fairly good results. This family of methods has led to numerous
modifications, some of which have became dificult to implement [
In addition to conventional approaches, alternative innovative methods are being developed based on the analysis of feature points, discriminant local features, cellular automata, deep neural networks and complex networks. Often these approaches use feature dictionaries in the form of bag-of-visual-words (BOVW). The main advantage of the methods based on the theory of complex networks is related to their ability to map relationships between structural elements of a texture. However, the reduction of the dimensionality of features and the search for new ways to build complex networks remain an area of interest.
The deterministic tourist walk method, which belongs to the agent-based category, can be
considered as a predecessor of CNs in texture analysis [
A combination of CNs and LBPs was introduced in [
In [
Ribas et al. [
CNs find their application not only in texture analysis, but also in image segmentation, when
a cluster of nodes is related with image segmentation. The fundamental limitation of image
segmentation based on CNs due to the excessive numbers of nodes in the network is overcome
by the concept of super-pixels [
From a brief literature review, we see that CNs are used in combination with other interesting techniques aimed at achieving invariance properties. The paper proposes a texture classification method for aerial photographs based on the use of complex networks calculated for various scales and refined the final classification.
Currently, the theory of complex networks is used in many fields, such as physics, sociology,
biology, mathematics, computer science, medicine, ecology, linguistics, and others. Complex
networks can be defined as graph structures with statistical mechanisms that give them an
interdisciplinary nature. Complex networks demonstrate the flexibility and versatility of
representing natural structures, including dynamic topology changes. In linguistics, CNs are suitable
for automatic summarization due to strong correlation between the metrics of such networks
and important text features. In [
Let us define a texture as a two-dimensional pixel structure. In grayscale images, each pixel has an intensity value, which is an integer in the range = 0, . . . , , = 255. Let (, ) = , = 1, . . . , and = 1, . . . , be the intensity of the pixel with coordinates (, ) in image . Let us build a graph = (, ), in which each pixel (, ) is a node edge ∈ , = (,, ′,′ ), where Euclidian distance is no more that radius : , ∈ and connections between two pixels (, ) and (′, ′) are defined by an undirected = {︀ = (︀ ,, ′,′ ︀) ∈ | ︀( ,, ′,′ ≤ ∧ (, ) < (′, ′)}︀ , ︀) where (,, ′,′ ) is an Euclidian distance between two pixels, defined by following ex(,, ′,′ ) = √︀( − ′)2 + ( − ′)2.
Each edge has a weight () computed by (2):
As one can see, the multiplier pression: of (3).
() = 2(,, ′,′ ) · 2 |(, ) − (′, ′)| ∀ = (,, ′,′ ) ∈ .
Weights are often normalized to the range [
(), for example [
if = 1, 22 = · 2(,, ′,′ ) −
22 · is a constant for diferent values . Thus, we can pre-calculate these values and store them in a special matrix. According to (4), value increases with a larger value . We can interpret (4) as an inverse filter with a minimum value at = 1.
Note that the network constructed according to (1)–(3) has the same number of connections,
in other words such network is a regular graph. A regular graph is not a complex network and
requires further transformations. The simplest transformation is to use the threshold value
for the original set of edges in order to form a subset ⊆ , where each edge ∈ has a
weight () equal to or less than the value . As a result of this transformation, a new network
= (, ) is formed, which is an intermediate stage in the evolution of the network:
* = { ∈ |() ≤ }
selection of these parameters. By applying a set of threshold values , ∈ , to the original
(1)
(2)
(3)
(4)
(5)
network , we can study the behavior of texture properties based on calculations of statistical
features. The degree (connectivity) of a node determines the number of edges connected to this
node. Knowing the degree, we can build a histogram and calculate the mean, contrast, energy,
and entropy. These features have a significant role in the description of the texture because
they describe local properties (at the level of neighboring nodes) and global properties (at the
level of histogram). Moreover, it is possible to simulate the dynamic behavior of a complex
network by transforming the original network and then calculating its specific properties [
Typically, texture signatures are evaluated using Linear Discriminant Analysis (LDA). The
LDA method is a supervised learning method and, therefore, requires a preliminary
determination of the number of classes. However, other classifiers are also used, for example, based on
distance metrics or histogram analysis by transforming a complex network into local spatial
patterns [
Various modifications of complex networks are known, for example, CNs for the categorization of textures, dynamic texture recognition, face recognition, texture classification using BOVW, etc.
The developed method of texture classification takes into account diferent image scales. Scale invariance is achieved by building a pyramid of images in such a way that the texture is analyzed by sliding windows of sizes 31 × 31, 15 × 15 and 7 × 7 pixels. Also we chose the values of parameters r and t during the CN learning. Aerial photographs are high resolution images, thus the number of extracted texture samples will be suficient for supervised learning. During testing, parameters such as mean , contrast , energy and entropy are calculated for each analyzed patch using (6)–(9), where , is the intensity value, (, ) is the number of pixels with values , , is the size of the sliding window in one direction, is the number of intensity levels, = 255.
=
1 × =1 =1 ∑︁ ∑︁ , , − 1 − 1 = ∑︁ ∑︁ (, )2, , =0 =0 74–83 (8) (9)
One of the interesting but dificult to calculate texture parameters is the fractal dimensionality. The main idea is to verify that CNs consist of self-repeating patterns at all scales, using fractal methodologies. This is a special issue for further investigation. Note that fractal dimensionality is a distinctive feature of natural and man-made objects.
Analysis of color aerial photographs leads to triple increase in the number of features in RGB
color space. In this case, we build multilayer CNs close to the approach presented in [
Aerial photographs, as a rule, contain a limited number of textures. Therefore, the
classification stage can be simplified using the Minkowski distance [
Since we could not find a publicly available aerial texture dataset, we used 12 video sequences
obtained from a dataset “Drone Videos DJI Mavic Pro Footage in Switzerland” [
We created our own dataset as a set of texture patches belonging to the classes “Forest”, “Meadow”, “Field”, “Road”, “Mountains”, “River”, “Sand”, and “Sky”. Each class includes 40– 60 patches, thus our dataset is balanced. The total number of patches is 420 patches of 192 × 192 pixels, which were divided into training set and test set as 70% and 30%, respectively. Examples of patches are depicted in Figure 2.
Texture analysis was performed in “The R Project for Statistical Computing”. The used sliding windows had sizes 31 × 31, 15 × 15 and 7 × 7 pixels. During the experiments, we adapted the parameters of and (see Figure 1) for each scale and each class. Such manual setting is a disadvantage of using CNs. However, this approach is suitable for solving the problem with
Caption First frame Resolution oNf ufrmambeers
limited classes. Since textures belonging to the same class are not homogeneous, the structures of CNs are diferent for diferent scaling. We calculated mean , contrast , energy and entropy for three scales and three color components (R, G and B) for each class of texture using CNs based on the training set. For classification, we used the Minkowski distance.
We compared our results with the conventional texture analysis based on a statistical approach
similar to texture analysis done in [
1 , 74–83 (10) (11) (12)
For texture classification was used artificial neural network of direct propagation with three hidden layers. The comparative results are shown in Table 2, where TR is True Recognition and HTER is Half Total Error Rate, which combines the False Rejection Rate (FRR) and the False Acceptance Rate (FAR).
The results presented in Table 2 show that the CNs approach classifies the homogeneous textures close to statistical approach. This can be explained by the fact that CNs build almost regular graphs on homogeneous textures. Mean TR values are in the ranges 84.91–98.32 and 87.29–99.08 for conventional texture analysis and proposed CNs analysis, respectively (in Table 2, the best true recognition results are highlighted in bold). Images with explicit fractal structures provide the best recognition accuracy. In our case, there are “Mountains” and “Forest”. Also good results were achieved due to the color diferences of the class samples in this limited dataset, despite the available color variations within the same class. The main advantages of our approach are the ability to reflect the physical nature of textures and low computational costs at the training step due to the use of simple metrics for classification. Multiscaling increases the recognition accuracy slightly and decreases the HTER values in a statistical sense, primarily due to the fractal properties of natural textures.
In this research, we study the application of CNs theory for texture classification in aerial photographs. We propose a new function for calculating the weights of the edges and introduce a CN training based on multiscale and multilayer properties. For experiments, own dataset was created from the video sequences “Drone Videos DJI Mavic Pro Footage in Switzerland” and photographs of the “Aerial Textures”. It includes a set of texture patches belonging to the classes “Forest”, “Meadow”, “Field”, “Road”, “Mountains”, “River”, “Sand”, and “Sky”. The conducted experiments show that despite the CNs approach classifies the natural textures close to statistical approach, the proposed CNs reflect the physical nature of textures and have low computational costs at the training step due to the use of simple metric for classification (Minkowski distance). In general, the results of true recognition are 1–3% higher compared to conventional texture analysis.