With the advent of space equipment that allows obtaining panchromatic images of ultra-high spatial resolution (< 1 m) there was a tendency to develop methods of thematic processing of aerospace images in the direction of joint use of textural and spectral features of the objects under study. In this paper, we consider the problem of classification of forest canopy structures based on textural analysis of multispectral and panchromatic images of Worldview-2. Traditionally, a statistical approach is used to solve this problem, based on the construction of distributions of the common occurrence of gray gradations and the calculation of statistical moments that have significant regression relationships with the structural parameters of stands. An alternative approach to solving the problem of extracting texture features is based on frequency analysis of images. To date, one of the most promising methods of this kind is based on wavelet scattering. In comparison with the traditionally applied approaches based on the Fourier transform, in addition to the characteristic signal frequencies, the wavelet analysis allows us to identify characteristic spatial scales, which is fundamentally important for the textural analysis of spatially inhomogeneous images. This paper uses a more general approach to solving the problem of texture segmentation using the convolutional neural network U-net. This architecture is a sequence of convolution-pooling layers. At the first stage, the sampling of the original image is lowered and the content is captured. At the second stage, the exact localization of the recognized classes is carried out, while the discretization is increased to the original one. The RMSProp optimizer was used to train the network. At the preprocessing stage, the contrast of fragments is increased using the global contrast normalization algorithm. Numerical experiments using expert information have shown that the proposed method allows segmenting the structural classes of the forest canopy with high accuracy.
Monitoring of forest areas, namely textural segmentation and forest mapping is an urgent task. One of the promising ways of global tracking of areas is the use of remote sensing data of the Earth (remote sensing). Due to the growth and diversity of information, there is a need to develop and modernize new methods of its processing. So, in recent years, due to the development of production capacities, one of the fastest growing areas is artificial intelligence. Thus, the idea of this experiment is to use remote sensing data and neural network methods to solve the problem of texture segmentation. In particular, there are works in the literature that are close to our experiment. In [1], the authors performed texture segmentation using the AGMSSeg-Net neural network, interactively selected by the user. Models based on convolutional neural networks, such as the second [2] and third versions of Deeplab [3], were also successfully used to create color labels on maps that allow solving the problem of textural segmentation of forest zones.
After the analysis of related works, we decided to use the Unet model to perform the task of textural segmentation of woodlands using Worldview-2 panchromatic images.
The main contribution to the work is as follows: 1. Preliminary image processing was performed using the global contrast normalization method. 2. The U-net model was trained on the original and pre-processed images. 3. Texture segmentation was performed by the final version of the trained model with the best result of metrics.
4. The errors were compared by the cross-validation method.
The research area is located on the territory of the Moscow region, Bronnitsky forestry in the immediate vicinity of the territory of geographical landings of the forester P.I. Dementieva. The stands of the Bronnitsky forestry have an age of 40 years or more and, according to the variety of species, they cover all the main forest-forming breeds of Russia. The selected site contains natural and forest-cultural plantings with diferent species composition and visible textural diferences of the forest canopy. The plot contains part of the territory of permanent larch forest-seed plantations, which have a pronounced regular structure. The plot also contains natural birch and pine (with an admixture of spruce) stands of various completeness.
Multispectral and panchromatic images of WorldView-2 with a spatial resolution of 1.85 and 0.46 m, respectively, were used as satellite information. The photo was taken on June 28, 2011, before the construction of the Novoryazansky Highway and the Central Ring Road began. For texture processing, a panchromatic image was used, which, after correction, has a spatial resolution of 0.5 m (Fig. 1).
This subsection presents an algorithm for preprocessing a satellite image, which consists of: 1. Converting fragments from the format .tif to .png format for further work with the neural network. In this study, fragments of images with a size of 27 × 27 pixels were prepared for training. 2. Increasing the contrast of fragments using the global contrast normalization algorithm [4]: ′,, =
{︃ √︂ max , + ,, − 1 3
∑︀ ∑︀ ∑︀ (,, − )2 3 =1 =1 =1 }︃ ,
(
In this work, the Xception model [5] was used, due to the fact that with the help of this convolutional network architecture, it is possible to obtain a better result compared to Inception V3 [6], as presented in [7]. The Xception architecture represents a fully connected convolutional network that is able to work with a small number of training examples for segmentation tasks. The generalized U-net architecture [8] is shown in Figure 3.
To calculate the efectiveness of the trained model, we used the mAP and IoU metrics [ 9]. IoU is just a score indicator. Any algorithm that provides predicted bounding rectangles as output can be evaluated using IoU.
1. Reliably marked areas manually by an expert.
2. Certain results of the trained network (
⋓ where is the label of ground truth, and ⋓ are the labels of prediction and truth.
Additionally, 1 − (
, = ∑︁ = =1 1
∑︁ (),
= , (
+
= (
+ where is a true positive result, is a false positive result, and is a false negative result.
The sparse categorical cross entropy () (
∑︁ ( * log( ())) =1 or is the normalized exponent.
where () = ∑︀ =1
The RMSprop optimizer was used to train the network. The number of epochs was 250. The time spent for one epoch is about 5–6 minutes. The training was carried out on the Google Colab platform [11]. The learning process is shown in Figure 4. The quality of using normalized images was compared.
As you can see, increasing the contrast slightly improves the quality of the model, a more detailed comparison of the results is presented below.
At the exit from the network, a mask is formed that corresponds to a certain forest structure. An example of the final processing of a test image is shown in Figure 5.
The main metrics of this work are presented in Table 1. and − take an average value for small images of the test area.
a b c Figure 4: The value of the error function (a) and accuracy value (b and c) at each epoch for the test and training processed data.
After training, the model was tested on test images. The results of the predicted masks were compared with the test masks using several quality metrics. The results are presented in Table 1.
Cross-validation was also carried out for evaluation on independent data. The parameter was equal to 5. The results of the sliding control are presented in Table 2.
As can be seen from Figure 4, c, we have a predominance of the third class, and also sometimes the model considered the 3rd class as the 1st class, perhaps due to the close intersection of these classes, this error occurred, you can also see the erroneous definition of the ForestMixedNormal class as the LarchRegularNormal class. This was due to the similar data structure of the classes.
Based on the results, we can conclude that the U-net model copes with the processing of satellite images of forest areas for segmentation tasks. The main structure of each type of forest is clearly highlighted in the image for a better result, a larger set of image data is still needed, on which several classes will intersect.
In the future, it is planned to process high-resolution images (36 pixels) this will allow using several classes on one image, and it is also planned to use classical architectures of convolutional models of neural networks with a change in architecture to increase eficiency and compare with the U-net model in new areas.
The research was carried out with the financial support of the RFBR (projects No. 19-01-00215 and No. 20-07-00370).