The essential requirement for accurate classification is the high resolution of input images. Among known classification problems, which caused by low-resolution images, are the mixing of training samples and the absence of boundaries between objects of different classes. The mentioned above problems were reduced by imagery spatial resolution enhancement and a hybrid approach to classification, which allows unmixing training samples and improving the quality of images and their classifications.
Classification is the process of dividing a mass of data into classes (groups) according to some
criterion. In the process of computer classification, each pixel of the image is assigned to one of the
selected classes. Classification of satellite images is a widely-used remote sensing tool for solving
such tasks as land-cover mapping and change detection [
Computer classification is used to automatically separate objects displayed on images and obtain a map of the area. For computer processing, each image is presented as a table, each cell of which - an element of the image resolution (pixel) - contains a number indicating the brightness of this element. In multispectral imagery, the values of the brightness of objects are recorded in separate rather narrow areas - spectral bands - of the visible and infrared parts of the spectrum. Then, during computer processing, for each pixel of the image, several digital values of brightness are used at once in different spectral bands.
The main difference between visual decryption and automated processing is that a person (decoder) sees the entire or almost the entire image, whereas a computer in most cases analyzes digital values for only one pixel or a small group of pixels, comparing them with the rest. A person can use different decryption signs to recognize terrain objects: size, shape, length, relative position of objects, etc., but a computer can simultaneously analyze several images in different spectral bands, and, as a rule, much faster than a person can.
When carrying out any classification, some obstacles affect the quality and accuracy of the
classification:
the use of supervised classification causes errors associated with the human factor, uneven
samples, a discrepancy between the selected thematic classes and their spectral characteristics, etc.
- this can be called a radiometric obstacle [
This study presents tools for solving the problems described above: applying a hybrid approach to classification to correct radiometry and the spatial resolution equalization of multispectral satellite images. 2.1.
The reasonable interpretation of land cover classes could only be achieved by supervised classification since it applies expert knowledge, which describes each type of land cover. Supervised classification requires training samples of classes selected by an expert. However, those classes, as a rule, are subjective as well as expert-selected training samples are not accurate. Unlike supervised classification, unsupervised one provides objective classes, obtained by clustering. The hybrid approach to classification [5] is applied to form the interpretable and objective classes. The scheme of the described above approach to classification is shown in Figure 1.
This approach to classification implies subdividing training samples of expert-selected classes into objective clusters by unsupervised classification. After that, formed clusters are used as training samples for supervised classification. Those steps are aimed to reduce the inaccuracy and subjectiveness of the selected samples. In turn, it increases the classification accuracy in comparison with both supervised and unsupervised types of classification.
The first step is the training samples clustering by unsupervised classification. Clustering is necessary to avoid the subjectiveness of expert-selected classes. Also, due to the high heterogeneity of some land cover classes, they overlap each other and it means that their training samples, as a rule, are mixed. This problem could be also solved by the initial training samples clustering. Basic Sequential Algorithmic Scheme (BSAS) [6] is applied as a method of unsupervised classification. The main benefit of this method among others is that the number of clusters may not be known in advance. However, instead of the number of clusters, this scheme requires two other input parameters, which are the threshold of the dissimilarity and the maximum allowed number of clusters. The first one is defined as a distance between each cluster and feature vector, which corresponds to the set of training samples. The second parameter is required not to divide the input data into a greater number of clusters than it’s defined by an expert. This parameter should be defined taking into account the computational costs of those operations and/or limit of clusters, overcoming of which will provide inadequate and not interpretable training samples subdividing. This method applies separately to the training samples of each class. As a result of this step, initial expert-selected training samples would be subdivided into objective subclasses and their heterogeneity would be reduced by transforming them into dense clusters.
The next step is the supervised classification of the study area. Support vector machine [7] is applied as a method of supervised classification for this task. This method is proved its reliability in conditions of high heterogeneity of land cover classes [8]. The required input data for this procedure are an image of the study area and training samples of each class. Subclasses obtained at the previous step are used as training samples. Therefore, each subclass is interpreted as a particular class. After this step, the input image will be classified into that number of classes, which corresponds to the number of subclasses obtained after clustering of initial training samples.
In case if the task is to obtain classification, which consists only of initial classes, then merging of subclasses into initial classes is required. The subclasses’ merging defines the initial class of pixels subclass, and then the value of a pixel is converted to the value of the initial class. This procedure is performed separately for each pixel of classification obtained at the second step. 2.2.
It is very important to carry out the spatial resolution equalization procedure when satellite images or separate image bands of different spatial resolutions are used together.
A widely used method is to simply divide a pixel into subpixels according to the nearest neighbour rule [9], while the resulting subpixels retain the value of the output pixel. Although the use of the nearest neighbour rule preserves the average radiometric value of subpixels in a pixel, it does not increase the information content of the resampled bands.
Another widely used method for calculating subpixel values is the application of the selected type of interpolation based on a certain number of adjacent pixels in the original image. Consider, for example, a bicubic interpolation technique [10] that slightly smoothes the image, giving a sense of more detail. However, this procedure does not keep the radiometry within the original pixel. The average value of the subpixels in pixels is different from the initial value of the input image.
It is proposed to use a method [11] based on image segmentation by spectral signatures, as well as optimization of decision making when redistributing values in subpixels, taking into account both the similarity of the original spectral signatures and the spatial relationships of the topology for each type of land cover.
The Nearest Neighbour Oversampling procedure quadruples a pixel and ultimately generates 4 identical subpixels. To improve the overall physical resolution of the image bands, but not degrade their quality, it is necessary to correctly redistribute the signals in these subpixels, while maintaining their average radiometric value.
This should be done using the scanning window and taking into account the adjacent territory, that is, the nearest eight subpixels (numbered 1 .. 4 and 6 .. 9) around the current one (numbered as 5) as shown in Figure 2.
including 4 subpixels (1, 2, 4, 5) of one low‐resolution pixel
When processing images, the process of determining different types of the earth's surface is necessary. This is done by spectral segmentation of the image by spectral characteristics. Fuzzy logic methods are widely used to solve such problems [12].
For the correct spatial redistribution of subpixels, it is necessary to analyze the topological properties of the main classes and obtain appropriate topological descriptions. Multiple valued logic (MVL) methods are used to solve the problem of changing the subpixel class according to the relationships and classes of the nearest surrounding subpixels.
MVL is a type of logic in which the level of truth can be m-valued or infinite, not just binary, as in Boolean logic. MVL requires a mathematical approach to express the relationships between input logic values and the result of certain phenomena. A logical function with a value having the following form was used: f m : n , (1) where n is the number of multivalued variables, and the set M = {0, 1, ..., m – 1} is the set of certain truth levels.
MVL is used to reclassify surface types depending on their spatial distribution, which is used in our analysis of remote sensing data because this approach will allow a fairly efficient reclassification.
The pixel redistribution procedure takes into account the following topological properties of the analyzed segment: compactness - when subpixels of one class are localized by compact inseparable groups; orientation (linearity) - subpixels of one class are arranged linearly: horizontally, vertically, diagonally if you can determine the orientation of these lines; texture (homogeneity) - subpixels of one or more classes are arranged in a checkerboard pattern or close to it, as well as when several different classes fall within the window and compact or linear structures are not defined.
Five significant types of the relative location of subpixels of one class are defined in Figure 3. These five types represent specific shapes of subpixels, such as all columns and rows - T5, diagonals (T2), and right triangles with a certain location (T1 - the right angle at the edges, T3 - the right angle in the middle of the outer columns and rows in the matrix, T4 - the right angle in the middle of the matrix). The number of occurrences of such types can be defined as an input variable for a 3-digit logical function, which will give a decisive result for the central pixel of the class change matrix.
The values of the types of mutual arrangement of subpixels are consistent with the possible number of this type of arrangement in the considered block.
The next step is performed in a matrix consisting of subpixels of a 3 × 3 sliding window, where the analyzed pixel is central. The classification in the scan window is checked sequentially. A logical function is used to decide on the replacement of the central subpixel class (Figure 4). According to this function, the central pixel changes if the value of the function is 2, and does not change if the value of the function is 0. Changing the class of the central pixel requires additional analysis if the value of this function is 1. 2.3.
The hybrid classification method described above requires high-resolution source images. In turn, to increase the resolution of the bands of a multispectral image, stable clustering is required, which is based on the spectral signatures of the earth's covers. This leads us to the solution of using radiometric spatial feedback. This method is an iterative process described in Figure 5.
radiometric‐spatial feedback.
The iterative process continues until the spatial resolution of the bands is increased. As soon as the increase becomes insignificant compared to the previous iteration, the process stops and the final classification is carried out. 3. Test 3.1.
The method of multispectral imagery classification local quality improvement with radiometricspatial feedback in the selected study area was tested.
The study area was located around the city of Novomyrhorod in Kirovohrad Oblast (administrative province) of central Ukraine (Figure 6). It encompassed agricultural, wetland, and urban landscapes which elements substantially vary in spatial characteristics. For instance, extensive and spatially homogeneous croplands have sharp delineations in a form of roads and tree lines, while wetlands and wet grassland have meandering and vague boundaries. Relatively small built-up areas extended along streets and surrounded by highly heterogeneous household plots. However, most of these elements have similar spectral characteristics especially, during the mid-summer season. For their better recognition from satellite images, the application of spectral indices strongly recommended [13-15].
In this study, we focused on seven land cover types. They are artificial surfaces, croplands, treecovered areas, grasslands, wet grasslands, wetlands, and water bodies (Table 1).
Cloud-free Sentinel-2A multispectral instrument (MSI) image acquired on 06 July 2020 was downloaded from the U.S. Geological Survey (USGS) archive through the EarthExplorer interface (https://earthexplorer.usgs.gov/). The images were obtained at Level 1C: top of the atmosphere (TOA) reflectance (https://sentinel.esa.int/web/sentinel/user-guides/sentinel-2-msi).
The MSI measures the Earth's reflected radiance in 13 spectral bands from the visible near-infrared (VNIR) to shortwave infrared (SWIR) wavelength regions, with spatial resolutions from 10 to 60 m, as shown in Table 2.
Band 1 (0.43–0.45) Band 2 (0.46–0.52) Band 3 (0.54–0.58) Band 4 (0.65–0.68) Band 5 (0.7–0.71) Band 6 (0.73–0.75) Band 7 (0.76–0.78) Band 8 (0.78–0.90) Band 8A (0.85–0.87) Band 9 (0.93–0.95) Band 10 (1.36–1.39) Band 11 (1.56–1.65) Band 12 (2.10–2.28)
Resolution (m) 60 10 10 10 20 20 20 10 20 60 60 20 20
To provide atmospherically corrected images essential to calculations of spectral indices, Level 1C data was processed to Level 2A: bottom of the atmosphere reflectance (BOA).
The Sen2Cor tool (https://step.esa.int/main/snap-supported-plugins/sen2cor/) from the European Space Agency (ESA) Sentinel Application Platform (SNAP) was used to perform the corrections for the Sentinel-2 image. During the processing, Sen2Cor discarded the three bands (B1, B9, and B10) that consider the effects of aerosols and water vapour on reflectance. Then, the Sentinel-2 bands acquired at 20 m data were previously resampled using the nearest neighbour method to obtain a layer stack of 10 spectral bands at 10 m. At the final stage of data pre-processing, the obtained image was resized by an area of 1500 × 1000 pixels (Figure 6) for testing the method proposed in the study. 3.3.
Spectral indices, being nonlinear transformations of original spectral bands, substantially enhance
the classification quality of complex classes [
The SI used in this study are: (a) the normalized difference vegetation index - NDVI [17]; (b) the red-edge NDVI -RENDVI [18]; (c) the red edge ratio vegetation index – RERVI [19, 20]; (d) the normalized difference water index -NDWI [21]; (e) the modified normalized difference water index - MNDWI [22]; (f) the normalized difference moisture index - NDMI [23]; (g) the normalized difference built-up index – NDBI [24].
The formulations and the bands used to calculate the spectral indices from Sentinel-2A MSI are shown in Table 3.
NDVI RNIR Rred B8 B4
RNIR Rred B8 B4 RENDVI RNIR Rred _ edge B8 B6
RNIR Rred _ edge B8 B6
RNIR Rred edge Rgreen RNIR Rgreen RNIR Rgreen RSWIR Rgreen RSWIR RNIR RSWIR RNIR RSWIR RSWIR RNIR RSWIR RNIR B B6
8 B B
3 B3 B8
8 B B11
3 B B11
3 B B12
8 B B12
8 B11 B B11 B8 8
Guided by the data flow diagram described in Figure 5, a two-iteration process was carried out, which included 2 hybrid classification blocks, 2 blocks of spectral bands spatial resolution enhancement, and the final classification.
The results of evaluating the spatial resolution of spectral bands after the second iteration using the edge spread function are presented in Table 4 and show a steady increase of spatial resolution in bands 5-7 and 8a. It is assumed that a more stable increase of the spatial resolution in bands 11 and 12 can be achieved after attracting additional spectral signatures for reference clustering.
An illustration of the multispectral imagery classification local quality improvement with radiometric-spatial feedback is shown in Figure 7.
spectral bands, b ‐ a subset of the area of interest, c ‐ final classification after the spatial resolution enhancement of spectral bands
In the presented test section, you can see that the class divergence has increased, as a result of which there are no obvious errors in the final classification when the pixels of the artificial object class appear in the middle of the field class. All linear extended features, such as forest belts and roads, retained their shape and got rid of distortions caused by larger pixels from bands with a lower spatial resolution.
The article presents a solution to a radiometric obstacle by using a hybrid approach to classification, as well as a solution to a spatial obstacle by the spatial resolution enhancement of lower-resolution spectral bands. Radiometric-spatial feedback has also been established.
Further research should be aimed at developing an algorithm for determining the most appropriate clustering method for each particular class. And to increase the quality of the spatial resolution enhancement in all spectral bands, it is planned to use spectral signatures databases.
[5] A.A. Andreiev, Hybrid approach to classification of remote sensing data, CERes Journal, vol. 6 (2), 2020, pp. 32–37. [6] S. Theodoridis, K. Koutroumbas, Pattern Recognition, fourth edition, Elsevier, 2009, pp. 633– 635. [7] B. Tso, P. M. Mather, Classification Methods for Remotely Sensed Data, 2nd ed., CRC Press,
Boca Raton, USA, 2009. [8] L. Bruzzone, B. A. Demir, A review of modern approaches to classification of remote sensing data, Land use and land cover mapping in Europe, I. Manakos, M. Braun, Eds. Springer: Dordrecht, Netherlands, 2014, pp. 127–143. [9] O. Rukundo, H. Cao, Nearest neighbor value interpolation, International Journal of Advanced
Computer Science and Applications, 2012, pp. 25–30. doi:10.14569/IJACSA.2012.030405. [10] R. Keys, Cubic convolution interpolation for digital image processing, IEEE Transactions on
Acoustics, Speech, and Signal Processing, vol. 29, 1981, pp.1153–1160. [11] I. Piestova, S. Stankevich, J. Kostolny, Multispectral imagery superresolution with logical reallocation of spectra, Proceedings of the International Conference on Information and Digital Technologies (IDT 2017), IEEE, Žilina, 2017, pp. 322–326. doi:10.1109/DT.2017.8024316. [12] E. Zaitseva, I. Piestova, J. Rabcan, P. Rusnak, Multiple-Valued and Fuzzy Logics Application to Remote Sensing Data Analysis, 26th Telecommunications Forum (TELFOR), Belgrade, Serbia, 2018, pp. 1–4, doi:10.1109/telfor.2018.8612109. [13] S. Taddeo, I. Dronova, N. Depsky, Spectral vegetation indices of wetland greenness: Responses to vegetation structure, composition, and spatial distribution, Remote Sensing of Environment, 2019, 111467. doi:10.1016/j.rse.2019.111467. [14] M. Amani, B. Salehi, S. Mahdavi, B. Brisco, Spectral analysis of wetlands using multi-source optical satellite imagery, ISPRS Journal of Photogrammetry and Remote Sensing, vol. 144, 2018, pp.119–136. doi:10.1016/j.isprsjprs.2018.07.005. [15] X. Yang, S. Zhao, X. Qin, N. Zhao, L. Lian, Mapping of Urban Surface Water Bodies from Sentinel-2 MSI Imagery at 10 m Resolution via NDWI-Based Image Sharpening, Remote Sensing, vol. 9, 2017, 596. doi:10.3390/rs9060596. [16] S. Kokhan, A. Vostokov, Using Vegetative Indices to Quantify Agricultural Crop Characteristics, Journal of Ecological Engineering, vol. 21, 2020, pp.120–127. doi:10.12911/22998993/119808. [17] T. N. Carlson, D. A. Ripley, On the relation between NDVI, fractional vegetation cover, and leaf area index, Remote Sensing of Environment, vol. 62, 1997, pp. 241–252. [18] J. Chen, C. Yang, S. Wu, Y. Chubg, A. Linton, C. Chen, Leaf chlorophyll content and surface spectral reflectance of tree species along a terrain gradient in Taiwan’s Kenting National Park, Botanical Studies, vol. 48, 2007, pp.71–77. [19] Y. Kanke, B. Tubana, M. Dalen, D. Harrell, Evaluation of red and red-edge reflectance-based vegetation indices for rice biomass and grain yield prediction models in paddy fields, Precision Agriculture, vol. 17, 2016, pp. 507–530. doi:10.1007/s11119-016-9433-1. [20] Q. Cao, Y. Miao, H. Wang, S. Huang, S. Cheng, R. Khosla, R. Jiang, Non-destructive estimation of rice plant nitrogen status with Crop Circle multispectral active canopy sensor, Field Crops Research, vol. 154, 2013, pp. 133–144. [21] S.K. McFeeters, The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features, International Journal of Remote Sensing, vol. 17, 1996, pp. 1425–1432. [22] H. Xu, Modification of normalised difference water index (NDWI) to enhance open water features in remotely sensed imagery, International Journal of Remote Sensing, vol. 27, 2006, pp. 3025–3033. [23] B. C. Gao, NDWI—A normalized difference water index for remote sensing of vegetation liquid water from space, Remote Sensing of Environment, vol. 58, 1996, pp. 257–266. [24] Y. Zha, Y. Gao, S. Ni, Use of normalized difference built-up index in automatically mapping urban areas from TM imagery, International Journal of Remote Sensing, vol. 24, 2003, pp. 583– 594.