A classification system for hyperspectral images using convolutional neural networks is described. A specific network was selected and analyzed. The network parameters, ensured the maximum classification accuracy: dimension of the input layer, number of the layers, size of the fragments into which the classified image is divided, number of learning epochs, are experimentally determined. High percentages of correct classification were obtained with a large-format hyperspectral image, and some of the classes into which the image is divided are very close to each other and, accordingly, are dificult to distinguish by hyperspectra.
Classification of the land areas is gaining in importance for a wide variety of applications, and one of the most efective systems of classification features is hyperspectral data. It is known that the greatest advances in the field of recognition in recent years have been obtained using deep learning and convolutional neural networks. This report examines exactly this problem. The most important question in this case — what features to use at the input of the neural network. Earlier it was shown that a high probability of correct classification is possible only using spatial-spectral features. The dimension of the input image of convolutional neural network is limited; therefore, a shortened system of features — the principal components — is formed from the spectral components. Spatial features are obtained by forming fragments from the resulting system of spectral features, and the methods of this formation significantly afect the quality of the classification. The analysis of these methods is the main subject of research in this report.
The object that will be investigated in this report has appeared in many publications (Figure 1) [
Works in the direction of using neural networks for GSI classification have already been carried
out [
The use of networks pre-trained on millions of data is recommended. However, we have a
special case. Our training and recognizable images are small terrain fragments that cover the
marked (i.e. classified) areas of the GSI. Therefore, we will use a neural network that is not too
complex and without such layers as Max Pooling, Dropout, etc. [
The most efective way to do this is by converting the array of spectral features to the
principal components. The number of principal components will be determined by analyzing
the “rocky talus” — the graph of eigenvalues decreasing. It is presented in [
The most important role is played the MxN parameter, the dimension of the fragment cut out from the input layer. Too small size of the fragment will not allow revealing its spatial features. A large size of fragments reduces their number in the class, since the areas belonging to the classes have an arbitrary shape, as a rule, curvilinear, and the fragments are rectangular, so that too few (and sometimes no one) fragments fall on some classes. Thus, it is necessary to find a compromise between the size of fragments and their number in an area belonging to one class, and this is the main theme of this report.
Let us explain how the training sample is formed in our case. Its elements are fragments of the GSI, divided into marked areas. Each area contains its own number of elements, depending on the size of the section and its configuration. When forming a sample, all GSI is covered with square fragments of a given size, and if all the pixels of a fragment belong to the same class, this fragment is considered as an element of the corresponding class.
The sequence of classification stages.
1. The principal components of the GSI are calculated. 2. Directories of classes from 1 to 57 are formed. 3. From the file containing the GSI separation into classes (Figure 2), using a sliding window of size × and shift_M, shift_N, fragments are selected, all elements of which belong to the same class. Classes, the number of fragments of which exceeds the specified threshold, participate in training. 4. Network parameters are adjusted: number of layers, kernels size, number of feature maps, number of classes. 5. Parameters of the training procedure are adjusted: numbers of classes, number of training epochs, objects of each class are divided into training and validation samples (as a rule, in a ratio of 7 : 3). 6. Training procedure is started.
Trained network is visualized in Figure 4.
The number of weights adjusted as a result of training is 158184. There are 3 convolution layers in the network, 3 layers of normalization (batchnorn layers), which speed up the learning procedure; three activation layers (ReLu layers), which perform nonlinear transformation, and softmax and classoutput layers, provided the recognition procedure. Using the input layer (imageinput), training and recognizable images are introduced.
Let’s consider the results of experiments. Note, that the only criterion of the classification efectiveness is the classification accuracy, which is defined as the ratio of correctly objects classified number to the total number of objects (this term — accuracy — is used along with the term “probability of correct classification”). Let us note the feature of our method of a training sample forming, training and classification. With diferent sizes of fragments into which sections of classes are divided, and limiting the number of elements in a class, the number of classes will be diferent, which will not allow determining the actual dependence of the classification accuracy on the size of fragments, since two factors afect here: the size of the fragment and the number of classes. Therefore, we calculated the number of classes (18) for the maximum fragment size of 16x16 and then trained the network for all fragments with this number of classes. The dependence of the classification accuracy on the fragments size with number of classes 18 is shown in Figure 5.
It can be seen from this graph that the optimal fragment size is near 14× 14.
A very important factor afecting the classification accuracy is the dimension of the input layer, which is equal to the number of principal components used at the input. This dependence is presented in Table 1.
It can be seen that, starting with 5 principal components, the classification accuracy increases insignificantly.
A very important parameter of the network is the number of learning epochs. Dependence of classification accuracy on this parameter for a fragment 14 × 14 and the number of principal components — 5 is shown in Figure 6.
The classification accuracy monotonically increases with the number of epochs, taking a sharp jump from 20 to 30 epochs, although this function depends on the size of the fragments.
For the fragment 14× 14, it is already at 30 epochs actually comes out in saturation, and for a 5x5 size the classification accuracy continues to grow even at 50 epochs, as can be seen from Figure 7.
The classification accuracy also depends on the number of layers. In the previous work [
So, we have chosen the following network parameters: fragment size — 14× 14, number of principal components — 5, number of network layers 13, number of learning epochs 50. Classification results with the number of classes 33, (as seen in the learning function shown in Figure 9) — 99.72%, which, in our opinion, is a very good result.
The resulting table with the number of elements in each class and the probability of class recognition is shown in Table 2.
It can be suggested, that high values of classification accuracy are obtained due to overfitting of the neural network. This is an undesirable phenomenon that occurs under solving learning problems by precedents, when the probability of the trained algorithm average error on the test sample is significantly higher than the error on the training sample. From Figure 9 (bottom part), characterizes the behavior of the error in the learning process, follows that the error on the test sample is very insignificantly (by a part of a percent) higher than the error on the training sample, which means that there is no overfitting in this case. Table 2 shows the classification results, indicating the probabilities for each class.
From the class names it is clear that we did not integrate closely related classes into one (for
example, crops of corn, crops of soybeans), as it was done in other publications [
It should be said that the results obtained in this work significantly exceed the results of [
Thus, in this report we have analyzed the influence of the neural networks parameters on the accuracy of hyperspectral images classification. The network parameters and methods of forming a training sample are selected, which provide a very high classification accuracy (integral accuracy is 99.72%), and such a high accuracy is ensured on close classes (11 types of corn plowing, 14 types of soybean plowing). Analyzing such high classification accuracy, the following should be said. This is largely due to the way as the training and validation samples are formed, characterized by their very close mixing. At the same time, it is obvious that this method shows a certain limit of classification accuracy, from which it’s possible to deviate, for example, by increasing the fragments coverage step or forming the training and validation samples spatially separated.