-The known feature of hyperspectral images is a high spectral resolution, which allows us to identify materials and classify objects in images with high accuracy. However hyperspectral images contain substantial redundancy, which can be eliminated with the aid of dimensionality reduction techniques. In this paper, we propose and study several dimensionality reduction techniques based on the pretraining the encoder-decoder neural network with the results of the nonlinear mapping and principal component analysis techniques. The experiments performed on an open dataset show that the proposed techniques both provide the discriminative low-dimensional features and allow us to reconstruct source hyperspectral data with little error.
INTRODUCTION
Hyperspectral images are widely used nowadays in different fields such as agriculture, medicine, biology, chemistry, and so on. The known feature of hyperspectral images is high spectral resolution, which allows us to identify materials and classify depicted images with high accuracy.
However hyperspectral images contain substantial
redundancy, which can be eliminated with the aid of
dimensionality reduction techniques. The images obtained
after the dimensionality reduction stage can be processed
efficiently as much less data volume is involved in
processing. It is worth noting that dimensionality reduction
techniques are often used in different problems of image
analysis (see [
The most commonly used techniques for the
dimensionality reduction of hyperspectral data are linear
techniques such as Principal Component Analysis (PCA).
While a number of general-purpose nonlinear
dimensionality reduction procedures exist [
In the last years, neural network approaches become more and popular. In particular, autoencoder neural networks [5] were used for the dimensionality reduction of hyperspectral images. Such neural networks perform both nonlinear dimensionality reduction and provide the inverse mapping, which allows us to restore the source hyperspectral data up to some reconstruction error.
Recently, it was shown [
However, it was also shown [
The structure of the paper is as follows. In the next Section II, we give necessary theoretical information on the neural network architecture and the nonlinear mapping algorithm. In Section III we describe the training procedures used in the experimental study and describe the results of experiments. The conclusions and the list of references are given at the end of the paper.
II.
The autoencoder neural network proposed in [5] was earlier referred to as the autoassociative neural network. It consists of two consecutive parts called the encoder and decoder.
The encoder part takes a multidimensional vector x ϵ RM as input and produces corresponding low-dimensional representations y ϵ Rm so that m<M. The encoder consists of at least two fully – connected layers. The first layer contains some number of neurons (defined by the parameters of the neural network architecture) connected to all the components of an input vector. The last layer of the encoder contains the number of neurons equal to the desired dimensionality of the reduced space.
The decoder usually has the mirror-reflected architecture. It has the same number of layers with the same number of neurons, but this is not the necessary requirement. Anyway, the input layer of the decoder takes the reduced representation y ϵ Rm from the output of the encoder and restores the multidimensional vectors ~x ϵ RM. So the output layer of the decoder has the number of neurons equal to the input dimensionality M. The number of hidden layers and neurons is defined by the parameters of the neural network architecture.
As the number of neurons in the output layer of the encoder is less than the number of neurons in the input and hidden layers, this layer is often referred to as a bottleneck layer, and the whole network architecture is often referred to as a bottleneck architecture.
The autoencoder architecture is usually trained in selflearning mode by applying the same multidimensional vectors x ϵ RM to both input and output layers of the autoencoder. The training process itself is based on the minimization of the following cast function: E 1 N xi ~xi 2
N i 1 where N is the number of samples, and x ϵ RM, ~x ϵ RM are inputs and outputs of the network. After training the encoder can be used to perform the dimensionality reduction of the source data (direct mapping), and decoder can be used to restore the source data by its reduced representation (inverse mapping).
In this paper, we study, if the encoder and decoder parts can be trained separately to force the neural network to perform the mapping with the desired properties. It was shown earlier that the separate pre-training of encoder and decoder with the PCA results helped to perform the training more efficiently compared to the standard training.
In particular, the approach proposed in [
In this paper, we follow the similar scheme but use the
results of the nonlinear mapping algorithm instead of the
PCA, and perform the fine-tuning optionally to study if such
an approach can be more efficient than the standard PCA,
nonlinear mapping or the proposed recently autoencoder
pretrained with the PCA [
The nonlinear mapping is a numerical procedure that
performs the mapping (nonfunctional) of data into
lowdimensional space so that the data structure is preserved (see
[
As the pairwise distances cannot be preserved exactly in a common case, the so-called data mapping error is introduced:
N i , j d ( x i , x j ) d ( y i, y j ) 2
i , j 1 (i j ) Here N is the number of data points, d(xi,xj) is the distance between points xi and xj in the multidimensional space, d(yi,yj) is the distance between the corresponding points yi, yj in the reduced space, µ and are some constants. Usually, µ is the inversion of the sum of square distances between all the possible pairs of data points in multidimensional space, and ij are equal to one.
The minimization of the data mapping error is usually performed using the gradient descent technique. The coordinates of data points yi ϵ Rm are the tunable parameters.
In this paper, we use the stochastic gradient descent based on mini-batches to minimize the data mapping error. The overall algorithm for dimensionality reduction using the nonlinear mapping consists of the initialization of the coordinates yi with the results of the principal component analysis with the subsequent refinement of yi using the stochastic gradient descent. The optimization process (refinement) stops when the coordinates of the data points yi in the reduced space become stable.
As it was outlined in the introduction, in this paper, we study several variants of training the autoencoder-like encoder-decoder network. In particular, we consider the following techniques:
- The autoencoder network pretrained with the results of
the PCA technique (AE-PCA), as it is described in [
- The same autoencoder network pretrained with the results of the nonlinear mapping technique and fine-tuned using the standard approach (AE-NLM).
III.
In this section, we describe the results of the experiments,
which were performed using the Indian Pines dataset. This
dataset was acquired using the AVIRIS hyperspectral sensor.
This dataset contains 145 x 145 image pixels and 224
spectral components [
In all the described experiments, for the implementation of the neural networks, we used the Keras framework and Python language. The experiments were carried out on GeForce GTX 1070 ti.
For each considered neural network technique, we varied the number of hidden layers in the encoder and decoder and performed experiments for one and two hidden layers that correspond to four and six layers in the corresponding autoencoder networks.
The number of neurons in the input layer of the encoder and the output layer of the decoder was defined by the dimensionality of the input space that is the number of channels in the hyperspectral image. The number of neurons in the bottleneck layer varied from 1 to 10 according to the dimensionality of the reduced space. We also varied the number of neurons in the hidden layers. In particular, we used 64, 128, and 256 neurons in hidden layers.
According to the recommendations given in [
To measure the effectiveness of each particular approach,
we estimated both the reconstruction error as it is defined in
(1) and the classification accuracy using the reduced
representation. The latter indicator plays an important role in
hyperspectral image analysis problems, for example, in
vegetation type recognition [
For the latter indicator, we used the overall accuracy of the one nearest neighbor (1-NN) classifier. The accuracy itself was measured as a fraction of correctly classified image pixels. To measure the accuracy, at first, we performed dimensionality reduction using one of the studied techniques for all the pixels in the considered image. Then we split all the ground truth pixels into training and testing sets in the proportion 60/40. After that, we trained the classifier using the training set and estimated its accuracy using the test one.
In our first experiment, we compared different techniques described in Subsection II.C and different architectures from the viewpoint of the reconstruction error (1). The results of this experiment are shown in Fig. 1. In particular, we pretrained the encoder and decoder of the AE-PCA network for 50 iterations, fine-tuned the entire network for 50 iterations, and then measures the reconstruction quality. quality indicator. The experiment was carried out for a different number of layers and neurons.
As can be seen in the figure, the reconstruction error decreases with the growth of the dimensionality m of the reduced space defined by the number of neurons in the bottleneck layer, which is an expected result.
While we cannot highlight any winner technique in this experiment, we should note, that the AE-NLM technique often shows better results. It means that the nonlinear mapping result, which was used for training, provide the ability to restore the source data with quite a good quality. This also means that the decoder trained on the NLM data can be used as an inverse mapping for the NLM.
For the AE-NLM network, we trained the network with the same strategy, but used the NLM results instead of the PCA results at the pretraining stage. For the ED-NLM network, we trained separately encoder and decoder for 100 epochs. After the training, we measured the error (1) as the
In our second experiment, we compared the considered techniques from the viewpoint of the classification accuracy. The results of this experiment are shown in Fig. 2. In this figure, we added the results for the classical linear (PCA) and nonlinear (NLM) dimensionality reduction techniques.
As can be seen, the proposed techniques provided better results than the classical approaches in most cases. Again, it is difficult to outline any approach. Nevertheless, we do not observe any substantial advantages in the fine-tuning of the NLM initialized network over the version with separate encoder and decoder.
CONCLUSION
In this paper, we studied several dimensionality reduction neural network techniques based on autoencoder architecture. We compared the proposed techniques from the viewpoint of the reconstruction error and the accuracy of the per-pixel classification.
We showed that the proposed techniques outperformed the baseline (PCA and NLM) approached in terms of the classification accuracy in almost all the considered cases. The decoder trained using the results of the NLM can be successfully used as an inverse mapping for hyperspectral image analysis.
The work was partly funded by RFBR according to the research project 18-07-01312-a in parts of «2. Method» - «3. Experiments» and by the Russian Federation Ministry of Science and Higher Education within a state contract with the «Crystallography and Photonics» Research Center of the RAS in parts «1. Introduction» and «4. Conclusion».