Recently, graph neural network (GNN) has drawn increasing attention for hyperspectral image (HSI) classification, and there are many related works have been proposed. However, previous works are focus on either the superpixel-level local spatial information or the pixel-level global spectral information. In this paper, a novel Graph Neural Sparsification Network (GNSN) is proposed for HSI classification. In our method, a fully connected graph is adaptively constructed to make full use of local spatial information and global spectral information. Besides, we apply a neural sparsification technique to remove potentially task-irrelevant edges in case of misleading message propagation. The resulting network (GNSN) is end-to-end trainable. We conduct experiments on three popular benchmarks, including Indian Pines, Pavia University, and Kennedy Space Center, and achieve state-of-the-art performance on all three datasets.
computation and limits its applicability. Superpixel-level adjacent graph is another commonly used graph for HSI HSI classification is a fundamental and challenging prob- classification [ 4, 5], which is usually accomplished by a lem in HSI processing [1], which aims to assign a specific superpixel segmentation algorithm. However, this graph label to each pixel in the image. It has been widely ap- only focuses on the local structure of the data. In order to plied to many scenarios, such as military target detection, capture the long-range contextual information, multiple vegetation monitoring, and disaster prevention and con- convolution layers are required to be stacked, which is trol. highly ineficient and will bring potential concerns of
Convolutional neural networks (CNNs) have proven over-smoothing. their efectiveness in image processing by automatically Aiming to address the above issues of the superpixelextract discriminative features. Many CNN-based vari- level adjacent graph, we propose a novel GNN-based HSI ants have been designed for HSI classification from the classification algorithm (GNSN), in which a fully con1D-CNN to the 3D-CNN, from the single CNN to the nected graph is adaptively constructed to make full use hybrid CNN. Although the existing CNN-based methods of local spatial and global spectral information. Furtherhave achieved good performance to some extent, they more, in order to avoid potentially misleading message need many training parameters and tend to be overfitting propagation, we apply the neural sparsification technique due to the scarcity of the training samples. Meanwhile, to remove potentially task-irrelevant edges. The main CNNs tend to blur the classification boundary using a contributions of this paper are summarized as follows: ifxed shape kernel around the central pixel to extract features. Therefore, the accurate classification of HSI is still challenging.
To build more flexible models, many GNN-based methods have been proposed for HSI classification. Generally, there can be two categories according to the types of adjacent graph. One is pixel-level adjacent graph[2, 3].
This type of graph can directly propagate information between nearby and distant regions, while it takes each pixel as a node of a graph, leading to a massive amount of • We incorporate both local and global information by a learnable graph, which will automatically decide whether the model will likely pay more attention to the nodes near it or pay more attention to the nodes far away from it; • GNSN applies a sparsification strategy that favors subsequent classification task by removing potentially task-irrelevant edges; • Experiments on three benchmark HSI datasets show that GNSN outperforms other compared HSI classification methods.
C o n v ( 1 * 1 ) S o f t m a x
computational complexity and meanwhile preserve the local structure of HSI, the GNN-based models usually The task for hyperspectral classification aims to predict work on superpixel-based nodes but not the pixel-based the labels of all the pixels in the HSI by a classification nodes. Similarly, we first apply the simple linear iterative model given the supervised data set. As illustrate in Fig. clustering (SLIC) method to partition the entire image 1, we present a Graph Neural Sparsification Network for into many spatially connected superpixels. By treating the HSI classification model to adaptively capture both each superpixel as a graph node and taking the average local and global contextual information. GNSN takes spectral signatures of each superpixel as the node feature, the original image data ∈ R× × and the asso- the graph projection assigns to a set of vertices and ciation matrix ∈ R × produced by a superpixel gets the feature matrix of the vertices in the graph by segmentation algorithm as input, where , and the form of matrix multiplication as follows represent the height, width, the number of bands, and the number of superpixels respectively. By treating superpix- = ˆ Flatten(), (1) els as the vertexes in the graph, we transform the image to the graph representation ∈ R× . Applying the where ˆ is the normalized by column, i.e., ˆ, = adaptive graph construction discussed below, we can , / ∑︀ , . Flatten(· ) denotes flattening the HSI obtain the similarity matrix ∈ R× that captures data by the spatial dimension, and the association matrix both local and global structure in the image data. Subse- ∈ R × is introduced by SLIC that is defined as quently, the sparsification operation removes most taskirrelevant edges in the graph. Following the paradigm of {︃1, ∈ , GCN, a two-layer GCN network is applied on the graph to , = 0, otherwise, (2) further propagate information, resulting in a high-level feature map ̃︀ ∈ R× , where is the dimension of where is the -th superpixel that consists of several the feature map. After the graph reprojection, the corre- homogeneous pixels. sponding coarse segmentation result ̃︁ ∈ R× × is We perform graph reprojection to transform the resultobtained in the original grid form. Finally, the refined fea- ing features back to the original coordinate space. The ture map concatenated with the original feature passes graph reprojection operation takes the inputs of transthrough a conventional 1 × 1 convolutional layer to get formed vertex features ̃︀ and the assignment matrix , the final classification result. GNSN consists of four main and produces the corresponding 2D feature map ̃︁. This blocks: Graph Projection and Reprojection, Graph Con- operation is defined as struction, Sparsification, and GCN. In the following, we will describe each block in detail. ̃︁ = Reshape(̃︀ ), (3)
Graph Projection and Reprojection. To reduce the of the flattened data. where Reshape(· ) denotes restoring the spatial dimension cency matrix in our formulation) is constructed according to the similarity between diferent nodes. Denote as the (, )-element of the adjacency matrix , we have:
= [ || ] + (, ), tion. And the learnable graph will automatically decide whether the model will likely pay more attention to the nodes near it or pay more attention to the nodes far away from it.
Sparsification Network.
We focus on -neighbor graph for graph neural sparsification. The -neighbor graph is obtained by repeatedly sampling edges for ferentiable, we apply Gumbel-Softmax to generate differentiable discrete samples. Firstly, softmax function is employed to compute the probability to sample the edge, which is , = ∑︀ exp (︀ ,
︀) ∈N exp (,)
(5) Softmax as follows , = ∑︀ exp (︀ log (︀ ,
︀) + )︀ / )︀ ∈N exp ((log ( ,) + ) / ) where , is a scalar that represents weather to sample the edge between and , = − log(− log()) with randomly drawn from Uniform (0,1), and is a hyperparameter that controls the interpolation between discrete distribution and continuous categorical densities.
Graph Convolution Network (GCN).
operation is defined as
gate information on the graph. Specifically, for a single
graph convolution with its parameter ∈ R× ′ , the
Then we can generate diferentiable samples using Gumble- sessions to obtain stable results, and the best results are
We perform two-layer GCN from [7] to further propa- traditional machine learning and deep learning models
highlighted in bold.
, (6)
3.2. Classification results
In our experiments, we compare our method with several
HSI classification methods: SVM, Multilayer Perception
(MLP), 3DCNN [8], NLGCN [9] and GSAGE [
Note that NLGCN and GSAGE neglect fusing the information of both local and global information, thus resulting pared to the adjacency graph described in [5, 6], where the receptive field is restricted to the neighbors, we can where (, ) = 1 if the -th region and the -th region are spatially related, and otherwise (, ) = 0. Com- continuous imaging of 115 spectral bands with a size of see that Eq. (4) provides both global and local informa- For classification purposes, 13 classes representing the each node in the original graph. To make samples dif- three metrics, including overall accuracy (OA), average efectiveness of our proposed method GNSN on three real-world benchmark datasets, namely Indian Pines (IP), Pavia University (PU), and Kennedy Space Center (KSC).
The Indian Pines dataset is captured over the agricultural (4) test site in the Indian Pines, which contains 200 spectral bands with a size of 145 × research and analysis. The Pavia University dataset is the
145 and 16 terrain classes for 340. This dataset includes nine land-cover classes.
The KSC data contains 176 bands with a size of 512× 614. various land cover types that occur in this environment were defined for the site. 3.1. Experimental settings To quantitatively evaluate diferent models for hyperspectral image classification tasks from various aspects, accuracy (AA), and kappa coeficient, are used to evaluate the performance of all the compared methods. For all the datasets, we randomly select 30 labeled pixels in each class for training and choose 15 labeled pixels if the corresponding class has less than 30 samples. Besides, the learning rate and the number of training epochs are set to 0.001 and 2000, respectively. All the reported results are calculated based on the average of ten training
Graph Construction. The graph (which is the adja- In this section, we conduct experiments to validate the ̃︀ = ( ),
(7) in lower performance than our GNSN. where can be any nonlinear activation function. We can see that GCN calculates new features of a vertex as a weighted average of features of its neighbors on a graph, which allows the vertices in the same cluster to have similar features, making the subsequent classifications much easier.
In this article, we propose a novel Graph Neural Sparsiifcation Network (GNSN) for hyperspectral image classification. Diferent from prior way of constructing the adjacent graph, the proposed GNSN constructs a learnable graph that reveals the local and glocal structure of HSI. Besides, a diferentiable sparsification technique is applied to favor subsequent classification task by removing potentially task-irrelerant edges. Experimental results reveal that our method significantly outperforms its counterparts in terms of OA, AA, and Kappa.