Unsupervised hyperspectral image (HSI) classification is an important but challenging task in the hyperspectral processing community. Despite great success, previous HSI clustering approaches belong to ofline clustering which is often performed in a transductive scheme, thus failing to generalize to large-scale and unseen scenes. In this paper, we propose an online and deep clustering model for large-scale HSI clustering, termed spectral-spatial contrastive clustering (SSCC). Specifically, SSCC performs contrastive learning based on a series of semantic-preserving spectral-spatial augmentation to simultaneously maximize the intra-class agreement and inter-class variation, which are implemented by an instance-level contrastive loss and a cluster-level contrastive loss, respectively. The SSCC model is trained in an end-to-end fashion with minibatch, allowing it to eficiently handle large-scale HSIs. We assess the performance of SSCC on real HSI and show that SSCC signicfiantly advances the state-of-the-art results with 8.41% improvement on accuracy.
they focus on ofline HSI clustering tasks, i.e., the
clustering is dependent upon the whole dataset, which limits
Hyperspectral image (HSI) consists of hundreds of nar- their application on large-scale online learning scenarios.
row bands with rich spectral and spatial information, To address the first drawback, some attempts, e.g.,
revealing the spectral property of the area or object of [
Without label information, unsupervised HSI classifi- a result, most current works compromise on this
probcation becomes a challenging task, thus leading to un- lem by verifying on smaller scenes, lacking dependable
competitive accuracy. Many eforts have been devoted to performance evidence from large-scale HSI data.
bridging the gap between supervised models and unsu- Fortunately, self-supervised learning (SSL) has emerged
pervised models [
ResNet-18 Shared ⋅ as the backbone and the projection heads respectively, in which the instance/cluster projection head transforms data into 128 and dimension, where denotes the number of targets. We use the cluster projection head to perform clustering at the inference stage.
⎞ (2)
SSCC contains two projection heads. We use (· ) and (· ) to denote the instance-level projection head and cluster-level projection head. Each head takes and as inputs and produces a pair of predictions, i.e., denoting as and for (· ) and and for (· ). The goal of (· ) is to encourage the intra-class agreement, instead and (· ) aims to encourage the inter-class variation. Specifically, we achieve these by defining the following contrastive losses. Let {(1), ..., (+1), ..., (2)} (), be 2 augmented samples with batch size of . The instance-level contrastive loss over sample is given by Motivated by recent contrastive clustering developments in visual representation learning [20], which performs clustering jointly with contrastive learning, we introduce a novel SSCC approach for large-scale HSI clustering. ⎛ exp⎛⎜⎝ sim(︂ (),())︂ ⎟⎞⎠
The core of SSCC is to maximize the similarity between representations of positive pairs from both instance space and cluster space, as shown in Fig. 1. The SSCC conducts a spectral-spatial augmentations, then the augmented pairs are forwarded into a weight-sharing backbone encoder, (· ), resulting in deep representations ℎ and ℎ. Behind the encoder, projection heads consisting of instance projection head and cluster projection head are carried out to maximize the similarity between prediction pairs. More specifically, we adopt ResNet-18 and MLPs ℒ() = − log ⎜⎜⎝⎜ ∑=︀1⎢⎡⎣exp⎛⎜⎝ sim(︂ (),())︂ ⎞⎟⎠+exp⎛⎜⎝ sim(︂ (),())︂ ⎞⎟⎠⎤⎥⎦ ⎟⎟⎠⎟ (1) Here, denotes a temperature parameter and is a similarity function. We adopt cosine similarity in the paper, i.e., sim ︁( (), ())︁ = ︁( ())︁ (︁
())︁ ‖()‖ ‖()‖ 1https://pytorch.org/ ⎞ (4) (6) 7.5 7.0 6.0 5.5
s 6.5 soL 0 100 200 300 Training Epoch 400 500
(3) where is another temperature parameter. Furthermore, the cluster-level contrastive loss can be defined as ℒ = 1 2 =1 ∑︁ (︁ ℒˆ () + ℒˆ())︁ − ( ) , where ( ) denotes the entropy of cluster assignment probabilities across the whole augmented minibatch, which is used to avoid the trivial solution, and can be computed by ( ) =∑︀ ︁( (· ))︁ log ︁( ︁( (· ))︁ =1 (5) Finally, the complete training loss function of SSCC is indicated as + ︁( (· ))︁ log ︁( ︁( (· ))︁ .
ℒ = ℒ + ℒ .
The proposed SSCC can be trained in an end-to-end fashion. Specifically, we adopt the widely-used Adam approach as the optimizer with a learning rate of 0.00002, batch size of 128, and 2 regularizer of 0.00005. During the inference stage, we feedforward any given sample and lock the spectral-spatial augmentation process, the output of the cluster-level projection head is regarded as the prediction of the sample.
In Fig. 2, we show the efect of patch size and training epoch. From the curves, we can conclude that: 1) The clustering ACC of SSCC is dramatically increased along with the training; 2) SSCC achieves considerable clustering ACC at a completely random initial status (epoch=0), signifying the feature representation power of SSL; 3) A larger patch size is often more beneficial to the SSCC model, especially when using a small train epoch. 3. Experiments We further present the evolution of feature representations across the training process of SSCC, as shown in Fig. 3.1. Datasets and Setup 3. It can be seen that the features tend to become more compact within a certain class and more separable from In this paper, we conduct experiments on the widely-used each other class. This proves that our SSCC can capture Indian Pines dataset. We follow the training settings sug- the intrinsic spectral-spatial information of HSI and obgested in [20] and the baselines reported in [11]. Several tain superior clustering performance and generalization clustering metrics are used to quantize the clustering ability. performance, including producer’s accuracy, overall accuracy (OA), Kappa coeficient (Kappa), and purity. It should be noted that more datasets and more extensive 4. Conclusion experiments will be provided in our future work.
This paper presented a novel SSCC model for large-scale HSI online clustering task. The SSCC model follows a contrastive learning pipeline and consists of two projection heads associated with the instance-level and the cluster-level contrasting. Furthermore, we introduced a semantics-preserving augmentation pool based on the characteristic of HSI. SSCC is featured by ofline clustering, minibatch and end-to-end training, making it easy to deal with large-scale HSI. Experimental results on real HSI show that SSCC can achieve state-of-the-art clustering performance with significant margins over previous works. The success of SSCC ofers a powerful alternative for unsupervised HSI classification. It should be noted that this is a preliminary work and further analysis on the proposed method will be conducted in our future works.
[10] J. Lei, X. Li, B. Peng, L. Fang, N. Ling, Q. Huang,
Deep spatial-spectral subspace clustering for hyperThis work was supported in part by the National Nat- spectral image, IEEE Transactions on Circuits and ural Science Foundation of China (NSFC) under Grant Systems for Video Technology 31 (2021) 2686–2697. Nos. 61773355 and 61973285, the Fundamental Research [11] H. Zhai, H. Zhang, L. Zhang, P. Li, Sparsity-based Funds for National Universities, China University of Geo- clustering for large hyperspectral remote sensing sciences (Wuhan) under Grant CUGGC03 and 1910491T06, images, IEEE Transactions on Geoscience and and the National Scholarship for Building High Level Uni- Remote Sensing (2020) 1–15. doi:10.1109/TGRS. versities, China Scholarship Council (No. 202006410044). 2020.3032427. [12] L. Jing, Y. Tian, Self-supervised visual feature learnReferences ing with deep neural networks: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence (2020) 1–1. doi:10.1109/TPAMI.2020. 2992393.