Accurate detection of landslides plays an important role in post-disaster search and rescue operations. In this paper, we propose SwinLS for eficient landslide detection in remote sensing images using the swin transformer model. We explore how to eficiently utilize the self-attention mechanism in swin transformer for landslide detection tasks from two aspects. The first aspect is the spectral selection and data enhancement. The second aspect is to reduce imbalanced interference. After that, the performance of the improved swin transformer model is greatly improved, which provides a preliminary exploration for the application of the visual transformer model for remote sensing landslide detection tasks and even anomaly detection tasks. Finally, the proposed SwinLS, achieved the 2nd place in the test leaderboard with 73.99% F1 score, and it difers from the 1st place of 74.54% by only 0.55% F1 score.
supervised pixel-level classification task. Among these models, they found through experiments that ResUnet Landslides have become more frequent due to drastic achieved the best verification performance on landslide climate change, surface activity, and accidents, threaten- detection tasks, which is due to its reasonable utilization ing the lives and properties of residents in these areas. of multi-scale features.
Accurate detection of landslides plays an important role Nonetheless, we believe that this is not enough,
bein post-disaster search and rescue operations. As an efi- cause two important issues of landslide detection are
cient and convenient solution, automatic interpretation ignored. The first is the spatial correlation of landslide
of landslide areas from remote sensing images has re- data and the second is the imbalanced problem in
landceived extensive attention from scholars [
The Landslide4Sense dataset contains multi-spectral of landslides is much smaller than that of non-landslide,
imagery from multiple regions and cities collected by which is in line with the anomaly detection problem. To
Sentinel-2 satellites. The data format is pixel blocks of address these issues, we introduce the swin transformer
size 128 with 14 spectrum bands including RGB, VEG [
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License tra are suitable for performing self-attention based
feaCPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) ture aggregation. Finally, we use the RGB spectrum to
train the swin transformer model. To complement it, we
use CUTMIX [
To solve the imbalance problem in landslides detection, we design a two-stage balanced training strategy to make the model better focus on foreground (landslides) categories. In the first stage, we train the feature extractor and classifier with weighted cross entropy loss to get better feature representation. In the second stage, we ifx the feature extractor and fine-tune the classifier with ordinary cross entropy loss to weaken the bias of the classifier. This strategy better mitigates the misleading of the classifier due to the imbalance between landslide classes and non-landslide classes.
Finally, the proposed method called SwinLS, achieved the 2nd place in the test leaderboard with 73.99% F1 score, and it difers from the 1st place of 74.54% by only 0.55% F1 score.
As shown in Figure 1, SwinLS is a network of codec structure, and there are hop links between codecs. Its encoder is composed of the base structure of swin transformer, which has a powerful feature representation capability. Its decoder uses a convolutional structure for decoding and fusing multi-level features for output. 2x 4x 8x 16x The loss is the image-level loss performed in highlevel semantic features in the encoder to assist training, which is defined as follows, = − 1
∑︁ () log (()),
| | ∈
(2)
where is pointer function. When there is a pixel stand
for positive sample (landslide) in , its value is 1,
otherwise it is 0. (· ) is a fully connected layer with a
global pooling operation. stands for the total data set.
The loss is defined as follows,
= −
1 ∑︁ log (()),
| | ∈
(3)
where stands for the number of negative samples
(non-landslides) and stands for the number of
positive samples (landslides) in any input image . As
mentioned in [
arg min + .
(4)
proposed above, respectively. Due to the limited number Encoder Decoder of submissions of test data in the final stage, the data provided in our ablation experiments are all performance on the validation set.
Input RGB image Swin transformer Multi-scale Fusion Pixel-level loss Spectral selection Since the Transformer model SSttaaggee12::TFriaxiending ImLNaagone:d0-slelYivdeeesl:?l1osSSsttaaggee21:: TTrraaiinniinngg anteteednstiotno mpeecrhfaonrmismfetoateuxrteracatghgirgehg-alteivoenlsseumsainngticsfeelaf-tures. If irrelevant spectral information occupies domi
Model structure nant information, it will have a significant impact on the Figure 1: Network structure diagram. performance of swin transformer. To this end, we perform a set of experiments verifying the efect of diferent
For self-attention mechanism in swin transformer to spectral inputs, as shown in Table 1.
work better in the landslide detection, we performed In Table 1, we discovered an interesting phenomenon.
spectral selection experiments (see Tabel 1 and Figure 1), With the increase of spectral banks, the performance of
Finally, we selected the RGB spectrum from the multi- the fully convolutional models, such as deeplabv3 and
spectral input into the model. To alleviate the foreground Unet, show a gradually increasing trend, while the
perand background imbalance in landslide detection, we de- formance of swin transformer is severely degraded. We
sign a two-stage training strategy. In the first stage, the ifnd out it is because the dimensionality enhancement
codecs are trained simultaneously. For any input samples in the fully convolutional model may attenuate the
neg ∈ × ℎ× 3, we use weighted cross-entropy loss ative efects of irrelevant channels. Swin transformer,
and Lovasz loss [
In addition, the swin transformer model has a large
capacity and is easier to memorize and lose generalization
under such simple data. To this end, we designed data
augmentation experiments to verify the transformation
methods for landslide detection using only RGB spectral
information, as shown in Table 2. We also add the Unet
model that uses all banks to compare with it.
[
are visualized in Figure 4.
Precision (%) Recall(%) F1(%) 73.4 65.2 69.3 72.4 78.2 74.7 80.5 79.5 77.1 74.2 73.9 72.7 73.7 74.9 76.1
In Table 4, we found that when is small, the accuracy call rate will improve significantly. This is because when the is small, the selected landslide area is only located in the center of the landslide, and the pixels in the surrounding area will be ignored due to low confidence. This makes the self-trained model tend to predict all surrounding similar blocks as landslides, resulting in increased over-detection of landslides. As the selected landslide area continues to increase, the accuracy of the model This shows that with the addition of many inaccurate pseudo-labels, it has played a strong role in preventing over-detection. And the model can learn more knowledge about the samples to be tested from the noisy training data, which increases the accuracy. rate after self-training will degrade seriously, but the re- pseudo-labels with diferent values, as shown in Figure continues to rise, and the recall rate begins to decline. (Program No. 2022ZDLGY01 -12) and National Key R&D