Each image patch of the Landslide4Sense dataset has 14 bands, which is diferent with the RGB satellite imagery. A new model called Multispectral U-Net was proposed. We conduct experiments and apply two modifications to traditional U-Net to improve the performance. Firstly, the skip-connection and spectral normalization regularization are used to enhance the model ability of feature extraction. Secondly, an extra Inverted Residuals and Linear Bottlenecks branch is introduced for 10 meters resolution bands. We split the oficial dataset into two parts, with 3539 images for training and 260 images for testing. Multispectral U-Net accomplishes the best performance, with an F1-score of 77.83%, followed by the baseline U-Net and Deeplabv3+. The model has better performance on multi-spectral data and small objection for landslide detection.
Landslides are the most frequent geological disasters in nature, which lead to heavy damage to the infrastructure every year. It is very important to detect the landslide position and take some measures to avoid more loss, and landslide interpretation from satellite images has already become a reliable technique for landslide investigation [1]. However, manual landslide interpretation is dificult, as it is time-consuming and over-reliance on professional experience.
In recent years, the deep learning methods have made a large number of achievements in most computer vision tasks, including classification, object detection and semantic segmentation [2]. Semantic segmentation model requires to obtain the exact shape of the object, which is widely used in high precision tasks, such as medical image segmentation and the road area segmentation. For example, Ronneberger at et. [3] proposed a symmetrical and malleable structure, called U-Net, for the biomedical image segmentation.
With the development of astronautical technology, it is easy to access to the remote sensing data for a wide range of area. Meanwhile, a growing number of landslide datasets are released, which make it possible to adopt the deep learning methods in landslide detection, such as [4, 5, 6] and etc. The most common datasets provide the RGB image to keep the same input format with the original deep learning methods. It can take some advantages, for example, researchers can adopt these methods without any changes, and the pre-trained weights can shorten the training time. However, most satellite imagery has more than three bands, for example, Landslide4Sense [7] provides the data has 14 bands. If the model only uses the RGB bands, it can lead to information loss. In this paper, we will try to use the multi-spectral data for landslides detection tasks.
duced in the MobileNetv2 [11]. The high-dimensional The Landslide4Sense data has more spectral bands than feature maps contains manifolds, which can be comRGB image. Considering the diferences, we introduce pressed until they span the whole space. However, it some modification on U-Net, so that it can adopt the the dimension is reduce a lot, Relu will cause informamulti spectral satellite imagery. tion loss. The features propagate from low-dimension to high-dimension by the expansion layer, and the pro2.1. Multispectral bands selection jection layer does the opposite. Projection convolution uses the linear activation instead of Relu to alleviate the Landslide4Sense [7] dataset provides remote sensing im- conflict between dimension reduction and non-linear opage patches with 14 bands. One of the general methods is timization. In the Multispectral U-Net the features in that feeding all the bands together to the model, but could this branch will be fused with U-Net features, which can not achieve a high score in our experiment. In the valida- get more refined feature maps to improve the model’s tion dataset, U-Net [3], Deeplabv3 [8] and Deeplabv3+ [9] performance. can achieve 0.65, 0.66 and 0.67 f1 scores respectively. The Inspired by the Real-ESRGAN [12], we use U-Net with multispectral data from Sentinel-2 has diferent Spatial skip-connection as the backbone, and use the spectral resolution [10], as table 1. normalization regularization for a stable training process. Due to the data size is only 128 pixels, we only use three 2.2. Multispectral U-Net downsampling layers to avoid information loss. Additionally, we change the activation function to SMU, which can improve the model’s performance in this dataset.
In the Multispectral U-Net model, we input all 14 bands to U-Net branch and 10 meters resolution bands (B2, B3, B4, B8) to Inverted Residuals and Linear Bottlenecks branch, and this strategy can get the best results.
Net, which aim to enrich the semantic features extracted from the 14 bands. Multispectral U-Net’s architecture is illustrated in figure 1. The model structure includes two parts, Inverted Residuals and Linear Bottlenecks (upper part) and U-Net (lower part).
Recall We train the model on NVIDIA GeForce RTX 3090 GPU, and the training parameters are 8 batch size, adam optimizer, cosine learning rate (0.00001) with warmup and restarts and the cross-entropy loss. As we explore the common image data augmentation strategies, such as lfip and rotate, and do not get an improvement on the Figure 2: The landslide detection maps obtained by diferent model performance, we do not add the augmentation in deep learning methods. the following experiments.
Then, we split the oficial dataset into two parts, with 3539 images for training and 260 images for testing. After 4. Conclusion training 200 epochs, we evaluate the model with recall, precision and f1 score metrics, which is illustrated in the In this study, we first adopt three diferent models, intable-2. The recall of the Multispectral U-Net is signifi- cluding U-Net, Deeplabv3, Deeplabv3+, to train the multicant higher than other two models, with 75.41, and by spectral remote sensing data without any changes and this reason, although the precision of Multispectral U-Net pre-trained weights. It shows that there is only slightly is lower than U-Net, its f1 score is also better than that diferent of the models’ performance. Then, we first proof U-Net. Deeplabv3+ do not have a very high precision, posed a new U-Net-based approach denoted as Multiand we think the reason is that the more downsampling spectral U-Net for the better use of the multi-spectral layers cause the edge information loss. data. It combines two branches, called Inverted Residuals
Figure 2 shows the prediction of diferent models, and Linear Bottlenecks and improved U-Net, and the feaand each row represents a single data. In the first data, tures from two branches will be fused by concatenation the landslide segmentation results of the three models in the final predicting stages. The former pay attention are similar, and we can see that the area predicted by to extract the features with high resolution, with 10 meDeeplabv3+ is bigger but coarse. This is a representa- ters/pixel in our experiments while the latter is used to tive example that Deeplabv3+ tend to have a high recall capture the main features with a large number of convobut not precision. In the second data, although the land- lution layers. Besides, to make the model more adaptable slide morphology cannot be seen from the RGB image, for the dataset, we reduce the number of downsampling all three models can detect landslide. This means besides layers and add more skip-connection and use the SMU from the RGB bands (B4, B3, B2), which can be observed activation instead of the Relu. To evaluate the perforby humans, other bands also make contribution to the mance quantitatively, our model is compared with U-Net landslide detection. The third data is a very complex and Deeplabv3+ in experiments. The results show that landslide scenario. It is showed that the three models can Multispectral U-Net have the significant improvement in detect the landslide successfully, but Multispectral U-Net the recall metrics, with 75.41 and 2.87 absolute percent has better performance in some details. greater than the second place, and the f1 score of our model is also the best.
Overall, this study demonstrated the potential usage of multi-spectral satellite imagery in landslide detection.
and achieve a better performance compared with the traditional deep learning models.