A large number of landslide disasters occur every year in the world, causing a large number of casualties and huge economic losses, landslides are particularly common during mine operation. It is essential to predict landslides correctly. The traditional landslide prediction model has problems such as a low utilization rate of information between historical data, and the correlation between multi-variables cannot be well mined, resulting in insufficient prediction accuracy. This paper proposes a landslide prediction model (CNN-Attention-BiLSTM-Attention Model) based on the dual attention mechanism of the convolutional neural network and double-layer LSTM for multi-feature landslide data. The dual attention mechanism mines the information between variables better. The CNN-Attention-Bi-LSTMAttention Model is compared with RNN, double-layer LSTM, and single-layer LSTM based on attention mechanism to conduct the experimental comparison of landslide prediction model, and the results show that the RMSE values are increased by 9.84, 8.68 and 3.76, respectively, MAPE values were increased by 4.2266848%, 0.1880235% and 2.6510467%, respectively. The validity of the proposed model in landslide prediction is confirmed. The improvement of the prediction accuracy of the landslide displacement in the mining area has effectively helped us to realize the landslide stability monitoring and the safe and reliable operation of the mine.
Landslides are frequent and destructive natural disasters. Landslide geological disasters are reported worldwide, causing thousands of casualties and causing significant damage to human property and infrastructure. A landslide is a process of gradual movement due to the destruction of rocks, soils, sediments, etc., under the action of gravity, and finally caused by the disintegration of the total rock mass [1-2]. Due to its particular geographical environment, southwest China is a high-incidence area of landslides. The southwest mine area is located in the trough area where the Eurasian and Indian plates are strongly collided and squeezed. The geological activity is intense, with a series of deep and significant faults and secondary fractures. The frequent and frequent occurrence of geological disasters in the southwest region poses significant constraints, impacts the construction and operation of mine, and may cause substantial economic losses simultaneously. Landslides can cause surface deformation, destroy the environment around mine area, causing mine to fail, and also affect the safety of people's lives and property.
As the landslide fixed-point monitoring sensor equipment becomes more advanced, the monitored data becomes more accurate, and the types of landslide data that need to be measured become more and more abundant, and experts have more fully studied the topographic and geological conditions around the landslide. The field distribution location is also more thoughtful and scientific, and the corresponding points can be arranged according to the specific situation of the landslide. The monitoring of the landslide can be more comprehensive. Using deep learning-related technologies to conduct corresponding intelligent analysis of landslide data and establishing a landslide data prediction model is an excellent solution for processing a large amount of landslide data.
At present, landslide prediction research is based on machine learning and deep learning models. Zhao Hongbo[3] proposed an evolutionary support vector machine method. Through the continuous education of the machine learning model, the strength parameters of the landslide rock were continuously optimized to predict the landslide. Panahi et al. [4] used the support vector machine landslide regression model to predict landslides. Zhang Baiyi et al.[5] established a landslide deformation prediction model by using BP neural network, and carried out short-term prediction of landslides. Yue Qiang et al. [6] also used BP neural network to predict landslide displacement data. However, the landslide is a relatively complex process, and the BP neural network easily falls into the problem of local minimization, so scholars have gradually adopted the combination of complex prediction models. Lee et al. [7] used the ANN method to predict landslides. Caniani[8] also used ANN for landslide prediction and risk assessment. Song Liwei et al. [9] used LSTM to predict landslide displacement. This method solved the local minimization problem in the BP neural network well and the RNN prediction problem, but its prediction accuracy was still not very good.
The existing models mainly have the following problems: • RNN, due to its network structure, when the error is in the backpropagation process when the input time series accumulates too long, it will cause problems such as gradient disappearance and gradient explosion, so it cannot rely on long-term time series data.
• Due to the characteristics of LSTM structure, when inputting time series data, LSTM only uses the forward information of historical data in the process of information decoding, and the influence factors of time data after a certain time are not considered at that time. , so the temporal mutual information between the entire historical data cannot be fully considered.
• The data at a particular time in the historical data has a more profound correlation with the data at a closer time and the characteristics and change rules between the time data. LSTM cannot extract it well, and LSTM dramatically affects landslide prediction accuracy.
Based on the above, this paper takes the Limestone Mine’s landslide in Gulin County, Luzhou City, Sichuan Province in Southwest China as an example,analyzed the landslide monitoring data through various types of sensor data collected in the field, Convolutional Neural Network Double Layer LSTM Landslide Prediction Model (CNN-Attention-Bi-LSTM-Attention Model) is proposed. The attention mechanism of the model can well mine the characteristics and change rules between the landslide data time series. The double-layer LSTM solves the problem that the backward time information cannot be well used in the historical data in the single-layer LSTM.
The model proposed in this paper uses a convolutional neural network to extract the spatial features of the data, adds a layer of attention mechanism for the processing of different feature data, and proposes a double-layer LSTM multivariable landslide data prediction model based on the double-attention mechanism of CNN feature extraction. (CNN-Attention-Bi-LSTM-Attention), the model structure is shown in Figure 1.
It is worth noting that the two-layer attention mechanism, the first-stage Attention mechanism, extracts the primary trend information in the feature sequence. The second-stage Attention mechanism focuses on the hidden state of key time steps and selects the critical information in the long-term correlation. The information is possessed at a later time. In the CNN-Attention-Bi-LSTM-Attention hybrid neural network, the convolutional layer and the pooling layer are located in the front. The prediction work first extracted the hidden features in the displacement, anti-sliding pile deformation, pipeline strain, and rainfall vector. The information data contained in the hidden features can more effectively reflect the change law of the landslide displacement data, which is helpful in improving the learning efficiency and prediction accuracy of the model. Firstly, the multivariate historical data is extracted through CNN to extract the feature information between different variables. The attention mechanism layer is introduced into the CNN convolution layer and the pooling layer to extract the main trend information between the features, and then the feature information is obtained and then input. The information between time series is obtained in the double-layer LSTM, and then input to the second-layer attention mechanism to focus on the hidden state of key time steps, select the key information in the long-term correlation, and finally obtain the output result through the output layer. Prediction of final landslide displacement data.
Due to the characteristics of the landslide-related monitoring equipment and its placement in the mine area, the sensors are easily affected by the surrounding geological environment and manual operationor or due by sudden failure of the instrument, resulting in some outliers and missing values in the data. Data processing directly into the model for model training will significantly affect the model's accuracy, so processing missing values and outliers is crucial. The basic idea of processing adopted in this paper is to remove outliers correspondingly, treat them as missing values, and then fill in missing values together with the actual data of the data itself. In this paper, the least squares support vector machine[10] (Least Squares Support Veotor Maohine, LSSVM) in machine learning is used to fill in missing data. LSSVM avoids the quadratic programming problem in traditional support vector machines. Compared with SVM, LSSVM has To a certain extent, the speed of learning is improved. The following is an introduction to the principle of the LSSVM algorithm.
Let the sample set be{( xi , yi )}, i = 1,…, n where xi ∈ R d , yi ∈ R the essence of LSSVM is to solve the regression function, such as formula (1):
Then to solve the regression function is to solve the following optimization problem, such as formula (2)
After LSSVM transforms the constrained optimization problem into an unconstrained optimization problem, according to the KKT conditions, the solutions of b and an can be obtained as equation (3) To avoid finding H n−1 set H n−1l = γ , H n−1 yn = ε , because H n it is a symmetric positive definite matrix, use the Cholesky decomposition method to solve H nγ = l , H nε = yn so thatγ 、ε it can be obtained. The value in the regression function is formula (4): (4)
Use the data without missing values as the training sample Htrain to get the LSSVM model, and then use the LSSVM model to fill in the missing values.
The input data are deep displacement data,surface displacement data,anti-sliding pile deformation data, pipeline strain data, and rainfall data. The data is obtained through the convolution layer to get the convolution results to extract different features of the input data and then sampled through the pooling layer to bring new dimensions. Smaller features and local features are integrated through the fully connected layer to obtain the final global feature information. Among them, the feature extraction process of CNN is shown in Figure 2.
The input data is deep displacement data,surface displacement data,anti-sliding pile deformation data, pipeline strain data, and rainfall data. Different features are sampled through the pooling layer to obtain new features with smaller dimensions. Finally, the local features are integrated through the fully connected layer to get the final global feature information.
Bi-LSTM consists of two LSTM layers with two opposite directions[11]. The hidden layer Ht ncodes the information features in the forward direction, while the hidden layer Ht ncodes the information features backwards. The calculation formulas are as follows formula (5), (6) and (7), and then connect the two sets of forward and reverse hidden layers for output. Bi-LSTM can learn information more accurately by using forward and reverse encoding. Therefore, it is particularly It is suitable for processing long-term series data, and its network structure is shown in Figure 3. Ht = LSTM ( Ht−1, xt , ct−1 ) , t ∈ [1,T ] Ht = LSTM ( Ht+1, xt , ct+1 ) , t ∈ [T ,1]
Ht = Ht , Ht (5) (6) (7) (8) (9) where T is the sequence length.
The specific calculation process of the attention mechanism can be represented by Figure 4, which can be divided into three stages. In the first stage, the correlation between the query and the key is first calculated. There are many different calculation methods for the calculation of the correlation. The most common calculation methods are dot product, cosine similarity or introducing other neural networks. network, etc. Since the size of the similarity generated in the first stage will be different depending on the calculation method, there is no way to directly use the attention weight, so the Softmax function needs to be normalized, and its calculation formula is as shown in equation (8):
The third stage is to calculate the attention value, and the weight coefficient value ai obtained through the second stage calculation can be obtained by weighted summation, as shown in formula (9). ai = Softmax (Simi ) =
eSimi
The last part is the output layer. The fully connected layer performs feature dimension reduction and outputs it. The output layer Y =[y1,y2 , ym ] is responsible for calculating the output with a prediction step m. The specific calculation formula is shown in formula (10): yt = f ( wo st + bo ) (10)
In the formula, represents the weight matrix, represents the bias, and represents the load forecast value at time t.
The experimental site of this paper is selected from the landslide in a limestone mine located in Gulin County, LuZhou City, Sichuan Province. The mine is located in the southwest of Gulin County, about 12km away from the county seat. The landform in the area where the landslide is located belongs to a monoclinic low-medium mountain and narrow valley area. The slopes on both sides are relatively steep with a slope of 10-30°. The landslide is located at the lower part of the slope 15°~20°, the overall slope is stepped, and paddy fields and dry land are distributed.
The experiments in this paper use solar cell-powered sensors to perform the following four types of monitoring and collect their data:
• Carry out pipeline deformation monitoring on the places where the pipeline is deformed and subjected to the greatest stress, mainly monitoring the longitudinal strain of the pipeline through strain gauge sensors;
• Monitoring the position, displacement and displacement direction of horizontal displacement of surface measuring points; • Fix the inclinometer on the anti-sliding pile to monitor the inclination angle of the anti-sliding pile; • At the position with the largest potential deformation of the landslide body, lay multiple deep displacement monitoring holes at different elevations, and place sensors to obtain the vertical displacement deformation curve of the entire monitoring section; • Use rain gauges to monitor rainfall information in landslide areas.
The time span is from April,2021 to October,2022. Part of the data collected is shown in Table 1, and the displacement data is shown in Figure 5.
Using the proposed two-layer LSTM multivariate landslide data prediction model (CNN-AttentionBi-LSTM-Attention) based on the dual attention mechanism of CNN feature extraction, the multivariate historical data is extracted through CNN to extract features between different variables. First, the first layer of attention mechanism is used to extract the main trend information between features, and then the obtained feature information is input into the two-layer LSTM to obtain the relationship information between time series, and then input to the second layer of attention mechanism , the final predicted value of landslide displacement data is obtained through the output layer. The parameter settings of the entire convolutional neural network prediction model are shown in Table 2. Convolution Kernel Size Fully Connected Layer Neurons Learning Rate Training Size Per Batch Total Iteration Rounds Test Round
To illustrate the influence of input features on the landslide displacement prediction model, different variable combinations are used as model inputs. Model 1 only uses deep displacement data as input, models 2, 3, 4 and 5 add anti-sliding pile deformation data, pipeline strain data, rainfall data and surface displacement to model 1, respectively, and model 6 uses these five variable data as a model. Then, the experiment compares the landslide prediction effect of different input features, and the experimental results are shown in Table 3.
As can be seen from Table 3, after the four variables of anti-sliding pile deformation data, pipeline strain data,rainfall,surface displacemen have been added,the model has a certain degree of improvement. Among the six models used, model 6 has the best effect, so we decided to use these five features together as the input of the model to construct the CNN-Attention-Bi-LSTM-Attention model, and the obtained prediction results are shown in Figure 6,the predicted and actual values obtained are shown in Table 4. 2021/4/12 14:00
It can be seen from Figure 6 that the blue curve and the red curve in Figure 6 represent the actual value and the predicted value, respectively. The model is trained on the landslide displacement data, and the fitting data obtained can better reflect the change law of the actual value. From the analysis in Table 3, it can be seen that the difference between the predicted value and the actual value is always kept in a relatively stable interval, the minimum difference is 0.07498, the maximum difference is 3.3261, and the average difference is 1.07048.
In this paper, the root mean square error (RMSE) and the mean absolute percentage error (MAPE) are used as the evaluation indicators of the prediction model, and their calculation formulas are shown in equations (11) and (12).
MAPE y, y =
RMSE =
n yi − yi i=1
yi n i=1( yi − yˆi )2 n n ×100
Among them, yi presents the actual value, yˆi presents the predicted value, and n represents the number of values. The smaller the RMSE and MAPE values, the better the accuracy of the prediction model. In order to further verify the validity and accuracy of the double-layer LSTM landslide data prediction model based on the attention mechanism proposed in this paper, this chapter uses RNN, double-layer LSTM, and single-layer LSTM model based on the attention mechanism to predict the data and evaluate its performance. The parameters used in all models are optimal, and the prediction results are compared with the models proposed in this chapter. The comparison results obtained are shown in Figure 7. (11) (12)
It can be seen from the analysis in Figure 6 that the change trends of the prediction result curves of the four different models basically reflect the change law of the real value. close, the predicted result curve is more in line with the actual value curve, and the basic and actual value changes. The prediction result of RNN is relatively poor, and its prediction curve is farther away from the true value curve. Using the evaluation metrics described above, the prediction results of different models are calculated for performance evaluation. The values of RMSE and MAPE are shown in Table 5 and Figure 8 below.
Analysis of Table 5 and Figure 8 shows that among the four models, the CNN-Attention-Bi-LSTM model proposed in this paper has the highest accuracy, and the RMSE and MAPE values of its prediction results are 2.206741476 and 6.848253, respectively. Compared with the comparison Model CNN-Attention-LSTM model, Bi-LSTM model and RNN model, RMSE increased by 3.759028784, 8.677539835 and 9.8392235 respectively, MAPE increased by 2.6510467%, 0.1880235% and 4.2266848% respectively. Compared with the other three prediction models, the evaluation indicators of the CNN-Attention-Bi-LSTM prediction model have improved to a certain extent. The CNNAttention-Bi-LSTM prediction model has better prediction results. The CNN-Attention-Bi-LSTM prediction model has better fitting and regression ability and has certain practical application prospects.
Aiming at the complexity of landslide prediction model, this paper combines various types of monitoring data to improve the landslide prediction model, uses convolutional neural network to extract the characteristics of each variable data, and introduces the attention mechanism to better mine the information between variables. The convolutional neural network double-layer LSTM landslide prediction model (CNN-Attention-Bi-LSTM-Attention) based on the double attention mechanism improves the prediction accuracy of the displacement prediction model. In order to verify the effect of this model, we also compared the displacement prediction results with the landslide prediction model of RNN, double-layer LSTM, and attention-based single-layer LSTM, showing that the RMSE values were increased by 9.8392235, 8.677539835, and 3.759028784, respectively. MAPE The values are improved by 4.2266848%, 0.1880235% and 2.6510467%, respectively.
In short, compared with RNN, double-layer LSTM, and single-layer LSTM based on attention mechanism, CNN-Attention-Bi-LSTM-Attention solves the problems of other models to a certain extent, and improves the accuracy of displacement prediction models,which provides important data support for the realization of the landslide stability monitoring and the safe and reliable operation of mine.