<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Optimization of CNN and LSTM Based Application on RC Frame and Long-Span Structural Health Monitoring</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Rui Du</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Zhejiang University of Science and Technology</institution>
          ,
          <addr-line>318 Liuhe Road, Xihu District, Hangzhou, 310000</addr-line>
          ,
          <country country="CN">China</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>As an important field of artificial intelligence, neural network combines computer vision and image processing to extract the surface features, displacement and other parameters of the object to be measured. This paper mainly studies the optimization of CNN and LSTM based application on concrete frame and large-span system in structural health monitoring covered with concrete surface defects and possible damage prediction. The detection techniques including CNN, FCN and LSTM neural networks.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Convolution Neural Network</kwd>
        <kwd>Recurrent Neural Network</kwd>
        <kwd>Chaos Theory</kwd>
        <kwd>Frame and Largespan Structural</kwd>
        <kwd>Structural Health Monitoring</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Based on the physical mechanism of network topology knowledge, artificial neural network is
distributed and transmitted through a large number of neurons [1]. Their common characteristics are
large-scale parallel processing, distributed storage, elastic topology, high redundancy and nonlinear
operation. Therefore, it has high operation speed, strong association ability, strong adaptability, strong
fault tolerance and self-organization ability. Since it was first applied to the research field of civil
engineering in 1989, neural network has been involved in geotechnical engineering, building
construction, traffic engineering and other fields to deal with damage assessment, system identification
and optimization, structural control and regression analysis.</p>
    </sec>
    <sec id="sec-2">
      <title>2. BP Neural Network</title>
    </sec>
    <sec id="sec-3">
      <title>2.1. Application of</title>
    </sec>
    <sec id="sec-4">
      <title>Identification</title>
      <p>In 1986, Rumelhart and McClelland proposed BP neural network [2]. The characteristic of BP neural
network is the correction of error back propagation. For the specific principle, please refer to
"combining artificial neural network with principal component analysis and cross validation technology
to predict the compressive strength of high performance concrete [3]”.</p>
      <p>BP</p>
    </sec>
    <sec id="sec-5">
      <title>Neural</title>
    </sec>
    <sec id="sec-6">
      <title>Network in</title>
    </sec>
    <sec id="sec-7">
      <title>Spatial</title>
    </sec>
    <sec id="sec-8">
      <title>Lattice</title>
    </sec>
    <sec id="sec-9">
      <title>Structure</title>
      <p>In the existing SHM [4] structural health detection system, the modal parameters of spatial grid
structure are greatly affected by the environment. At the same time, the lack of research on local damage
and sensor layout makes the health status of the whole grid structure unable to get a good response.
Changes in material properties may also result in the inability to establish an objective and applicable
evaluation system in monitoring.</p>
    </sec>
    <sec id="sec-10">
      <title>2.2. Identification Principle and Development Direction</title>
      <p>The identification uses the grid structure finite element analysis method to release the nodes and
bending degrees of freedom at both ends of the element member, and only limit the translational degrees
of freedom in three directions. The shear wall and floor are simulated by shell element, and the beam
Copyright © 2022 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
and column are simulated by element frame with the help of MATLAB and BP neural network, the
nonlinear relationship between finite element displacement and overall structural stability is reflected.</p>
      <p>The structural health assessment system is divided into four levels. The first level is structural health
assessment. The second system layer is divided into two parts: building structure and elements. The
third layer corresponds to the structural position, stress ratio and damage degree in the building
components. Then data transmissions to the indicator layer to quantify the function and status of the
third layer.</p>
      <p>In the component system the weight is related to the ‘comprehensive importance’ of the elements
which represents the consequence of RC components failure and the possibility of damage set as IGi:
= =σi(q)</p>
      <p>Ri refers to the response of the ‘i’th element under the most dangerous working condition, Rmax is
the maximum value of all elements responses, σi is the stress response of the ‘i’th element, and ‘q’ is
the most disadvantage load combination of the elements. Index ‘I’ greater than 0.7 refers to the
‘important components’. When calculating the weight coefficient of the index ‘I’, the weight method is
used that wi is the weight coefficient of the ‘i’th component and ‘m’ is the amount of components.
=∑</p>
      <p>=1  </p>
      <p>The structure system includes the establishment of weight coefficient and the structure
comprehensive evaluation method based on fuzzy inference theory. The establishment of weight
coefficient selects the 9/9～9/1 scale method in Table 1, and the function model f(x,y) is used to
determine the importance of index X and Y of the evaluation objective [5].
(3)</p>
      <p>Equal importance
Partially importance
Obviously importance</p>
      <p>Very importance
Extremely importance
1
3
5
7
9
9/9
9/7
9/5
9/3
9/1
10/10～18/2
10/10(1.000)
12/8(1.500)
14/6(2.333)
16/4(4.000)
18/2(9.000)</p>
      <p>Index
90(1.000)
91/9(1.000)
93/9(1.000)
96/9(1.000)
91/9(1.000)</p>
      <p>The fuzzy comprehensive evaluation includes the establishment of membership function and the
centralized analysis of fuzzy evaluation vector. The function relationship between membership degree
R and index value I is presented by trapezoidal and approximate triangular distribution. For the
centralized analysis of fuzzy evaluation vector, the fuzzy evaluation vector is</p>
      <p>The health status grade of the structure is obtained from the index grade classification table
corresponding to Iz (Table 2).</p>
      <p>Using the test set to evaluate the accuracy of neural network, the identification accuracy of BP Neural
Network is relatively high, and the minimum classification accuracy error of neural network structure
in the test is 4.3%, which is the theoretical basis for realizing accurate evaluation of automatic
identification. Through the training of BP Neural Network, the increase number of layers and nodes
Serial Number
1
2
3
4
Index Range</p>
      <p>1.00≤ IZ
0.95≤ IZ &lt;0.95
0.90≤ IZ &lt;0.95</p>
      <p>IZ &lt;0.90
show great impact on the evaluation accuracy of neural network, the identification accuracy of neural
network will be efficiently enhanced with the increase of layers and the number of unit nodes. Further
research probably focus on the impact indicator types of existing evaluation models, experimental
design models, optimal design of sensors and life prediction of lattice structures.</p>
    </sec>
    <sec id="sec-11">
      <title>3. Convolution Neural Network(CNN)</title>
      <p>The convolution idea of Convolution Neural Network comes from BP neural network. As a
onedimensional fully connected structure, BP neural network is prone to problems such as local minimum,
slow convergence and weak generalization ability. Compared with the method of updating weights
through back propagation of BP neural network, CNN adopts a multi- perception structure with local
connection between layers and two-dimensional convolution template to share weights to reduce the
parameter scale and the probability of over fitting.</p>
      <p>The defects of CNN include that when the network layer is too deep, the parameters near the input
layer will change slowly by using BP propagation to modify the parameters; The gradient descent
algorithm is easy to make the training results converge to the local minimum rather than the global
minimum; The pooling layer will lose a lot of valuable information and ignore the correlation between
the local and the whole. There are some classic CNN models below.</p>
    </sec>
    <sec id="sec-12">
      <title>3.1. AlexNet Migration Learning Network</title>
    </sec>
    <sec id="sec-13">
      <title>3.1.1. Overview and Related Previous Research</title>
      <p>In the neural network structure, the transfer learning network can reuse the model developed for one
task in another target as the starting point, which saves a lot of computing and time resources required
for training the neural network (Fig. 1).</p>
      <p>The advantage of the AlexNet including calculation acceleration powered by CPU(GTX 580 3GB);
the concept of pooled step size is proposed to reduce the error rate; ‘Dropout’ technique is proposed to
reduce over-fitting and increased independence between neurons; in view of the oscillation of
convergence speed, Alexnet adopts BN normalization method to reduce internal Covariant Shift to a
certain extent. However, in the training process of Alexnet, the convolution kernel with large size is
used, which has too many parameters and is prone to be over-fitting. Moreover, Alexnet has high
requirements for storage and computing time, so it is difficult to deploy it on an appropriate GPU.</p>
    </sec>
    <sec id="sec-14">
      <title>3.1.2. Application of AlexNet in RC Frame Structure Identification</title>
      <p>The damage identification of reinforced concrete frame structure based on convolution neural
network includes the collection of damage data of frame structure, the damage identification based on
improved transfer learning network and the visual interface of damage frame structure identification.
The alexnet network model is trained by the feature extractor [6], and does not need to be trained many
times from top to bottom through a large number of convolution layers. However, it still takes a long
time to extract features by pretraining the filter parameters and weights of CNN layer.</p>
      <p>Finding sensitive damage index is the key link of identification. The natural frequency can reflect
the overall characteristics of the structure and has high test accuracy, but the identification of symmetry
problems is weak, which is more suitable to judge whether the structure is damaged. The input
parameters of the network are mostly discrete, and stress redistribution occurs near the damage location.
The test accuracy and sensitivity of low-order vibration modes and strain modes are relatively high. In
the damage feature extraction stage, the structural damage model is established by reducing the elastic
modulus of components (Fig. 2), and the natural frequency, vibration mode and strain mode parameters
are extracted.
()
()</p>
    </sec>
    <sec id="sec-15">
      <title>3.2.1. Overview of joint CNN &amp; LSTM</title>
      <p>CNN &amp; LSTM regarded as a neural network is able to uniformly extract the spatial and temporal
features of bridge signal data [8]. By comparison, CNN has weak damage degree identification ability
while LSTM and MLP is vulnerable in position identification ability and damage degree identification
ability. CNN &amp; LSTM which can extract the spatial and temporal information of damage signals at the
same time, has a good prospect in the application of bridge damage identification.</p>
      <p>The selection of loss function, classifier and optimizer of joint CNN &amp; LSTM has a great impact on
the actual performance of the network [9]. The loss function is the main index used to measure the error
between the prediction and correct result of neural network model. The commonly used loss functions
are Cross Entropy, CE and Mean Squared errors, MSE [10]. Cross Entropy is often used to calculate
the loss of network output z1 and real label z of classification problems LCE(z,z1):
LCE(z1,z)=-∑ [
1 + (1 −  )
(1 −  1)]</p>
      <p>MSE loss function is often used to calculate the loss of network output z1 and real label z of
regression problem LMSE(z,z1):</p>
      <p>LMSE(z,z1)=

1 ∑ ( −  1)</p>
      <p>Classifier is used for mapping the input feature vector to the predicted category label [11]. Compared
with SVM, Softmax classifier is the generalization of logic regression model in multi-classification
problems and has stronger classification ability.</p>
    </sec>
    <sec id="sec-16">
      <title>3.2.2. Application of CNN&amp;LSTM in Bridge damage identification</title>
      <p>Problems for solution: The main methods of bridge damage identification contains CNN, RNN,
MLP. Present problems to solution including (1)Deep neural network can carry out ‘end-to-end’
learning, it does not need feature index extraction and directly uses the original data for training.
However, the ‘end-to-end’ learning method performs poorly [12]in bridge damage identification.
(2)Because the performance of different neural network models in bridge damage identification is also
quite different [13]. Therefore, it is particularly important to establish a depth neural network model
suitable for bridge damage identification. (3)When building bridge damage samples, the training sample
data extracted from the simplified finite element model is different from the corresponding damage of
the actual bridge to a certain extent, that is, the training set and the test set obey different data
distribution laws [14] which will inevitably affect the final result of damage identification.</p>
      <p>Establishment of bridge damage sample database: The establishment of bridge damage sample
database includes the establishment of bridge finite element model, obtaining accelerated response data
and establishing damage sample database. The finite element model is designed as the plain jolter model
of OPENSEES [15] cable-stayed bridge. In order to more accurately simulate the damage of the bridge
under vibration excitation, the data such as the quality of the main beam, the elastic modulus and initial
tension of the cable, the stiffness of the support, the elastic modulus and unit weight of the concrete are
corrected. The acceleration measuring points are arranged according to the vulnerable position of the
structure [16].</p>
      <p>The damage analysis data include the instantaneous vibration frequency, average vibration
frequency, instantaneous energy, average energy, energy density and combination characteristics of the
bridge, which need to be obtained by decomposition and transformation of the acceleration response of
the bridge. Compared with EMD, CEEMDAN effectively solves the problem of "mode aliasing" of
EMD [17]. CEEMDAN adds adaptive Gaussian white noise in each section, and obtains each modal
component IFM by calculating the only residual signal. The obtained IMF signal forms an analytic
signal with the original signal s (t) through Hilbert transformation [18], the instantaneous amplitude and
instantaneous frequency are obtained through derivation and obtains the establishment of bridge finite
element model, obtaining accelerated response data and establishing damage sample database [12].</p>
      <p>Damage identification of joint CNN &amp; LSTM: The damage identification of joint CNN &amp; LSTM
includes the input, training and testing of joint CNN &amp; LSTM. The physical meaning of the input data
of CNN and LSTM is in Table 3.</p>
      <p>Iterative inputting it into the built CNN &amp; LSTM to get the recognition result of the corresponding
position. The error between the predicted damage identification result and the theoretical damage
identification result is calculated by Cross Entropy. The appropriate network parameters are obtained
by error back propagation and update the whole network. The verification set is used to test the network
performance, and a relatively suitable joint CNN &amp; LSTM model for bridge damage identification is
obtained.</p>
      <p>The test stage of joint CNN &amp; LSTM shows that the damage location and damage degree
identification effect of the combined feature sample library is the best, which can better reflect the
damage of the finite element model of cable-stayed bridge [12].</p>
      <p>The bridge damage identification combined with CNN &amp; LSTM has defects. The damage accuracy
of the measured damage sample library is slightly lower than that of the theoretical finite element
sample library, which may be caused by model simplification error and measured data error. Future
research directions may include obtaining more indicators that can effectively describe bridge damage
characteristics based on bridge vibration frequency and energy information, such as GAF(Gramian
Angular Fields) and MTF(Markov Transition Fields), and enrich the sample database of different types
of bridge damage.</p>
    </sec>
    <sec id="sec-17">
      <title>4. Full Convolution Neural Network (FCN)</title>
    </sec>
    <sec id="sec-18">
      <title>4.1. Features of Full Convolution Neural Network</title>
      <p>FCN extends the end-to-end convolution neural network to semantic segmentation for the first time
which is the basis of later neural network structure such as U-Net. The traditional CNN dimension
reduction processing method has several disadvantages: the memory overhead is inefficiently large with
the repeated adjacent pixel blocks; third, the size of the pixel block limits the size of the sensing area.
FCN is capable to accept images of any scale, solves the semantic level image segmentation; converts
the full connection layer into the deconvolution layer, recoveries the category of each pixel from the
abstract features (Fig.3). FCN neural network also has a skip layer structure, the first few layers
highlight more local details of the image, and the last layer contains more information of the original
image (Fig.4).</p>
      <p>However, FCN is improved based on CNN, the applicable spatial regulation in the segmentation
method based on pixel classification is ignored, which lacks spatial consistency.At present, the research
mainly focuses on the use of image processing technology to detect surface cracks and corrosion, for
example the successful application of the bridge coating quality evaluation. However, there are
relatively few studies on surface defects such as honeycomb, pitted surface, bubble, defect, corner
falling, and faulting of slab ends.</p>
    </sec>
    <sec id="sec-19">
      <title>4.2. Application of Full Convolution Network in Fracture Identification</title>
    </sec>
    <sec id="sec-20">
      <title>4.2.1. Previous Studies and Existing Problems:</title>
      <p>The depth learning methods for crack identification can be divided into sliding window algorithm
in image input stage and image segmentation method in crack feature extraction stage. In the traditional
sliding window algorithm, the use of full connection layer leads to that the input image size is only
limited to the design size of sliding window, and the batch recurrent input of a large number of windows
is very time-consuming. At this stage, full convolution neural network structure(FCN) is used to
transform the full connection layer at the end of CNN into the deconvolution layer to accept input
images of any size.</p>
      <p>In the stage of cracks feature identification, the threshold calculated by Ostu algorithm is higher than
the actual one, resulting in the extracted cracks wider than the actual one. The noises are difficult to be
removed by mathematical morphological image processing (open operation and close operation) or
judgment of connected isolated noise points. The proportion of narrow and long crack area in the whole
image is normally much lower relative to the background. The imbalance of samples will cause the
model training processing allocating more attention to the training of negative samples and partly
ignoring the positive samples, affect the detection effect.</p>
    </sec>
    <sec id="sec-21">
      <title>4.2.2. Improvement Method based on Traditional Neural Network</title>
      <p>Threshold Segmentation Method based on Improved Ostu Algorithm: Neural network crack
recognition process includes: image acquisition, preprocessing and enhancement technology; image
threshold segmentation based on Ostu algorithm, image cleaning and edge thinning processing; filter
the output image properties through the classifier, including length, width, area, perimeter, angle etc.</p>
      <p>In the image preprocessing stage, M2GLD algorithm (min-max gray level discrimination) is
introduced to improve the gray intensity of potential non cracked pixels and reduce the gray intensity
of potential cracked pixels. Meanwhile, M2GLD is able to convert the obvious single peak histogram
into a more separable bimodal histogram which is helpful to determine the optimal threshold by using
Ostu method. After image preprocessing and enhancement, Ostu algorithm is used to solve the image
segmentation threshold. Firstly, the overall average value of gray level is calculated:
μ=ω0(t)μ0(t)+ω1(t)μ1(t). The threshold top is calculated by using the optimization function:
 ()=ω()(μ()−μ)+ω()(μ()−μ)
()</p>
      <p>The formula on the right side of fs(t) represents the Ostu value between the target and the
background. However, in the case of single peak histogram and close to single peak histogram, this
method probably encounter the difficulty of threshold recognition. The target of the image cleaning
stage is to segment the image with pixels less than NP, and judge by the shape of the segment
images[19].</p>
      <p>The improved Ostu algorithm based on M2GLD can be easily integrated into many crack detection
and classification models developed in the future. The limitation of this method is that the user should
have to fine-tune two parameters: Adjustment Ratio and Margin Parameter. Therefore, future research
directions may include the application of optimization methods to automatically identify the appropriate
adjustment ratio and margin parameters.</p>
      <p>Crack Feature Extraction Method based on Progressive Cascade Convolution Neural
Network: In view of the fact that the characteristics of crack region are not significant compared with
the background region in the process of crack identification, the progressive cascade convolution neural
network is proposed for concrete surface crack identification. The method contains two stages. In the
first stage, a full convolution neural network is designed. The sliding window algorithm is used to
intensively scan the image, exclude most of the non crack regions, and take the windows which
containing cracks as the region of interest. In the second stage, a lightweight U-net image segmentation
network is designed to extract cracks from the region of interest which output in the first stage.</p>
      <p>In the first stage, each image is intensively scanned and segmented by setting the pixels and steps of
the training sample and test sample. In the image scanning process, compared with the convolution
neural network calculation of multiple sliding window images alone, the full convolution neural
network can input images of any size, which reduces the repetitive calculation of many windows
overlapping parts. The output probability P of the classification model is judged according to the
threshold TP, and the recommended value is 0.9 ~ 0.999:</p>
      <p>f(x) = {01,,  ≤&gt;   ()</p>
      <p>In the second stage, U-net uses the coding idea and jump layer connection to superimpose and fuse
the same scale feature images of the network shallow encoder and deep decoder, so as to make the
image segmentation effect more fine and accurate. U-net only extracts the region of interest with
obvious cracks. In view of the narrow and long crack characteristics, which are more sensitive to the
loss of information, the structural design replaces the pool layer operation by adjusting the step size;
Aiming at the characteristics that image cracks mainly contain edge texture properties, and there are
not many high-level semantic features compared with some complex feature objects, a lightweight
Unet neural network is used to improve the computational efficiency.</p>
      <p>This method can improve the performance of concrete crack identification. For the progressive
cascade convolution structure, the next research focus is probably to explore that if only constructing
one full convolution neural network can complete the screening of crack regions of interest and the
segmentation of cracks in the regions of interest at the meantime.</p>
    </sec>
    <sec id="sec-22">
      <title>4.3. Application of Full Convolution Neural Network in Steel Corrosion</title>
    </sec>
    <sec id="sec-23">
      <title>Identification</title>
    </sec>
    <sec id="sec-24">
      <title>4.3.1. Previous Studies and Existing Problems:</title>
      <p>Identification of reinforcement corrosion degree based on neural network was first proposed by long
j et al. Whom in the research attempted to put forward the convolution network neural structure. Lee et
al. Developed an automatic processor that can identify the corrosion defects of bridge coating [10], Son
et al. Used J48 decision tree algorithm to quickly and accurately determine the corrosion area [9]; Shen
et al. Proposed a identification method of reinforcement corrosion strength based on artificial neural
network [4]. Garcia Garcia et al. Analyzed and summarized the common semantic segmentation
network structures so far, and introduced the more successful segmentation networks such as Segnet,
Deeplab, Crfasrnn, etc. [12].</p>
      <p>With the detection of traditional image processing and identification technology, steel corrosion is
easily affected by noise such as illumination and shadow and the combination of several different
surface defects in surface color identification.</p>
    </sec>
    <sec id="sec-25">
      <title>4.3.2. Identification Principle</title>
      <p>Steel corrosion identification also has high requirements for image color identification, including
the treatment of the effects of lighting, shadow and the combination of several different surface defects
in the real environment. In the preprocessing stage, the image is converted from RGB color gamut to
YCbCr color gamut. In feature extraction process, the first layer of the hidden layer is set as the
convolution layer, and the input image is convoluted and symmetrically filled with the image edge, in
order to ensure that the image edge is included in the convolution process and avoid the premature loss
of the information at the image edge. Aiming at the defects that the various size of the feature images
after convolution operation, the L2 regularization function [4] is applied to batch processing of the
image, and then the Rectified Linear Unit(Relu) is used to the function activation.</p>
      <p>The improvement based on deep learning neural network is mainly to preprocess the image in the
early stage, including image denoising by using median filtering method, edge detection method based
on wavelet decomposition and image feature location method based on genetic algorithm. The training
accuracy reaches up to 91%, and the testing accuracy can also reach 85% through this methods. When
the algorithm is applied to the identification of plain carbon steel corrosion images, the identification
training accuracy and testing accuracy could reach more than 94% [4]. However, it usually takes more
than three years to generate a stable rust layer under actual environmental conditions. Due to time
reasons, the sample richness of reinforcement corrosion image sets is insufficient, and the accuracy of
reinforcement corrosion identification results is affected.</p>
    </sec>
    <sec id="sec-26">
      <title>5. Recurrent Neural Network in Deflections Prediction</title>
    </sec>
    <sec id="sec-27">
      <title>5.1. Application of LSTM Network in Deflections Prediction</title>
      <p>Long short-term memory (LSTM) is a special kind of RNN, which is used to predict the future
deflection values and reflects the changing trend of bridge structures. Compared with RNN with only
one transmission state, LSTM has two transmission states: cell state and hidden state. It solves the
problems of the gradient exploration and vanishing gradient descent problem in traditional RNN.</p>
    </sec>
    <sec id="sec-28">
      <title>5.1.1. Identification Principle</title>
      <p>The LSTM networks includes three layers, input layer, LSTM layer and regression layer
respectively. X represents the extracted features worked as input data samples while Y indicates the
deflection data as the output data trained by back-propagation with gradient descent.</p>
      <p>The hidden unit called memory cell as the core of LSTM neural network to memorize the
information of long-term input samples[9]. It includes three phases, ‘Forgot Gate’ is mainly used to
selectively forget the input from the previous node, control the ct-1 of the previous state through the
calculated z f, and determine the information to be retained. ‘Input Gate’ is mainly used to selectively
memorize the input data x t, the current input is represented as z, the selected gating signal is controlled
by z i. Add the results obtained in the above two steps to transfer to the next state c t. ‘Output Gate’
controlled by z o determines current output data and scaling data co from the previous stage through
activation function ‘tanh’ (Fig.5).</p>
    </sec>
    <sec id="sec-29">
      <title>5.1.2. Bridge Structure Deflections Prediction based on LSTM Network</title>
      <p>In bridge structure, the characteristics of grider deflection is selected as the reference reflecting the
bridge health condition. The deflection can timely reflect the bridge structure condition under the
moving load and external environment, according to which, it has strong robustness to noise and
sensitiveness to structure damage.</p>
      <p>The input references including the temperature, crack, humidity, strain and deflection. A 3-layer
LSTM model is trained with the total number of input and output vectors which is more than 8000,
while the dimension of cells state and hidden state is set to be 512. The time step, batch size, learning
rate is setting as 3, 128, 0.00044, respectively, to balance the performance and training time. In the
output stage, in order to minimize the mean square error (MSE), back propagation with gradient descent
is used to train the weight metrics. The calculation of MSE as follow:</p>
      <p>MSE=1 ∑

 =1(
−  )</p>
      <p>At the best predictive point, the proportion of error data is less than 1%, and 74% of the data in
worst-performing points</p>
      <p>meet this accuracy requirement[9]. Therefore, LSTM is available in
deflections prediction in bridge health monitoring.
()</p>
    </sec>
    <sec id="sec-30">
      <title>6. Existing Problems and Research Prospects</title>
      <p>Neural network has the following problems in civil engineering research: (1) the generalization
ability of neural network to identify building surface defect samples and structural deflection is weak.
(2) the calculation of curvature mode and strain mode is basically based on the assumption of elastic
deformation, which has limitations. (3) Neural network structure is difficult to identify building surface
defects and structural damage at the same time and establish the mapping relationship between them,
because too many parameters are easy to lead to over fitting. (4) RNN and LSTM recurrent neural
networks lack the research model of multivariable time series prediction in chaotic system. It is
necessary to strengthen the combined application of fuzzy theory and neural network in chaotic
structure analysis and prediction.</p>
    </sec>
    <sec id="sec-31">
      <title>7. Conclusions</title>
      <p>This paper studies the optimization CNN and LSTM based application on health monitoring of
frames and long-span systems. Alexnet model is introduced through BP neural network, and the damage
identification accuracy of RC frame structure is improved in combination with SGD and Adam
algorithm. R-cnn and CNN &amp; LSTM are used for bridge and road disease and structural damage
identification respectively. M2GLD and Ostu algorithm are combined with progressive cascade
convolutional neural network to detect building surface defects. In the future, neural network can
combine fuzzy and chaos theory to identify three-dimensional space model, establish the stress-strain
relationship between surface defects and structural damage degree, and establish time series damage
prediction through time recursive neural network.</p>
    </sec>
    <sec id="sec-32">
      <title>8. References</title>
      <p>2021.
[1] Xiao Guo, Kairui Yang, Haowei Jia, Zhengwu Tao, Mo Xu, Baozhu Dong, and Lei Liu. A New</p>
      <p>Method of Central Axis Extracting for Pore Network Modeling in Rock Engineering, Hindawi,
[2] Zheng Wuyuan, Jun Zhang. Feature extraction and image retrieval based on AlexNet, Eighth
International Conference on Digital Image Processing, 2016, 08.</p>
      <p>Structures, Multidisciplinary Digital Publishing Institute, 2017.
[3] Gangbing Song, Chuji Wang, andBo</p>
      <p>Wang. Structural Health Monitoring (SHM) of Civil
[4] Shuang Chen, "Image Recognition of Weathering Bridge Steel Corrosion based on Deep Neural</p>
      <p>Network", Master's Thesis of Beijing Jiaotong University, 2020.
[5] [5] Mohammed Majeed Hameed, Mohamed Khalid AlOmar, Wajdi Jaber Baniya &amp; Mohammed
Abdulhakim AlSaadi, Incorporation of artificial neural network with principal component analysis
and cross-validation technique to predict high-performance concrete compressive strength, Asian
Journal of Civil Engineering volume,2021, 22: 1019-1031.</p>
      <p>thesis of Southwest Jiaotong University, 2020.
[6] Tan Kangxi, Research on bridge damage identification combined with CNN &amp; LSTM, master's
[7] Wan Lei, Tong Xin, Sheng Mingwei, Qin Hongde, Tang Songqi. A review on the application of
classifier deep learning image classification methods. Navigation and control, 2019,18 (06): 1-9 +
47.
[8] Deng Jianguo, Zhang Sulan, Zhang Jifu, Xun Yaling, Liu ailing, research on loss function and
application in supervised learning. Big data, 2020,6 (01): 60-80.
[9] H. Son, N. Hwang, C. Kim, and C. Kim,”Rapid and automated determination of rusted surface
areas of a steel bridge for robotic maintenance systems,” Automation in Construction, 2014, 42:
13-24.
[10] B.Y. Lee, Y.Y. Kim, S.-T. Yi, and J.-K. Kim,”Automated image processing technique for detecting
and analysing concrete surface cracks,” Structure and Infrastructure Engineering, 2013, 9(6):
567577.
[11] Oh Shu Lih, Ng Eddie Y K, Tan Ru San, Acharya U Rajendra. Automated diagnosis of arrhythmia
using combination of CNN and LSTM techniques with variable length heart beats. Computers in
biology and medicine, 2018, 102.
[12] Gang Yao, Fujia Wei, Yang Yang and Yujia Sun,”Deep-learning-based Bughole Detection for</p>
      <p>Concrete Surface Image,” Hindawi, 2019.
[13] Shan Deshan, Sun Songsong, Huang Zhen, Lv Tiande, Li Qiao, Finite element model modification
of suspension and tension composite model bridge based on experimental data. Journal of Civil
Engineering, 2014,47 (10): 88-95.
[14] Ya Nan Yue, "Research on Damage Identification of RC Frame Structure based on Neural</p>
      <p>Network", Master's Thesis of Harbin Institute Of Technology, 2020.
[15] Ehsan Amirian, Eugene Fedutenko, Chaodong Yang. Artificial Neural Network Modeling and</p>
      <p>Forecasting of Oil Reservoir Performance. Springer International Publishing, 2012-10-05.
[16] Mcclelland, James L, Rumelhart, et. Paeallel Distributed Processing, Explorations in the micro
structure of Cognition, Volume 2: Psychological and Biological Model. Massachusetts Institute of
Technology Press, 1986.
[17] Xu Peng. Structural health monitoring based on deep learning. Jinan University, 2017.
[18] Hu Ze, Zhang Zhibo, Wang Xiaojie, Wu Yuchen, Xie Xinxin. Fault diagnosis of rolling bearing
based on Hilbert Huang transform and neural network. Electric tools, 2020 (01): 11-18.
[19] Li Feng, Lin Yangyang, Zhao Hui, Zhao Sunquan, Wang Hao. Research on fault diagnosis method
of hydraulic pump based on ceemdan-snm. Hydraulic and pneumatic, 2016 (01): 125-129.
[20] Shan Deshan, Zhou Xiaohang, Yang Jingchao, Li Qiao. Bridge seismic damage identification
combined with seismic vulnerability analysis. Vibration and impact, 2017,36 (16): 195-201.</p>
    </sec>
  </body>
  <back>
    <ref-list />
  </back>
</article>