Structural shape reconstruction of non-linear systems via Variational Autoencoders

Abstract

The application of Variational Auto-Encoders (VAE) for full-field displacement reconstruction of a general non-linear system is shown in this contribution to the framework of Structural Health Monitoring (SHM). The presented approach aims to combine sensor data with a Finite Element representation of the structure, addressing the problem of Model Order Reduction by using a Proper Orthogonal Decomposition approach. Physics knowledge of the system is embedded into the VAE model which is trained to correctly reproduce non-linearities in the system. The model is evaluated both from the reconstruction and prediction point of view in order to define possible limitation of the proposed approach as a tool for SHM.

-

1. Introduction ing (SHM) systems can improve the safety and reliability of aerospace and automotive structures during operational life by preventing structural components from catastrophic failure scenarios and identifying possible damages sooner than common manual inspection. The role of online parameters and load identification, together with stress field distribution, is nonetheless of paramount importance. With the advances in the development of predictive algorithms in the field of Artificial Intelligence tention in the field of SHM, by combining sensor data with the numerical model of the monitored structure.

The choice of installed sensors depends on the applica

tion. For example, Digital Image Correlation (DIC) and

Fiber Bragg Gratin (FBG) systems can be respectively em

ployed for shape reconstruction and critical stress paths identification. The challenge in SHM is to optimally deifne the map of such sensors which is able to retrieve not-measured sensor data. Inverse identification from a reduced set of measured data to full-field response is posnized by CINI, May 29–31, 2023, Pisa, Italy ∗Corresponding author.

Variational Auto-Encoders (VAE) [7] have been widely

used in literature to approach model order reduction problems [ 8, 9, 10, 11 ]. The structure of the AEs defined by an encoder, a latent space with a reduced order and a decoder, has several similarities with the principles of

ROM. The formulation in the latent space is user-defined.

If there is no knowledge of the dynamical system, the latent space is fully defined as a black-box model. In other hand, if there is perfect knowledge of the model, the latent space will be defined as a white-box model identified by the ROM of our system. In a real case scenario, black or white box modeling is never the case because the main dynamic of the system is assumed to be known from design and validation of the investigated structure. The (AI), physics-informed solutions have attracted great at- full-field response [ 4, 5, 6 ]. The main advantage of KF

2.1. State-space formulation of a structural dynamic system

The governing equation of motion of a dynamical system described by the mass matrix M, the stifness matrix K and the damping matrix D is: x = USV with U ∈ ℝ × and V ∈ ℝ × as left and right singular vectors respectively and S ∈ ℝ × as diagonal matrix containing the singular values on the main diagonal. The POD forms the reduction basis by selecting the first -left singular values ( < ) from the matrix U, such that the following minimization problem is reached:

2. Methodology

gives an advantage over KF methodologies, allowing for When non-linear systems are involved, standard ROM the estimation of missing physics in the FE model. approaches as Guyan [ 2 ] or Craig-Bampton [ 3 ] can not

In this contribution, an application of VAE to full-field be applied. The Proper Orthogonal Decomposition (POD) displacement estimation is presented by implementing [ 12 ] method is one of the most used approaches for nona sliding time-windowing approach. The use-case is a linear dynamics. POD identifies an optimal subspace non-linear cantilever beam and a Proper Orthogonal De- based on a set of snapshots, i.e. time series data, of the composition (POD) [ 12 ] reduction basis is adopted. The system solution space. In other words, the main prinarticle is structured as follows: the state-space formu- ciple of the POD is in the selection of the meaningful lation of a FE model, the POD approach, and VAE are shapes or modes of the solution provided by snapshots of described in Sec. 2; following, the use-case and the results e.g. displacements or velocities through a Singular Value obtained through VAE are shown in Sec. 3. Decomposition (SVD). Given the matrix of snapshots x ∈ ℝ × , with = number of time samples ( ≥ ), the SVD gives: (4) (5) (6) (7) Mẍ + Kx + Dẋ + f(x) = Bu (1) min ‖x − ‖2 Where x ∈ ℝ , ẋ ∈ ℝ , ẍ ∈ ℝ are respectively the Degrees of Freedom (DoFs) of the system with size and their first and second derivative with respect to time.

The time dependence of these variables has been omitted for simplicity. B and f(x) are respectively the Boolean matrix that distributes the inputs vector u over the DoFs of the system and a general nonlinear term.

By assuming f(x) = 0, Eq. 1 can be re-written in a state-space formulation as follows:

The number of selected values depends on the threshold

selected.

The POD has the disadvantage of requiring the snapshots matrix, which needs to be computed ofline using FOM simulations. However, as a main advantage, the POD reduction basis provides a higher accuracy for nonlinear and parametric models, as these characteristics are present in the solution space and captured via the snapshots. Furthermore, the selection of the meaningful singular values also has a physical interpretation on the ẋs = Axs + Bsu (2) solution. A good explanation can be found in [ 13 ]. If a { y = Cxs + Dsu parametric model is considered, other techniques need to be employed in order to have the POD basis working where ẋs = [x, ẋ] is the state vector, A and Bs are ob- for all the parameters samples, i.e. global reduction basis. tained from Eq.1. The second equation of the system The most common ways are by interpolating or concateif the measurement equations, where y ∈ ℝ is the nating the POD basis obtained for each sample. When measurements vector with size and C, Ds are matrix the number of samples is limited, the concatenation apdepending on the measurements’ type. proach is the easiest one. Given a set of concatenated basis, the SVD is recomputed in order to remove possible linear dependencies between all the subspaces, as follows:

2.2. Reduction basis: Proper Orthogonal

Decomposition approach for non-linear systems

SVD ([ 1 2 3 … ]) ROM can be required when the number of DoFs of a FEM is too large and, consequently, the computational cost is too high. The projection of the Full Order Model (FOM) to the ROM is defined by the following approximation: ẋ ≅ Φq (3) where Φ ∈ ℝ × is the reduction basis mapping DoFs to reduced DoFs q ∈ ℝ . where ż = [q, q̇] is the reduced state vector and all the matrices are in a reduced form.

Given thus as global reduction basis of the mechanical non-linear system, Eq. 2 projected in the reduced space can be re-written as:

ż = Arz + Bru { y = Crz + Dru

2.3. Variational Auto-Encoder for Model

Order Reduction

In this section, the principles of VAE addressing ROM are described and schematically shown in Fig. 1. The formulation used in this article is the same presented in [ 8 ] and regard the extension of the variational autoencoder model to their dynamic version. Specifically, by including a recurrent neural network into the standard fully-connected layer, such architecture can process time series data. So, starting from a subset of sensor data, the following module are used to process a multidimensional time-series data: • Encoder: data-driven, i.e. no physics of the sys- prior, while the second term encourages the VAE to retem considered at this stage. It is composed by construct the observed data accurately. Minimizing the two sub-networks: a Multilayer Perceptron (MLP) ELBO loss in a VAE corresponds to find the optimal tradeand Long Short Term Memory (LSTM) returning of between reconstruction accuracy and regularization respectively distributions of initial conditions at of the learned posterior distribution. Moreover, to get position q0 ∈ ℝ and velocity q̇0 ∈ ℝ level of a better reconstruction quality, the posterior ditribution the reduced model. The concatenation of q0 and can be explicitely derived in place of the standard mean q̇0 forms the initial state vector z0 ∈ ℝ2 squared loss. In this work, a normal distribution is used: • Latent Model: physics-informed, modelled as residual models, i.e. fbaseline(z) + fNN(z). From () = 1 exp (− ( − )̂ 2 ) (9) Eq. 7: fbaseline(z) = Arz. The non-linear term √2 2 2 is instead added fNN(z) to compensate for model where is a a time step of the input sequence, ̂ is truncation and for non-linearities (if any) as in the model output, is the standard deviation. This latter Eq. 1. In the latent space stage, the integration of term is a constant equal to 3 − 2 the state space model over time is performed. • Decoder: it represents the reduction basis Φ and can be trainable or not-trainable, e.g. pre- 3. Non-linear cantilever beam computed via POD. It allows to retrieve FOM data use-case from the latent space.

where KL((|)||()) is the Kullback-Leibler (KL) divergence between the learned posterior distribution and the prior distribution over the latent variables. The first term encourages the learned posterior to be close to the In table 1 is reported some details about the architecture. The object of the current study is a cantilever Euler

The Evidence Lower Bound (ELBO) loss is commonly Bernoulli beam, with an end non-linear grounded spring used with VAE as objective function to be minimized. (Fig. 2). This benchmark has been presented and studThis loss can be formulated as follows: ied in several contributions [ 14, 15, 16 ]. The model here is characterized by the following parameters: density ELBO = −KL((|)||()) + (|) [log (|) ] (8) se=cti2o7n0a0l/area 3=, c3r.2o3ss-s−e4c tio2,nsatlifninesesrtpiaar a=me4te.3r − 9 =4, 3 3 = 3 3 / , first natural frequency = 1 / , 4 = 2 4

, proportional damping factor = 1 −4.

The model has been discretized via 2D FE analysis with = 15 beam elements. The DoFs of the system are the vertical displacement in direction and rotation along − axis, for a total of = 2 . The system has been simulated via Euler implicit integrator in a time window of = 100 . The parameter , associated to the non-linear spring, is sampled in order to create the dataset for diferent values of non-linearities. The values of range in [1 −9, 9 −9] / and 5 uniformly distributed samples are selected. The beam is loaded at the tip with a sinusoidal force along − direction:

Diferentely from the contribution in [ 8 ], the global reduction basis Φ is computed through SVD by concatenating local reduction basis as in Eq. 6. Each local basis is ob- Figure 4: First three mode shapes of the POD basis. tained via POD which is applied to the snapshots of each solution in the range of parameter. The singular values after concatenation and the first three mode shapes are shown in Fig. 3 and Fig. 4. The mode shapes show section 3, a windowing approach is defined. Specifically, diferences with standard normal modes of a cantilever the temporal sequences of each DoF are split into several beam because of the efect of the non-linearities. A total temporal windows with a length defined a-priori. In this of 7 modes, i.e. = 7 is selected from SVD analysis. way, the dataset is augmented and the windowing allows This number results to be the optimal one by comparing the encoder to learn the model non-linearities and the reference with reconstructed time data. This reduction initial conditions of the latent model starting from the basis allows the model to capture all the non-linearities time-series of a subset of observations. presented in the simulated dataset. In [? ], the decoder Among the 5 simulated samples, 4 of these are used is defined by the reduction basis coming from eigenanal- for the training. Each sample describes the cantilever ysis of the linear model. That approach can however beam behaviour in a range of 10000 time steps. The size lead the model to be projected in a wrong reduction basis of the train set is thus equal to 40000 time steps. The and thus to a wrong reconstruction. The comparison windowing is performed with a fixed size of 200 time between the two approaches is out of the scope of this steps. The sample generated with a value of in the article. Anyway, the choice of using a POD is the most middle of the specified range of parameters identifies suitable for the analyzed system, according to theory in the validation set. A scheduler is also implemented such [ 12 ]. that if the validation loss doesn’t improve after 5 epochs, the learning rate decreases to the 80% of its previous 3.1. Displacement field estimation value. A batch size of 32 is used and the model reaches its optimal state after nearly 25 epochs. The reason why through VAE the loss converges after few epochs is justified by the high number of data that the model processes for each epoch. An early stopping of 20 epochs is used. The trend of the train and validation losses normalized with respect to the value of the training loss at the first epoch is shown in Fig. 5. The validation loss is always lower than the training loss because the validation set has been defined in the middle of the sampling parameter range. However, The scope of this work is to train a VAE to estimate the dynamical behavior of = 30 DoFs of the cantilever beam starting from a subset of DoFs. These limited number of measurements has been defined in order to guarantee the observability of the system and are the 7 measures marked in red in Fig. 2. To train such architecture on the data simulated following the process described in a more challenging validation set should be also tested in order to quantify the overall performances of the model. results are shown in Fig. 3.1 and Fig. 3.1 over a greater time horizon. For each window, the prediction coming from the previous window and the actual reconstruction are compared. This comparison is shown starting from time step 200, i.e. second window. From the reported results, the prediction is in some cases accurate. However, we observe that this estimation depends drastically from the input sequence and generally does not produce reliable estimation. A more accurate training could perhaps return more reliable prediction.

4. Conclusions

The results are reported for the sequences belonging to the validation set. The model is trained to reconstruct all The application of the Variational Auto-Encoder to nonthe DoFs of the system starting from a subset. In Fig. 6, linear dynamics is shown in this article. A non-linear are reported the results concerning both an un-measured cantilever beam is investigated by simulating the Finite and one measured DoFs. From these plots it is possible Element model with diferent values of non-linear paramto observe that the model is able to catch the dynamics of eters. The simulated dataset allows to form the reduction the signals. These DoFs have been chosen without loss basis by performing a snapshots-based Proper Orthogof generality on the overall performances. In Fig. 3.1 are onal Decomposition. This step leads to a pre-defined reported the result for the prediction task. In this latter decoder in the VAE architecture by reducing the numscenario, the model receives in input a sequence of 200 ber of total trainable parameters. However, a trainable time steps and predicts also the next 200 time steps. The decoder could give some advantages e.g. more adapt(b) unmeasured Dof able reduction basis with respect to training data. The latent model described as residual grey-box model of the system dynamics allows also for the identification of all the contributions that have not been implemented in the baseline model as e.g. external forces, parameters, and non-linearities. In a real test case, the available dataset is normally not enough for a deep learning application and thus the windowing approach has been tested in order to perform data augmentation. The analysis shown in this article led to accurate results in the estimation of un-measured data, starting from the knowledge of a subset of measurements. The structure of the presented approach with time windowing and a pre-trained model can be used for online structural monitoring applications.

References

[1] O. Zienkiewicz , R. Taylor , The Finite Element Method, Butterworth Heinemann , Oxford, 2000 .

[2] R. J. Guyan , Reduction of stifness and mass matrices , AIAA journal 3 ( 1965 ) 380 - 380 .

[3] R. R. Craig Jr, M. C. Bampton , Coupling of substructures for dynamic analyses , AIAA journal 6 ( 1968 ) 1313 - 1319 .

[4] R. Cumbo , T. Tamarozzi , K. Janssens , W. Desmet, Kalman-based load identification and full-field estimation analysis on industrial test case ., Mechanical Systems and Signal Processing 117 ( 2019 ) 771 - 785 .

[5] E. Risaliti , T. Tamarozzi , M. Vermaut , B. Cornelis , W. Desmet, Multibody model based estimation of multiple loads and strain field on a vehicle suspension system , Mechanical Systems and Signal Processing 123 ( 2019 ) 1 - 25 .

[6] C. E. Capalbo , D. De Gregoriis, T. Tamarozzi , H. Devriendt , F. Naets , G. Carbone , D. Mundo , Parameter, input and state estimation for linear structural dynamics using parametric model order reduction and augmented kalman filtering , Mechanical Systems and Signal Processing 185 ( 2023 ) 109799 .

[7] D. P. Kingma , M. Welling , Auto-encoding variational bayes , arXiv preprint arXiv:1312.6114 ( 2013 ).

[8] Z. Lai , W. Liu , X. Jian , K. Bacsa , L. Sun , E. Chatzi, Neural modal ordinary diferential equations: Integrating physics-based modeling with neural ordinary diferential equations for modeling highdimensional monitored structures, Data-Centric Engineering 3 ( 2022 ) e34 .

[9] T. Simpson , N. Dervilis , E. Chatzi, Machine learning approach to model order reduction of nonlinear systems via autoencoder and lstm networks , Journal of Engineering Mechanics 147 ( 2021 ) 04021061 .

[10] M. Banihashemi , F. and Weber , W. Lang , Model order reduction of building energy simulation models using a convolutional neural network autoencoder , Building and Environment 207 ( 2022 ) 108498 .

[11] M. Mrosek , C. Othmer , R. Radespiel , Variational autoencoders for model order reduction in vehicle aerodynamics , AIAA Aviation 2021 Forum ( 2021 ) 3049 .

[12] P. Benner , S. Gugercin , K. Willcox , A survey of projection-based model reduction methods for parametric dynamical systems , SIAM review 57 ( 2015 ) 483 - 531 .

[13] J. Weiss , A tutorial on the proper orthogonal decomposition , AIAA Aviation 2019 Forum ( 2019 ) 3333 .

[14] M. Gürgöze , On the eigenfrequencies of a cantilever beam with attached tip mass and a spring-mass system , Journal of Sound and Vibration 190 ( 1996 ) 149 - 162 .

[15] M. W. Sracic , M. S. Allen , Numerical continuation of periodic orbits for harmonically forced nonlinear systems , Civil Engineering Topics 4 ( 2011 ) 51 - 69 .

[16] M. S. Allen , M. W. Sracic , System identification of dynamic systems with cubic nonlinearities using linear time-periodic approximations , International Design Engineering Technical Conferences and Computers and Information in Engineering Conference 49019 ( 2009 ) 731 - 741 .