PCA-NuSVR Framework for Predicting Local and Global Indicators of Tunneling-induced Building Damage 1⋆

Abstract

Today, numerous construction projects aimed at urban expansion, such as subway systems, underground utilities, and transportation tunnels, pose significant environmental challenges, including ground settlement, vibration, and alterations in groundwater flow. Accurately predicting potential building damage is vital for assessing and mitigating some of these impacts on nearby infrastructure, allowing safe development practices. Leveraging Machine Learning (ML) tools facilitates the creation of quick and efficient prediction models for building damage assessment. In this paper, the authors generated a comprehensive synthetic dataset by conducting nearly 1000 non-linear Finite Element Method (FEM) of building damage to tunneling simulations using High-Performance Computing. This dataset include eight local and global indicators crucial for evaluating building damage resulting from tunneling activities. To address this challenge, we devised a novel unsupervised-supervised framework by integrating Principal Component Analysis and Nu Support Vector Regression (PCA-NuSVR). We developed algorithms for training and applying the proposed framework. Modeling was conducted using 5-fold cross-validation and results were evaluated using different performance metrics. Comparative analysis against various existing ML methods, including ensemble techniques, revealed the superiority of the optimized PCA-NuSVR framework. Specifically, the utilization of this framework led to a notable enhancement in prediction accuracy. The increased accuracy offered by the PCA-NuSVR framework underscore its applicability in addressing numerous practical challenges within civil engineering.

Keywords

• eol>PCA
• NuSVR
• building damage
• tunneling
• local and global assessment metrics 2

1. Introduction

Urban expansion requires the urgent development of more efficient transportation methods. Underground tunnelling effectively addresses issues like traffic congestion and carbon emissions while minimizing surface disruption [ 1 ]. However, tunnelling operations cause settlements due to the volume difference between the excavated material and the void left after lining placement caused by overcut, soil disturbance, and stress relaxation. This volume discrepancy propagates through the ground, causing surface-level "Gaussian"-like settlement troughs. Buildings near these settlements experience differential settlements, potentially leading to aesthetic damage, serviceability concerns, or even collapse in extreme cases [ 2 ].

Significant research has addressed this issue, notably by Burland [ 2 ], [ 3 ], who first linked induced settlements to building damage and categorized damage levels using the Limiting Tensile Strain Method (LTSM). Later, Potts and Addenbrooke [ 4 ] integrated Soil-Structure Interaction (SSI) effects, demonstrating that buildings' stiffness contributes to their resistance to settlements, leading to less conservative damage predictions.

Recent advancements in tunnelling-induced building damage observed the extensive utilization of the FEM [ 5 ], [ 6 ], [ 7 ] and ML-based models [ 8 ], [ 9 ] to predict the level of damage both accurately and in real-time. However, these ML tools have been trained on datasets with overly simplified damage criteria and a limited number of simulations, lacking the detail needed for comprehensive damage evaluation. While the results of these models appear accurate, the assumption of simplified damage criteria and the limited number of simulations massively restrict the ML broader applicability.

Therefore, this paper aims to apply artificial intelligence tools to predict both local and global indicators for building damage caused by tunneling.

The main contribution of this paper can be summarized as follows: 1.

We created a tabular dataset by performing nearly 1000 non-linear FEM models using HPC, for the application of artificial intelligence tools in solving the problem outlined in the paper.

We developed a novel unsupervised-supervised framework based on the combination of Principal Component Analysis (for creating a single unified hyperbody of the object for predicting all outputs) and Nu Support Vector Regression (for predicting any output using the single hyperbody) to predict values of local and global indicators during building damage assessment caused by tunneling.

We optimized the operation of the proposed PCA-NuSVR framework and demonstrated its high efficiency compared to several existing machine learning methods, particularly ensembles.

2. Materials and Methods

In this section, we describe the dataset we created, providing the main characteristics of each input and output attribute. We provide a detailed description of the components of the proposed PCA-NuSVR framework, outlining the primary steps of its training and application procedures.

2.1. Dataset descriptions

In this paper, we created a tabular dataset for 974 observations for solving different scenarios of building damage due to tunneling by ML tools, described in detail in [ 10 ]. This number of simulations with such a level of detail significantly surpasses any of the existing literature's amount. Our input space comprises 15 extensively researched parameters checked against correlations and conditions, ensuring they are not randomly selected which may cause the creation of physically unrealistic or numerically unstable models. Unlike previous models that used a simplistic output for damage assessment, our approach addresses both local and global damage aspects for each simulation as detailed in figure (Fig. 1).

Local damage aspects include maximum crack width and the total number of cracks at the building's extreme fibers, along with an average value between them. Global aspects encompass the slope, tilt, angular distortion, and horizontal strain of the building's most damaged segment and an average between them. These comprehensive damage assessment criteria were collected from various literature works.

The main characteristics of the created dataset are presented in Table 1.

The parameters defined in the upper part of Table 1 are described as follows: E, Fc, Ft, and Gft are the elastic modulus, compressive strength, tensile strength and fracture energy of the building material, respectively. "Height" and "Length" are the in-plane global dimensions of the building wall. The "Opening rate" refers to the proportion of openings (doors and windows) in the building. "Distance" is the horizontal distance from the building’s mid-span to the tunnel centerline. E_soil and "Soil_Poisson's" are the soil’s elastic modulus and Poisson's ratio, respectively. "Trough width" is a parameter that determines the skewness of the settlement trough's shape. The "Friction coefficient" measures the level of friction in the area between the soil and the building foundation. VL represents the volume of ground removed during the excavation process. "Depth" is the depth of the tunnel from the ground surface, and "Diameter" is the tunnel's outer diameter.

2.2. PCA-NuSVR cascade scheme

The proposed PCA-NuSVR framework is based on the idea of combining PCA and NuSVR. Let's consider the necessity of using each of these components in the developed framework in more detail.

PCA, as a statistical method, is used to transform input data into a new dataset by replacing the original inputs of the problem with new ones, called principal components [ 11 ]. Detailed mathematical formulations of this method, its advantages, and disadvantages are provided in [ 12 ]. It should be noted that according to the idea of the proposed framework, PCA in our case is performed over both the sum of input and all output variables. As a result, we construct a single hyperbody of the object for predicting all 8 output attributes. Additionally, such an approach allows us to predict (if necessary) any of the input attributes based on the utilization of the aforementioned hyperbody of the object.

Another advantage of using PCA in the proposed framework is that it ensures the decorrelation of the dataset. This is a crucial aspect in our case since such an approach eliminates the need to determine whether a series of output attributes to be predicted are independent or interdependent. The latter scenario often arises in tasks from the Civil Engineering domain. Moreover, applying classical machine learning algorithms to predict each of the interdependent output attributes separately is not appropriate. Thus, incorporating PCA into the proposed framework will make our approach universal in terms of the aforementioned constraint.

To predict each of the output attributes, the authors utilized Nu Support Vector Regression (NuSVR). The Nu-SVR method is a variation of the Support Vector Machine (SVM) method, used for regression tasks. The main idea is to construct a regression boundary that is as close as possible to each training sample while minimizing the error. The main difference between this method and classical SVR is the use of the parameter Nu, which specifies the percentage of points that can be excluded from the support vectors. Additionally, Nu-SVR may have fewer optimization parameters, reducing its computational complexity compared to SVR with an RBF kernel.

Among the advantages of this method, high efficiency in handling complex dependencies, minimal sensitivity to overfitting, and the ability to work with small datasets should be highlighted. However, some drawbacks include the need for parameter tuning and high sensitivity to outliers in the data.

The combined use of both aforementioned methods allowed the development of the PCANuSVR framework to solve the problem outlined in the paper. Figure 1 illustrates the flowchart of the proposed PCA-NuSVR framework.

For better visualization of all the framework's steps, the designation "ML System" is introduced. Under this term, we understand a set of machine learning methods for prediction each of the required output attributes. In our case, the ML System consists of 8 NuSVR algorithms.

It should be noted that in general, the ML System can consist of any number of similar or different machine learning methods or artificial neural networks. They should be selected according to the task at hand, the number of output attributes to be predicted, the quantity, and quality of the training data, etc. To expedite the operation of the ML System, parallelization algorithms discussed in [ 13 ], [ 14 ] can be utilized.

The structure of the developed framework comprises three main components: the preparation block, training block, and application block. Since the training block relies on the results of the preparation block, for the convenience of visualizing the framework's operation, these two blocks are combined. Thus, we have two operating modes of the framework: training mode and application mode.

Let's delve into the training and application algorithms of the proposed PCA-NuSVR framework in more detail.

2.2.1. A training algorithm for PCA-NuSVR framework

The algorithmic implementation of the training mode of the proposed PCA-NuSVR framework based on the available training dataset involves the sequential execution of the following procedures: 1. Normalization of individual n-inputs and m-outputs of the specified training dataset. 2. Combining normalized inputs and outputs into a new n*m-set of dependent features and performing the Fit method of the PCA model. 3. Training ML System 1 to predict each (from m) output attribute using the initial n-inputs of the task. 4. Applying the pre-trained ML System 1 for intermediate prediction of all m-output attributes on the training dataset. 5. Combining normalized initial inputs and predicted outputs from step 4 into a new n*mset of dependent features and performing the Transform method of the PCA model to transition into the principal component space and forming a new training dataset based on them (creating a single hyperbody of the object for predicting all outputs). 6. Training ML System 2 for the final prediction of each (from m) output attribute using the new, extended, and decorrelated set of independent attributes created in the previous step. Performing reverse normalization of each (from m) output attribute (to compute training errors).

2.2.2. Application algorithm for PCA-NuSVR framework

The algorithmic implementation of the application mode of the proposed PCA-NuSVR framework, based on utilizing the current data vector with unknown output or an available test dataset, involves the sequential execution of the following procedures: 1. Normalization of n-inputs of the current data vector with unknown output attributes. 2. Applying the pre-trained ML System 1 for intermediate prediction of all m-output attributes for the current data vector based on the initial training dataset. 3. Combining normalized initial inputs and predicted outputs from step 2 into a new n*mvector of dependent features and performing the Transform method of the PCA model from the training mode to transition into the principal component space and forming a new extended data vector.

Applying the pre-trained ML System 2 on the current, already extended, and decorrelated data vector from the previous step for the final prediction of each (from m) output attribute.

Performing reverse normalization of the predicted outputs (to form the final value in the case of analyzing the current data vector or to compute method errors in the case of analyzing the test dataset).

The main advantages of using the proposed PCA-NuSVR framework are as follows:  Formation of a unified hyperbody of the object for predicting each of the required output attributes.  Decorrelation of the initial dataset by transitioning from the initial inputs of the task into the principal component space.  Expansion of the input data space of the task by utilizing m-output attributes along with n-inputs and transitioning into the principal component space.  Elimination of the need to determine whether the m-outputs of the task are interrelated or independent.

All of this ensures the universality of the proposed solution for addressing many civil engineering tasks using artificial intelligence means, in case there is a need to predict multiple output attributes formed based on the same independent attributes dataset [ 15 ].

3. Modeling and results

3.1. Data preprocessing

The operation modeling of the proposed PCA-NuSVR framework was conducted on a dataset created by us based on non-linear FEM models using. To clean the dataset from anomalies, the authors used the Z-score criterion. This characteristic helps identify values that significantly differ from the mean values in the data. Observations with a Z-score greater than 3 or less than 3 are outliers and will not be used for training and testing the model. Thus, the final dataset for further analysis contains 916 instances (instead of 974), each characterized by 15 input features and 8 outputs.

Next, the data was split into training and test datasets. The training dataset was normalized using MaxAbsScaler. According to this method, scaling and transformation of each variable are performed so that the maximum absolute value of each variable in the training set is equal to 1. This technique does not shift or center the data. It should be noted that normalization was performed separately for inputs and outputs. The obtained normalization coefficients were used to normalize the inputs and outputs in the test dataset accordingly. Additionally, reverse normalization was performed on the predicted data before calculating the errors of the proposed method.

To ensure the reliability of the prediction results, 5-fold cross-validation was performed in the work. It should be noted that the described data normalization scheme was performed before running each fold.

3.2. Optimal parameters selection.

Optimizing the operation of the proposed PCA-NuSVR framework, i.e., tuning its parameters for optimal performance, is an important stage of its practical use [ 16 ]. In this paper, optimization of the NuSVR operation was conducted as the foundational machine learning algorithm underlying the framework. Bayesian optimization technique was employed for this purpose [ 17 ]. Among the parameters optimized were Nu (the ratio of support vectors) and C (the penalty parameter for regularization). The optimization aimed at maximizing the R2 score during the prediction of each output attribute. The optimal parameters obtained for predicting each of the eight output attributes are summarized in Table 2.

3.3. Results

The results of the proposed PCA-NuSVR framework for global and local indicators during the assessment of building damage caused by tunneling are summarized in Table 2. It should be noted that Table 2 presents the prediction results using various performance metrics for a more comprehensive analysis of the obtained results. Additionally, the table includes the average values after performing a 5-fold cross-validation.

4. Comparison and discussion

For comparison of the accuracy of the proposed PCA-NuSVR framework, a series of existing machine learning methods were selected: the basic NuSVR used as the foundation of the proposed framework; classical SVR with RBF kernel as a very similar method to the previous one, and a series of ensemble methods such as Random Forest, Gradient Boosting, XGBoost, and LGBM regressor.

Figures 3 and 4 summarize the comparison results of all investigated methods based on the R2 score for assessing the accuracy of predicting the local and global indicators for the assessment of building damage caused by tunneling, respectively.

As evident from Figure 3, the unsatisfactory prediction accuracy of the local indicators is demonstrated by ensemble methods such as Random Forest, Gradient Boosting, and XGBoost. Somewhat better results were obtained when using classical SVR with RBF kernel and LGBM regressor. Significantly higher accuracy compared to the previous methods is demonstrated by the basic NuSVR used as the foundation of the proposed framework. However, the smallest errors during the prediction of the local indicators for the assessment of building damage caused by tunneling were obtained using the proposed PCA-NuSVR framework. It shows an increase in R2 from 1.8 to 4.6 depending on the predicted indicator.

Similar results were obtained during the prediction of each of the global indicators (Figure 4). In particular, NuSVR shows one of the best results compared to all other existing methods. However, the proposed PCA-NuSVR framework demonstrates an increase in the R2 value from 0.9 to 3.8 depending on the global indicators being predicted.

Among the prospects for further research, three main directions should be considered. The first is the replacement of PCA with an auto-associative SGTM neural-like structure with noniterative training [18], which will help obtain the principal components much faster compared to the basic method. The second direction involves the possibility of nonlinear extension of transformed inputs (principal components) to increase prediction accuracy. In this case, the second-degree Wiener polynomial can be applied, which is characterized by high approximation properties. However, such an approach significantly increases the dimensionality of the input data space and may provoke overfitting [19]. Because the inputs in the proposed PCA-NuSVR framework are principal components with different variances, it is possible to perform a nonlinear extension of inputs only for the first significant principal components (3-5 principal components) and add the obtained values as additional inputs to the initial dataset. The third direction involves investigating the effectiveness of using artificial neural networks as weak predictors of the developed method [20], [21], [22], [23]. Depending on the quality and quantity of the training dataset, such an approach can improve the accuracy of solving the problem at hand.

Such an approach will ensure (i) accounting for nonlinearity in the dataset being processed, (ii) without significant increase in the dimensionality of the problem, (iii) thus preserving the high generalization properties of the proposed PCA-NuSVR framework.

5. Conclusions

The authors proposed an innovative unsupervised-supervised framework, termed the PCANuSVR framework, which integrates Principal Component Analysis and Nu Support Vector Regression. The framework's methodology is elucidated through the provision of a flowchart, accompanied by the development of training and application algorithms.

The performance evaluation of the framework was conducted on a meticulously preprocessed dataset, free from anomalies, and normalized separately for inputs and outputs. To ensure the reliability of results, the study incorporated a 5-fold cross-validation approach. Subsequent optimization of the proposed PCA-NuSVR framework involved the meticulous selection of optimal parameters through the application of Bayesian optimization techniques. The optimization process aimed at maximizing the coefficient of determination for each of the eight output attributes individually.

A comparative analysis was undertaken against a spectrum of existing machine learning methodologies, including ensemble techniques, revealing the superior efficacy of the optimized PCA-NuSVR framework. Specifically, the utilization of this framework yielded a noteworthy enhancement in prediction accuracy. This renders the proposed PCA-NuSVR framework advantageous for the practical resolution of various challenges encountered within the domain of civil engineering.

Acknowledgements

Ivan Izonin, Roman Tkachenko, and Stergios-Aristoteles Mitoulis would like to acknowledge the financial support of the European Union’s Horizon Europe research and innovation program under grant agreement No 101138678, project ZEBAI (Innovative methodologies for the design of Zero-Emission and cost-effective Buildings enhanced by Artificial Intelligence). [18]R. Tkachenko, ‘An Integral Software Solution of the SGTM Neural-Like Structures Implementation for Solving Different Data Mining Tasks’, in Lecture Notes in Computational Intelligence and Decision Making, vol. 77, S. Babichev and V. Lytvynenko, Eds., in Lecture Notes on Data Engineering and Communications Technologies, vol. 77. , Cham: Springer International Publishing, 2022, pp. 696–713. doi: 10.1007/978-3-03082014-5_48. [19]I. Izonin, R. Tkachenko, R. Holoven, K. Yemets, M. Havryliuk, and S. K. Shandilya, ‘SGD-Based Cascade Scheme for Higher Degrees Wiener Polynomial Approximation of Large Biomedical Datasets’, MAKE, vol. 4, no. 4, pp. 1088–1106, Nov. 2022, doi: 10.3390/make4040055. [20]O. Mulesa, V. Snytyuk, and I. Myronyuk, ‘OPTIMAL ALTERNATIVE SELECTION MODELS IN A MULTI-STAGE DECISION-MAKING PROCESS’, EUREKA: Physics and Engineering, vol. 6, pp. 43–50, Nov. 2019, doi: 10.21303/2461-4262.2019.001005. [21]Y. Bodyanskiy, A. Pirus, and A. Deineko, ‘Multilayer Radial-basis Function Network and its Learning’, in 2020 IEEE 15th International Conference on Computer Sciences and Information Technologies (CSIT), Sep. 2020, pp. 92–95. doi: 10.1109/CSIT49958.2020.9322001. [22]I. Krak, V. Kuznetsov, S. Kondratiuk, L. Azarova, O. Barmak, and P. Padiuk, ‘Analysis of Deep Learning Methods in Adaptation to the Small Data Problem Solving’, in Lecture Notes in Data Engineering, Computational Intelligence, and Decision Making, vol. 149, S. Babichev and V. Lytvynenko, Eds., in Lecture Notes on Data Engineering and Communications Technologies, vol. 149. , Cham: Springer International Publishing, 2023, pp. 333–352. doi: 10.1007/978-3-031-16203-9_20. [23]S. Subbotin, ‘Radial-Basis Function Neural Network Synthesis on the Basis of Decision Tree’, Opt. Mem. Neural Networks, vol. 29, no. 1, pp. 7–18, Jan. 2020, doi: 10.3103/S1060992X20010051.

References

[1] J. Ninić , A. Gamra , and B. Ghiassi , ' Real-time assessment of tunnelling-induced damage to structures within the building information modelling framework' , Underground Space , vol. 14 , pp. 99 - 117 , Feb. 2024 , doi: 10.1016/j.undsp. 2023 . 05 .010.

[2] J. B. Burland and C. P. Wroth , Settlement of Buildings and Associated Damage . Building Research Establishment , 1975 .

[3] J. Burland , B. Broms , and V. Mello , ' Behaviour of foundations and structures' , Engineering, pp. 495 - 546 , 1977 .

[4] D. M. Potts and T. I. Addenbrooke , ' A structure's influence on tunnelling-induced ground movements' , Proceedings of the Institution of Civil Engineers - Geotechnical Engineering , vol. 125 , no. 2 , pp. 109 - 125 , Apr. 1997 , doi: 10.1680/igeng. 1997 . 29233 .

[5] G. Giardina , M. A. N. Hendriks , and J. G. Rots , ' Sensitivity study on tunnelling induced damage to a masonry façade' , Engineering Structures , vol. 89 , pp. 111 - 129 , Apr. 2015 , doi: 10.1016/j.engstruct. 2015 . 01 .042.

[6] W. N. Yiu , H. J. Burd , and C. M. Martin , ' Finite-element modelling for the assessment of tunnel-induced damage to a masonry building' , Géotechnique , vol. 67 , no. 9 , pp. 780 - 794 , Sep. 2017 , doi: 10.1680/jgeot.sip17.P. 249 .

[7] J. Zhao and M. DeJong , ' Three-dimensional probabilistic assessment of tunneling induced structural damage using Monte-Carlo method and hybrid finite element model' , Computers and Geotechnics , vol. 154 , p. 105122 , Feb . 2023 , doi: 10.1016/j.compgeo. 2022 . 105122 .

[8] B. T. Cao , M. Obel , S. Freitag , P. Mark , and G. Meschke, ' Artificial neural network surrogate modelling for real-time predictions and control of building damage during mechanised tunnelling' , Advances in Engineering Software , vol. 149 , p. 102869 , Nov . 2020 , doi: 10.1016/j.advengsoft. 2020 . 102869 .

[9] M. Obel , M. A. Ahrens , and P. Mark, ' Metamodel-Based Prediction of Structural Damages due to Tunneling-Induced Settlements' , ASCE-ASME J. Risk Uncertainty Eng . Syst., Part A : Civ. Eng., vol. 6 , no. 4 , p. 04020044 , Dec . 2020 , doi: 10.1061/AJRUA6.0001092.

[10] A. Gamra , J. Ninic , and B. Ghiassi , ' Optimizing Synthetic Datasets for Machine Learningbased Prediction 2 of Building Damage due to Tunnelling' , Tunnelling and Underground Space Technology incorporating Trenchless Technology Research , (submitted), 2024 .

[11] N. Shakhovska , V. Yakovyna , and N. Kryvinska , ' An Improved Software Defect Prediction Algorithm Using Self-organizing Maps Combined with Hierarchical Clustering and Data Preprocessing' , in Database and Expert Systems Applications , vol. 12391 , S. Hartmann , J. Küng , G. Kotsis , A. M. Tjoa , and I. Khalil, Eds., in Lecture Notes in Computer Science , vol. 12391 . , Cham: Springer International Publishing, 2020 , pp. 414 - 424 . doi: 10 .1007/978-3- 030 -59003-1_ 27 .

[12] I. T. Jolliffe and J. Cadima , ' Principal component analysis: a review and recent developments' , Phil. Trans. R. Soc. A. , vol. 374 , no. 2065 , p. 20150202 , Apr . 2016 , doi: 10.1098/rsta. 2015 . 0202 .

[13] L. Mochurad , K. Shakhovska , and S. Montenegro , ' Parallel Solving of Fredholm Integral Equations of the First Kind by Tikhonov Regularization Method Using OpenMP Technology' , in Advances in Intelligent Systems and Computing IV , vol. 1080 , N. Shakhovska and M. O. Medykovskyy , Eds., in Advances in Intelligent Systems and Computing , vol. 1080 . , Cham: Springer International Publishing, 2020 , pp. 25 - 35 . doi: 10 .1007/978-3- 030 -33695- 0 _ 3 .

[14] L. Mochurad , O. Kotsiumbas , and I. Protsyk , ' A Model for Weather Forecasting Based on Parallel Calculations' , in Advances in Artificial Systems for Medicine and Education VI , vol. 159 , Z. Hu , Z. Ye , and M. He, Eds., in Lecture Notes on Data Engineering and Communications Technologies , vol. 159 . , Cham: Springer Nature Switzerland, 2023 , pp. 35 - 46 . doi: 10 .1007/978-3- 031 -24468- 1 _ 4 .

[15] S.-A. Mitoulis et al., ' Conflict-resilience framework for critical infrastructure peacebuilding' , Sustainable Cities and Society , vol. 91 , p. 104405 , Apr . 2023 , doi: 10.1016/j.scs. 2023 . 104405 .

[16] V. Shymanskyi , I. Sokolovskyy, Y. Sokolovskyy , and T. Bubnyak, ' Variational Method for Solving the Time-Fractal Heat Conduction Problem in the Claydite-Block Construction' , in Advances in Computer Science for Engineering and Education , vol. 134 , Z. Hu , I. Dychka , S. Petoukhov , and M. He, Eds., in Lecture Notes on Data Engineering and Communications Technologies , vol. 134 . , Cham: Springer International Publishing, 2022 , pp. 97 - 106 . doi: 10 .1007/978-3- 031 -04812- 8 _ 9 .

[17] C. Ganguli , S. K. Shandilya , M. Nehrey , and M. Havryliuk , ' Adaptive Artificial Bee Colony Algorithm for Nature-Inspired Cyber Defense' , Systems , vol. 11 , no. 1 , p. 27 , Jan . 2023 , doi: 10.3390/systems11010027.