Knowledge-based Registration for Reliable Correlated Change Detection on High-pier Curved Continuous Rigid Frame Bridge

Abstract

Change detection on continuous rigid frame bridges is challenging due to its complex shapes. Accurate and effective change analysis is necessary to not only locate the changes but also investigate the root causes behind. Registration of 3D imageries collected at different times for structural change analysis, however, is an obstacle even with the help of advanced registration algorithms. This study proposed a knowledge-based registration approach that segments point clouds into structural elements for recovering the correlated torsional and bending behaviors of connected bridge elements. The proposed approach uses the joint equilibrium condition between structural elements as domain knowledge to guide the selection of correspondences for point cloud registration and element-level change detection. Besides, the approach iterates the correspondence locating process until change detection results comply with the joint equilibrium condition. The results indicate that the proposed knowledge-based registration approach provides more reliable correlated change detection results than using the Iterative Closest Point (ICP) algorithms.

1. Introduction

However, the current practice of bridge inspection still heavily rely on the judgment of field bridge engineers and tedious inspection works require large amounts of manual surveying (Hüthwohl, Lu and Brilakis, 2016). Terrestrial laser scanning (TLS) provides the potential for acquiring accurate geometric information of bridges and give a quantitative estimation of the as-is conditions (Meral, 2011) . Recent studies revealed the potential of using point cloud data collected by laser scanners at different times for detecting spatial changes of bridge structures over the years (Tang and Akinci, 2012; Kalasapudi et al., 2018) . Using acquired 3D data for supporting bridge inspection by aligning two point clouds captured from different times and visually comparing the differences is essential for change detection (Kasireddy and Akinci, 2015) . Such aligned process is the “registration” of two point cloud datasets into one common coordinate system. However, conventional registration approach by simply aligning raw point clouds not only requires a significant amount of human efforts (i.e., select control points, extract 2D features, and so on) but also challenging in terms of accuracy and computational efficiency. Analyzing changes of large-span bridges from 3D point clouds, which may contain billions of points, becomes more challenging (Kasireddy and Akinci, 2015) . Unreliable or inaccurate registration of 3D laser scanning datasets of a bridge collected at different times can lead to incorrect detections of spatial changes and eventually leading to unreliable condition assessment of bridge structures (Kalasapudi, Tang and Turkan, 2017) .

Structural deteriorations on bridges follow specific patterns depending on the structure type and designed loading capacity (Law, Silcock, and Holden, 2018) . Previous studies have claimed the potential of using similar deformation patterns as knowledge for supporting reliable change detection (Zong et al., 2016) . Changes on connected bridge elements are correlated and should follow the same deformation patterns according to fundamental structure theorem (Xu, Yang, and Neumann, 2018) . Besides, certain bridge elements contain different structural characteristics in terms of deformation. Girders on a curved continuous rigid frame bridge should suffer both bending and torsion due to the curvature and other causes (i.e., concrete shrinkage, prestress losses, and so on) (Feng et al., 2018; Mangalathu et al., 2018) . Plus, girders and piers on such bridge are rigidly connected and do not allow relative rotation. Piers on such bridge will have lateral displacement triggered by the torsion of the connected girders (Yuan and Harik, 2010) . Such structural engineering knowledge help not only determines correspondences on different datasets for accurate element-level registration and deformation analysis but also validate the change detection results using the moment equilibrant theorem. Registration is a process that aligns two point clouds. Global registration is an alignment method to align multiple three-dimensional shapes in arbitrary initial positions (Gelfand et al., 2005). Existing studies reveal the potential of applying advanced registration algorithms for global change detection with a limited explanation on the detected spatial changes (Antova, 2015; Zheng et al., 2016; Kalasapudi et al., 2018) . Others have discussed and examined the potential of applying the non-rigid registration method for revealing spatial transformations of local correspondence ( Bosché, 2010 ; Ma et al., 2017). Some studies examined an automated method for bridge element segmentation (Lu and Ioannis, 2018; Lu, Brilakis and Middleton, 2018) . Other studies developed automatic skeleton detection algorithms for object detection (Han and Lee, 2013; Saha, Borgefors and Sanniti di Baja, 2016) . Structure engineers, however, spend efforts in experimental testing and finite element modeling for understanding bridge failure modes (Barbato and Conte, 2006; Conte and Zhang, 2007; Moaveni and Conte, 2014) . However, limited research has considered the fundamental structural theorem when registering two datasets with partially overlapped points for change detection.

This study proposed a structural engineering knowledge-based registration approach that firstly segments point clouds into structural elements for recovering the torsional and bending behaviors of structural elements. The proposed approach uses the joint equilibrium condition between structural elements as domain knowledge to guide the selection of correspondences for point cloud registration and element-level change detection. Besides, the approach iterates the correspondence locating process until a finely registered scan. In the end, the approach correlates the detected changes for validating the change detection results and assessing structural system behaviors

2. Methodology

The proposed research has two objectives: 1) Establish a registration method using joint equilibrium condition between structural elements as domain knowledge for reliable 3D point clouds registration to detect element-level deformations; 2) Validate the correlated change detection results using structural engineering knowledge (equilibrium condition at the joints between/among structural elements). The proposed research then examined the established knowledge-based registration method for spatial change detection by using a prestressed concrete curved continuous rigid frame bridge as a case in China.

Structural deterioration knowledge is the information associated with particular structural deformation patterns of bridges. Besides, deformation patterns of adjacent girders and pier connected at a joint should follow specific structure rules depending on the boundary condition (i.e., bending-torsion coupling scenario occurred at rigid frame bridges). The proposed knowledge-based registration method uses joint equilibrium condition between structural elements as domain knowledge with bridge inspection experiences to help support reliable change detections. The pipeline of the proposed knowledge-based registration approach shown in Figure 1 below. The proposed method first segments point clouds into structural elements for recovering the correlated torsional and bending behaviors of connected bridge elements. Often, the local deformations could be “overwhelmed” by rigid body motions at the structure level when conducting element-level change analysis with global registration method. The authors first conducted structure skeletonization to extract skeletons of the upper (girder) and the lower part (pier) of the bridge. For the upper part of the bridge, the authors draw a polyline based on the centreline of the bridge from the top view. Based on the polyline (path), the authors define the distance along the path between two orthogonal sections in order to create multiple orthogonal sections along the polyline (the distance must be small in order to get more orthogonal sections in order to represent the shape of the bridge better). As for the pier, the authors extract the contour of each pier on its side view. After skeletonization of the bridge, the authors identify joints based on the contours of piers and orthogonal sections. Then, the authors isolate the pier from the superstructure and segment each span for preparation of local registration. The authors then assess the joint equilibrium condition at joints that connecting adjacent girders and pier based on the boundary condition. As a rigid frame bridge, the superstructure and substructure are rigidly connected at joints to act as a continuous unit. The joints connecting structure elements are rigid connections which transfer bending moment, axial forces, and shear forces. Besides, such rigid connections do not allow relative rotations between connected structural elements. However, boundaries of girders and the bottom part of piers tend to have relatively small deformations, even the girders and piers are experiencing bending and torsional behaviors. The authors first decided to use boundaries of girders and the bottom part of piers as correspondences for local registration to detect deformations of girders and piers. Next, the proposed method will conduct a local registration on each span and pier segmented from the previous step between different datasets. During the registration, a point-pair method has been used based on the correspondences selected according to the joint equilibrium condition assessment. For instance, Figure 2 shows the selected correspondence (four points) at the left end of a girder for registration of the third span. The authors conducted a Cloud-to-cloud (C2C) distance calculation for change detection. At last, the proposed method correlates and validates the change detection results of girders and piers according to the joint equilibrium condition at the joints between structural elements. If the correlated change detection results do not comply with the joint equilibrium condition, the proposed method will update the correspondences by using unchanged segments from the change detection results for another iteration of change detection. The method will iterate the correspondence updating and registration process until change detection results comply with the joint equilibrium condition.

3. Case Study: Inspection of a Curved Continuous Rigid Frame Bridge in China

The studied bridge is at Anhui Province, China, which has extremely mountainous and complex terrain conditions. The bridge has a total length of 1,010 m, including a six-span main bridge of 612m (66m + 4*120m + 66m). The main bridge uses the prestressed concrete continuous rigid frame with high hollow thin-walled piers (83m). The authors collected 3D point cloud data for the Bridge in China in 2017 and 2018 by using the FARO Focus S 350 laser scanner (37 scans for 2017; 31 scans for 2018). The authors performed the registration for each 3D laser scanning dataset by using Cloud-to-Cloud (C2C) registration through SCENE (FARO, 2019) and manually remove the redundant data (i.e., trees, cars, and so on) by using the segmentation tool in CloudCompare (Girardeau-Montaut, 2019). Due to the size of the point cloud data, the authors proposed a skeleton-based segmentation method to 1) isolate the upper part (deck), and the lower part of the bridge (pier) based on the joint detected within the skeleton that represents the bridge topology; and 2) segment each span into one individual part based on the joint for local registration preparation.

3.1 Structural skeletonization and joint detection

The authors first manually draw a polyline (appears in red) along the center line of the bridge from the top view of the point cloud (see Figure 3). CloudCompare then automatically generated multiple orthogonal sections along the polyline based on the defined width between sections (the distance along the polyline between two orthogonal sections are equal). After the generations of several sections, the authors ask CloudCompare to extract the corresponding cloud slices of each generated sections. Then, the authors rotate the point cloud to the side view of the bridge and try to extract the 2D contour of each pier by using the “Cross Section Segmentation” tool in CloudCompare.

3.2 Point cloud segmentation

Though existing research has applied automated algorithms for point cloud segmentation (Lu and Ioannis, 2018) , the focus of this study is not in the same direction. The point cloud segmentation process includes two steps: 1) isolate the upper and lower part of the bridge; and 2) segment each span into individual elements. The author first manually identified the joints connecting the cross sections of the upper part and the 2D contour of piers and use that joint in order to isolate the upper and lower part of the bridge (blue: upper part of the bridge; red: lower part of the bridge). The authors use the segmentation tool in CloudCompare to complete the segmentation process. Then, the authors segmented each span into individual parts. This segmentation uses the cross sections generated according to the polyline mentioned in the last section (see Figure 4).

3.3 Registration

Local deformations of structure elements are critical to be used for investigating changes of internal forces and assessing changes of bridge condition. The proposed knowledge-based registration method is a non-rigid local registration that integrates structural engineering knowledge for local deformations at the structure element level. Such a knowledge-based registration method could help revealing local deformations of the bridge elements (girders and piers) and eliminate the errors caused by other registration methods (i.e., ICP). The authors use correspondences that are established based on existing local features. Moreover, the authors selected these features at locations where deformations are unlikely to occur (both ends of girders and bottom of piers) based on bridge engineering knowledge.

According to the point cloud, the points are much denser in the 2nd, 3rd, and the 4th span of the studied bridge. The authors then examined the proposed local registration method in the 2nd, 3rd, and the 4th span of the studied bridge. Also, the authors registered these three spans globally (registration without segmentation of individual spans) by using the traditional registration algorithms (ICP) for comparison.

In the proposed knowledge-based registration process, the authors registered each span between the two-year point clouds (i.e., register the girder of the 3rd span from the 2017 dataset to the same girder in 2018 dataset) . To get better registration results, the authors manually select the corresponding points at both ends of the girder (the authors assume both ends of a girder tend to have relatively small deformations than the mid-span based on basic structural engineering knowledge) (see Figure 5). By using the Iterative Closest Point (ICP) registration method, the authors registered the whole three spans from the 2017 dataset to the same spans in 2018 dataset by using the traditional registration algorithms (ICP) (see Figure 6).

4. Change Detection Results and Discussion

The preliminary research findings suggest that the established knowledge-based registration method has the potential of accurate detections of structural changes on curved continuous rigid frame bridges. Such method resolves the difficulties of aligning the deformed structure with its original shape when finding correspondences for change detection during conventional rigid and non-rigid registration approach. Besides, change detection results show the detected spatial changes obey the fundamental structure theorem between each bridge elements and aligned with the documented deficiencies in the inspection report.

4.1 Change Detection – Knowledge-based Local Registration on Girders and Piers

The authors compute a Cloud-to-cloud (C2C) distance between the registered point cloud (2018) and the reference point cloud (2017). Results (see Figure 7) indicate the girders of the studies three spans are all suffered bending and torsion, and the piers are undergoing lateral displacement. According to the Right-hand Rule, the vector of the torsion can have two elements that are along the x (lateral direction) and y (longitudinal direction) directions of the bridge. Plus, the change detection results indicate the pier connecting the 2nd and the 3rd span of the bridge is undergoing lateral displacement (see Figure 8). The maximum lateral displacement occurred at the top of the pier, whereas the bottom of the pier has no displacement. The results help explain a bending-torsion coupling scenario where the torsion of the girders is balancing the moment of the high pier connected. The authors also checked the bending status of each spans to determine the direction (upward, downward) of the bending. For girders of each span, the authors select three points (joints connecting the deck and pier) from each dataset to fit a plane horizontally, then calculate the distance (d) from the vertex (point C) of this span to the fitted plane (plane1,2,3). Then, the authors compared the difference between the distances (Δd) calculated from the 2017 and the 2018 datasets to determine the direction of bending on this span (see Figure 9). Results (see Figure 10) indicates that the 2nd and the 4th span are suffered downward bending, whereas the 3rd span is bending upward. Plus, the authors found that the piers within this threespan are undergoing displacement in the longitudinal direction (y direction) of the bridge. The results indicate the consistency of the bending and torsional behaviors of connecting girders and high piers at the joints.

4.2 Change Detection – Rigid Local Registration using ICP Algorithm on Girders and

Piers

The authors compute a Cloud-to-cloud (C2C) distance between the registered point cloud (2018) and the reference point cloud (2017). Results (see Figure 11) indicate the girders of the studies three spans are all suffered only bending. However, the pier connecting the 2nd and the 3rd span of the bridge is undergoing lateral displacement (x-direction). The registration results indicate that the girders of the bridge are suffered to bend, whereas no displacement in the lateral direction should appear on the pier. Deformations shown from the registration results show inconsistency between girders and pier and do not align with the documented deficiencies in the inspection report.

5. Conclusion and Future Research

Due to the curvature of the studied bridge, the center of the gravity of the bridge is not on the bridge, which often reflects in the coupling of bending and torsion. Moreover, since the studied bridge is a rigid frame bridge, the joint connecting the girder and pier is fixed, which allows force and moment transfer. In other words, the torsion of the bridge must balance the bending of the pier in the lateral direction of the bridge. Moreover, the bending moment of the studied bridge in the three spans are also balanced by the longitudinal bending of the piers according to the joint analysis.

Most current research focuses on the use of "nearest" neighbor search (NNS) to compare two clouds, which effectively calculates the closest (or most similar) points. Direct comparison of two partially overlapping point clouds of the same object and measuring the changes between them could be hard with some popular computer methods (i.e., ICP). It is thus vital to understand small size changes of specific objects for interpreting deformations and require a subtler analysis of the measured scene. Besides, structure engineering knowledge specifies possible correlated deformations among connected bridge element, which is valuable information for validating the change detection results and support further registration. The proposed method does not only guide accurate local registration but also offers a potential for adjusting and supporting the planning of bridge inspection. This study reveals the fact that the traditional registration approach or register using the control network fixed on the job site might not be appropriate for understanding the structural behaviors of curved continuous rigid frame bridges. Moreover, the proposed method shows the potential of analyzing local deformations of bridge elements for assessing potential bridge failures.

Acknowledgment

This material is based on work supported by the science and technology project on transportation construction by the U.S. National Science Foundation (NSF) under Grant No. 1454654 and the Ministry of Transport of the People's Republic of China (Project No. 2013318223380, 2014318J14250) . The supports are gratefully acknowledged.

Gelfand, N., Mitra, J. N., Guibas, J. L. and Pottmann, H. (2005) ‘Robust Global Registration’, in Eurographics Symposium on Geometry Processing, pp. 156–160. DOI: 10.1016/0167-4781(89)90095-X. Girardeau-Montaut, D. (2019) CloudCompare - 3D point cloud and mesh processing software Open Source Project. Available at: http://www.cloudcompare.org/. Hüthwohl, P., Lu, R. and Brilakis, I. (2016) ‘Challenges of bridge maintenance inspection’, 16th International Conference on Computing in Civil and Building Engineering, (July).

Kalasapudi, V. S., Tang, P. and Turkan, Y. (2017) ‘Computationally efficient change analysis of piece-wise cylindrical building elements for proactive project control’, Automation in Construction. Elsevier, 81(May), pp. 300–312. DOI: 10.1016/j.autcon.2017.04.001.

Kalasapudi, V. S., Tang, P., Xiong, W. and Shi, Y. (2018) ‘A multi-level 3D data registration approach for supporting reliable spatial change classification of single-pier bridges’, Advanced Engineering Informatics. Elsevier, 38(June), pp. 187–202. DOI: 10.1016/j.aei.2018.06.010.

Kasireddy, V. and Akinci, B. (2015) ‘Challenges in Generation of As-is Bridge Information Model: A Case Study’, 32nd International Symposium on Automation and Robotics in Construction. DOI: 10.22260/ISARC2015/0037. Law, D. W., Silcock, D. and Holden, L. (2018) ‘Terrestrial laser scanner assessment of deteriorating concrete structures’, Structural Control and Health Monitoring, 25(5), pp. 1–15. DOI: 10.1002/stc.2156. Li, J., Yang, Y. and Yang, Z. (2015) ‘Long-span continuous rigid frame bridge construction control’, (Icadme), pp. 1316–1320.

Lu, R., Brilakis, I., and Middleton, C. R. (2018) ‘Detection of Structural Components in Point Clouds of Existing RC Bridges’, Computer-Aided Civil and Infrastructure Engineering, (July). DOI: 10.1111/mice.12407. Lu, R. and Ioannis, B. (2018) ‘Detection of Key Components of Existing Bridge in Point Cloud Datasets’, in, pp. 111–115.

References

Antova , G. ( 2015 ) 'Registration Process of Laser Scan Data in the Field of Deformation Monitoring', Procedia Earth and Planetary Science . Elsevier B.V. , 15 , pp. 549 - 552 . DOI: 10 .1016/j.proeps. 2015 . 08 .096.

Barbato , M. and Conte , J. P. ( 2006 ) ' Finite element structural response sensitivity and reliability analyses using smooth versus non-smooth material constitutive models' , International Journal of Reliability and Safety , 1 ( 1 /2), p. 3 .DOI: 10 .1504/ijrs. 2006 . 010688 .

Bosché , F. ( 2010 ) 'Automated recognition of 3D CAD model objects in laser scans and calculation of as-built dimensions for dimensional compliance control in construction' , Advanced Engineering Informatics , 24 ( 1 ), pp.

107- 118 . DOI: 10 .1016/j.aei. 2009 . 08 .006.

Chen , H. and Zheng , S. ( 2015 ) 'Seismic Response of Ultra High-pier Railway Bridge under Spatially Varying Site Conditions of Earthquake Excitations' , Journal of Highway and Transportation Research and Development , 9 ( 2 ), pp. 61 - 68 . DOI: 10 .3969/j.

Conte , J. P. , and Zhang, Y. ( 2007 ) ' Performance-Based Earthquake Engineering: Application to an Actual BridgeFoundation-Ground System' , In 12th Italian national conference on earthquake engineering , Pisa, Italy ., pp. 1 - 18 . Available at: http://jaguar.ucsd.edu/publications/Conference Proceedings/ANIDIS_Conte_Keynote_Paper_with_Header.pdf.

FARO ( 2019 ) SCENE 2019 . Available at: https://www.faro.com/products/construction-bim-cim/faro-scene/.

Feng , R. , Wang , X. , Yuan , W. and Yu , J. ( 2018 ) 'Impact of seismic excitation direction on the fragility analysis of horizontally curved concrete bridges' , Bulletin of Earthquake Engineering . Springer Netherlands, 16 ( 10 ), pp.

Ma , J. , Zhao , J. , Jiang , J. and Zhou , H. ( 2017 ) ' Non-rigid point set registration with robust transformation estimation under manifold regularization' , 31st AAAI Conference on Artificial Intelligence , AAAI 2017 , pp. 4218 - 4224 . DOI: http://0-dx.doi.org. wam.leeds.ac.uk/10 .1097/BRS.0b013e31823a012e.

Mangalathu , S. , Choi , E. , Park , H. C. and Jeon , J.-S. ( 2018 ) 'Probabilistic Seismic Vulnerability Assessment of Tall Horizontally Curved Concrete Bridges in California', 32 ( 6 ). DOI: 10 .1016/j.foreco. 2015 . 09 .026.

Meral , C. ( 2011 ) 'Evaluation of laser scanning technology for bridge inspection' , Measurement, ( March ).

Available at: http://idea.library.drexel.edu/bitstream/1860/3451/1/Meral, Cihan.pdf.

Moaveni , B. and Conte , J. P. ( 2014 ) 'System and Damage Identification of Civil Structures' , in Encyclopedia of Earthquake Engineering, pp. 1 - 9 . DOI: 10 .1007/978-3- 642 -36197-5_ 70 - 1 .

Pahlavan , H. , Zakeri , B. , Amiri , G. G. and Shaianfar , M. ( 2015 ) 'Probabilistic Vulnerability Assessment of Horizontally Curved Multiframe RC Box-Girder Highway Bridges' , Journal of Performance of Constructed Facilities , 30 ( 3 ), p. 4015038 . DOI: 10 .1061/(ASCE)cf. 1943 - 5509 . 0000780 .

Saha , P. K. , Borgefors , G. , and Sanniti di Baja, G. ( 2016 ) 'A survey on skeletonization algorithms and their applications', Pattern Recognition Letters . Elsevier Ltd., 76 , pp. 3 - 12 . DOI: 10 .1016/j.patrec. 2015 . 04 .006.

Tang , P. and Akinci , B. ( 2012 ) 'Automatic execution of workflows on laser-scanned data for extracting bridge surveying goals', Advanced Engineering Informatics . Elsevier B.V. , 26 ( 4 ), pp. 889 - 903 . DOI: 10 .1016/j.aei. 2012 . 07 .004.

Xu , X. , Yang , H. and Neumann , I. ( 2018 ) ' Deformation monitoring of typical composite structures based on terrestrial laser scanning technology', Composite Structures . Elsevier Ltd , 202 , pp. 77 - 81 . DOI: 10 .1016/j.compstruct. 2017 . 11 .049.

Yuan , P. and Harik , I. E. ( 2010 ) 'Equivalent Barge and Flotilla Impact Forces on Bridge Piers' , Journal of Bridge Engineering , 15 ( 5 ), pp. 523 - 532 . DOI: 10 .1061/(ASCE)BE. 1943 - 5592 . 0000080 .

Zheng , L. , Yu , M. , Song , M. , Stefanidis , A. , Ji , Z. and Yang , C. ( 2016 ) 'Registration of long-strip terrestrial laser scanning point clouds using RANSAC and closed constraint adjustment' , Remote Sensing , 8 ( 4 ). DOI: 10 .3390/rs8040278.

Zong , Z. , Xia , Z. , Liu , H. , Li , Y. and Huang , X. ( 2016 ) 'Collapse Failure of Prestressed Concrete Continuous Rigid-Frame Bridge under Strong Earthquake Excitation: Testing and Simulation' , Journal of Bridge Engineering , 21 ( 9 ), p. 4016047 . DOI: 10 .1061/(ASCE)BE. 1943 - 5592 . 0000912 .