Automated Landslide Monitoring through a Low-Cost Stereo Vision System

Abstract

In this paper we introduce an inexpensive yet e cient photogrammetry system that takes advantage of state of art computer vision techniques to monitor large natural environments. Speci cally, our system provides a precise evaluation of the terrain ow in wide landslides through optical ow applied to 2D image sequences and a back-projection of the resulting motion gradients to a 3D model of the landslide. Providing such a wide 3D model is one of the key issues and is addressed relying to a wide baseline stereo vision system. To initialize the stereo vision system, we propose an e ective multiview calibration process.

Keywords

• Photogrammetry
• Stereo Vision
• Multiview Calibration

Introduction

In last few years, the signi cance of environmental monitoring for natural hazards prevention and mitigation has been constantly growing. Responsiveness requirements along with an increased amount of data coming from ambient sensors led to the necessity of automated systems able to detect critical situations and alert authorities.

In this work we address the problem of landslide monitoring, that means detecting the ow of landslide material, a motion that is very limited: usually only few meters over several weeks. The slipping of the landslide material is often monitored analyzing image sequences exploiting optical ow techniques. One of the main limitations of such process is that the direction and intensity of the ow are the projection in the camera plane of the real world ows. Therefore, in order to obtain a correct estimation of the material motion, ow gradients must be back-projected to the landslide 3D model. Unfortunately, this 3D model is hard to obtain due to the wideness of the monitored area.

Several works presented in the literature rely on expensive laser-scanner or aerial photogrammetry systems. Di erently, our work propose an innovative stereo vision system that requires only two cameras, which ultimately makes it a low-cost alternative for large environmental 3D reconstruction. Stereo vision is one of the most widely used techniques for outdoor 3D reconstruction [ 1 ], especially in low cost solutions. Stereo vision only requires two cameras and can retrieve the distance of an area observed by both cameras from its slightly different appearance in the two images (see g. 1). In our case, the large distance of monitored areas and the presence of adverse natural conditions, like heavy wind or bad weather, make the calibration of a stereo vision system an hard task. In order to maintain a good reconstruction quality at farthest monitored areas, the distance between monitoring cameras (baseline, see sec. 2) needs to be a lot higher than what is commonly the case when calibrating a stereo camera pair. Indeed, the common way to calibrate a stereo vision system and nd its extrinsic patameters (i.e. cameras mutual position) has been proposed by [ 2 ] and makes use of a small pattern with known geometry; however, this pattern needs to be observed by both camera during the calibration process making this technique infeasible when cameras are too far from each other. In our work we deal this issue through a robust multiview calibration system which lesser constraints in terms of maximum baseline allow us to extend the distance between cameras without loss of precision in the calibration process. The main issue connected to the reconstruction of wide areas is the sensitivity of the stereo matching with respect to the distance of the target area. In stereo vision, the error in the detected distance z is related to the quantization error in digital images [ 3 ] and is computed by means of the following equation [ 4 ]: z2 z = d bf ; (1) where b [m] is the baseline between left and right cameras, f [px] is the focal length, z [m] is the target distance and d [px] is the quantization in the disparity map (usually one pixel).

Equation 1 shows that there is a quadratic dependency of the depth error to the distance between camera and observed region. Since farthest areas of the landslide are located at more than 700 m from the cameras, a wide baseline is needed in order to keep low the depth error. In table 2 estimated depth errors with respect to target distance and baseline are reported for our camera installation (18M px and 30 mm focal length).

distance [m] 10 12 To correctly process a couple of stereo images and obtain observed area distances, the roto-translation between the two stereo cameras (extrinsic parameters) needs to be known. The calibration process is usually performed observing a pattern with a known geometry [ 2 ] but this simple method is not applicable in our case: the wide baseline and the terrain conformation do not allow the observation of the calibration pattern from both cameras.

We obtained a good estimation of the extrinsic parameters exploiting a multiview calibration [ 5 ]. This technique makes use of a large set of images of the same area taken from several viewpoints. Visual correspondences between all possible couples of images are matched in order to impose constraints on the roto-translation between each viewpoint couple; this way it is possible to perform calibration exploiting the features available in the framed scene without the need of dedicated patterns. In our work we collected a large set of images of the landslide, taken from a number of di erent viewpoints; we then added such images to those acquired by the stereo camera pair. Exploiting the multi-view calibration it is possible to obtain the mutual position between all couples of views, including the extrinsic calibration of the stereo couple. 2.2

Landslide 3D Reconstruction Once obtained the extrinsic calibration of the stereo system, images taken from the two cameras are recti ed (see g. 2) and processed by a stereo matching algorithm called Semi Global Block Matching [ 6 ] in order to produce a disparity map. From the disparity map we retrieve the distance of each observed point and create a dense 3D point cloud representing the landslide reconstruction (see g. 3).

Results

The 3D reconstruction of the landslide allows us to precisely evaluate sliding of the ground. We detect particle ows in the image sequence using Normalized Cross-Correlation and then back-projecting the 2D ow onto the 3D landslide model, obtaining the motion ow of the rocks. The monitoring system proposed in this paper is completely autonomous and scalable and it is designed to issue an alert when the sliding e ect exceeds a given threshold.

As a future work, we will employ a continuous camera calibration algorithm [ 7 ] to prevent the system to lose its calibration. This way it will be possible to obtain good performance over time, since the capability of the multiview stereo calibration of providing good estimation of extrinsic parameters is strongly dependent on the mutual position of the cameras.

References

1. Strecha , C. , Von Hansen , W., Van Gool , L. , Fua , P. , Thoennessen , U. : On benchmarking camera calibration and multi-view stereo for high resolution imagery . In: Computer Vision and Pattern Recognition , 2008 . CVPR 2008 . IEEE Conference on. ( 2008 ) 1{ 8

2. Zhang , Z. : A exible new technique for camera calibration . Pattern Analysis and Machine Intelligence , IEEE Transactions on 22 ( 11 ) ( 2000 ) 1330 { 1334

3 . Chang , C. , Chatterjee , S. : Quantization error analysis in stereo vision . In: Signals, Systems and Computers , 1992 . 1992 Conference Record of The Twenty-Sixth Asilomar Conference on. ( 1992 ) 1037 { 1041 vol.2

4. Gallup , D. , Frahm , J.M. , Mordohai , P. , Pollefeys , M. : Variable baseline/resolution stereo . In: Computer Vision and Pattern Recognition , 2008 . CVPR 2008 . IEEE Conference on. ( 2008 ) 1{ 8

5. Hiep , V.H. , Keriven , R. , Labatut , P. , Pons , J.P. : Towards high-resolution large-scale multi-view stereo . In: Computer Vision and Pattern Recognition , 2009 . CVPR 2009 . IEEE Conference on. ( 2009 ) 1430 { 1437

6. Hirschmuller , H.: Stereo processing by semiglobal matching and mutual information . Pattern Analysis and Machine Intelligence , IEEE Transactions on 30(2) ( 2008 ) 328 { 341

7. Dang , T. , Ho mann, C., Stiller , C. : Continuous stereo self-calibration by camera parameter tracking . Image Processing, IEEE Transactions on 18(7) ( 2009 ) 1536 { 1550