In last few years, the significance of environmental monitoring for natural hazards prevention and mitigation has been constantly growing. Responsiveness requirements along with an increased amount of data coming from ambient sensors led to the necessity of automated systems able to detect critical situations and alert authorities.
In this work we address the problem of landslide monitoring, that means detecting the flow of landslide material, a motion that is very limited: usually only few meters over several weeks. The slipping of the landslide material is often monitored analyzing image sequences exploiting optical flow techniques. One of the main limitations of such process is that the direction and intensity of the flow are the projection in the camera plane of the real world flows. Therefore, in order to obtain a correct estimation of the material motion, flow gradients must be back-projected to the landslide 3D model. Unfortunately, this 3D model is hard to obtain due to the wideness of the monitored area.
Several works presented in the literature rely on expensive laser-scanner or aerial photogrammetry systems. Differently, our work propose an innovative stereo vision system that requires only two cameras, which ultimately makes it a low-cost alternative for large environmental 3D reconstruction. Stereo vision is one of the most widely used techniques for outdoor 3D reconstruction [1], especially in low cost solutions. Stereo vision only requires two cameras and can retrieve the distance of an area observed by both cameras from its slightly different appearance in the two images (see fig. 1). In our case, the large distance of monitored areas and the presence of adverse natural conditions, like heavy wind or bad weather, make the calibration of a stereo vision system an hard task. In order to maintain a good reconstruction quality at farthest monitored areas, the distance between monitoring cameras (baseline, see sec. 2) needs to be a lot higher than what is commonly the case when calibrating a stereo camera pair. Indeed, the common way to calibrate a stereo vision system and find its extrinsic patameters (i.e. cameras mutual position) has been proposed by [2] and makes use of a small pattern with known geometry; however, this pattern needs to be observed by both camera during the calibration process making this technique infeasible when cameras are too far from each other. In our work we deal this issue through a robust multiview calibration system which lesser constraints in terms of maximum baseline allow us to extend the distance between cameras without loss of precision in the calibration process.
The main issue connected to the reconstruction of wide areas is the sensitivity of the stereo matching with respect to the distance of the target area. In stereo vision, the error in the detected distance z is related to the quantization error in digital images [3] and is computed by means of the following equation [4]:
where b [m] is the baseline between left and right cameras, f [px] is the focal length, z [m] is the target distance and d [px] is the quantization in the disparity map (usually one pixel). Equation 1shows that there is a quadratic dependency of the depth error to the distance between camera and observed region. Since farthest areas of the landslide are located at more than 700 m from the cameras, a wide baseline is needed in order to keep low the depth error. In table 2 estimated depth errors with respect to target distance and baseline are reported for our camera installation (18M px and 30 mm focal length). 1. Estimated depth errors with respect to the target distance and cameras baseline.
To correctly process a couple of stereo images and obtain observed area distances, the roto-translation between the two stereo cameras (extrinsic parameters) needs to be known. The calibration process is usually performed observing a pattern with a known geometry [2] but this simple method is not applicable in our case: the wide baseline and the terrain conformation do not allow the observation of the calibration pattern from both cameras.
We obtained a good estimation of the extrinsic parameters exploiting a multiview calibration [5]. This technique makes use of a large set of images of the same area taken from several viewpoints. Visual correspondences between all possible couples of images are matched in order to impose constraints on the roto-translation between each viewpoint couple; this way it is possible to perform calibration exploiting the features available in the framed scene without the need of dedicated patterns. In our work we collected a large set of images of the landslide, taken from a number of different viewpoints; we then added such images to those acquired by the stereo camera pair. Exploiting the multi-view calibration it is possible to obtain the mutual position between all couples of views, including the extrinsic calibration of the stereo couple.
Once obtained the extrinsic calibration of the stereo system, images taken from the two cameras are rectified (see fig. 2) and processed by a stereo matching algorithm called Semi Global Block Matching [6] in order to produce a disparity map. From the disparity map we retrieve the distance of each observed point and create a dense 3D point cloud representing the landslide reconstruction (see fig. 3).
The 3D reconstruction of the landslide allows us to precisely evaluate sliding of the ground. We detect particle flows in the image sequence using Normalized Cross-Correlation and then back-projecting the 2D flow onto the 3D landslide model, obtaining the motion flow of the rocks. The monitoring system proposed in this paper is completely autonomous and scalable and it is designed to issue an alert when the sliding effect exceeds a given threshold.
As a future work, we will employ a continuous camera calibration algorithm [7] to prevent the system to lose its calibration. This way it will be possible to obtain good performance over time, since the capability of the multiview stereo calibration of providing good estimation of extrinsic parameters is strongly dependent on the mutual position of the cameras.