As an active sensor, light detection and ranging (LiDAR) has an important position in Simultaneous Localization and Mapping (SLAM). In particular, it is not affected by lighting conditions, so that it can work well at night. Infrared image is also less affected by illumination, so it can be well fused with LiDAR. Generally, the LiDAR SLAM system consists of a frontend odometer and a back-end optimization module. The back-end with loop closure detection plays an important role in systematically improving the positioning and mapping accuracy of LiDAR SLAM. However, LiDAR works at a single wavelength (such as 905nm), and few features are extracted, which limits the performance of loop closure detection and graphics optimization based on point cloud matching. In order to improve the performance of LiDAR SLAM, a SLAM back-end which is not affected by illumination conditions is proposed in this paper. This method combines infrared image and the geometric features of LiDAR as loop detection. Firstly, the bag of word (BOW) model, describing the visual similarities, was constructed to assist in the loop closure detection. Then, through the time interval of infrared image and the spatial distance of point cloud, the loop closure detection is verified, and accomplish graph optimization. We conducted experiments in different scenes at night to evaluate our method. The results show that the addition of infrared images effectively helps the loop closure detection and improves the performance of LiDAR SLAM.
With the coming of the "New Technology Revolution", human beings have higher and higher
requirements for automation, intelligence and even intelligence, and the demand for positioning
services is also increasing day by day. In outdoor environments, the global navigation satellite system
(GNSS) is a very mature localization technology that can provide accurate outdoor localization services
[
Graeter proposed the LiDAR-monocular visual odometry [
Most of the visual information used in the above scheme is RGB image. In the case of poor lighting
conditions such as at night, these schemes often have poor effects or even useless. Infrared image is
less affected by light and can be used normally at night. It can be well fused with LiDAR. At present,
scholars have few SLAM schemes for the fusion of infrared image and LiDAR. Pierre proposed a
SLAM method, which is able to solve various difficulties (lack of GPS signal, lighting conditions,
smoke) [
This paper constructs a LiDAR / infrared SLAM based on loop closure detection and global graph optimization to improve the accuracy of localization trajectories and the consistency of point cloud images in the night environment. In LiDAR / infrared SLAM system, a loop closure detection method based on infrared image information and point cloud re-matching is realized. We conducted experiments in different scenes at night to evaluate our method. The results show that the addition of infrared image effectively helps the loop closure detection and improves the SLAM performance of LiDAR in night environment.
optimization
In this section, we will introduce the system and structure of our method in detail. As shown in Figure 1, our method consists of sensor data, algorithm front-end and algorithm back-end.
The sensor data of our algorithm includes LiDAR point cloud data and infrared image data. LiDAR point cloud data is collected by 16-line LiDAR, and infrared image data is collected by Azure Kinect DK camera.
The front-end input of our algorithm is LiDAR sensor data. Through NDT-LOAM algorithm [
The technical route of loop closure detection is mainly based on geometry and appearance. The loop
closure detection method based on geometry refers to verifying whether there is a loop closure when
the platform returns to a previous position according to the front-end odometer or other known
information [
Infrared image and RGB image have certain similarity, they can reflect the structural information of the scene, so the principle of loop closure detection using the two devices is the same. However, compared with RGB images, infrared images have less information and poor quality. The accuracy and recall of loop matching of infrared image are worse than that of RGB image. Therefore, improving the accuracy and recall of infrared loop matching is an important problem that need to be solved urgently before we optimize the back-end. In order to improve the accuracy and recall of infrared loop matching, the loop closure detection method adopted in this paper combines geometric detection method and appearance detection method. The algorithm flow chart is shown in Figure 2.
Firstly, we use Bow model to test the similarity of infrared images and get the similarity score results
of each image. The program we use here is the open source library DBOW3
(https://github.com/rmsalinas/DBow3, accessed on 7 June 2021) to assist the implementation. However,
the characteristics of infrared image and RGB image are different. Direct loop closure detection will
lead to inaccurate loop closure candidates[
1) Because there is usually a standing time at the beginning and end of data acquisition, we eliminate the start and end 1 second data; 2) The time length threshold 2 of the loop, and the elapsed time of the loop is greater than this threshold; 3) The interval of each loop is 3. In each time interval 3, select the point with the highest similarity as the suspected loop closure frames. 4) The threshold L1 of the search area, and only the loop closure frames less than the threshold can be the potential loop closure frames; 5) The space interval threshold L2 of the loop, and the interval between two loops should be greater than this threshold; 6) The loop length threshold is L3, and the loop distance should be greater than this threshold; 7) After the above 6 conditions are met, the point cloud re-matching verification is carried out.
Match the current frame with the suspected loop closure frame one by one in order. If the mean square of the distance from the source point cloud to the target point cloud is lower than the threshold L4, the frame is the final loop closure frame of the current frame.
When detecting the loop closure, there are usually four conditions: true positives (TPs), false positives (FPs), true negatives (TNs), and false negatives (FNs). An FP indicates that the truth is not a loop but the algorithm determines that it is a loop. A TN indicates that the truth and algorithm detection are not loop closures. An FN indicates that the truth is a loop, but the algorithm determines that it is not a loop.
In our results, we want TP and TN to appear as much as possible, and FP and FN to appear as little as possible, or not at all. According to the frequency of the four possible results, we can calculate the accuracy rate and recall rate as two evaluation indexes:
With the accumulation of time, the trajectory of the robot will be longer and longer, and the scale of the map will continue to grow. Since the scanning space of LiDAR is always limited, the scale of point cloud or road sign can not grow indefinitely with the map, and the constraint relationship between the current frame and earlier historical data may no longer exist. In addition, there are errors in direct matching and feature point optimization, the cumulative error obtained at the front end of the algorithm will be larger and larger, and the inconsistency of the global map will be more and more obvious.
In order to improve the pose accuracy of odometer and ensure the quality of global point cloud map, we save the trajectory obtained from the front-end of the algorithm and construct a back-end global map optimization with only trajectory to reduce the cumulative error. The global pose map takes the pose of odometer as the node. The relative motion estimator between the two pose nodes obtained by point cloud matching is used as the constraint edge. Finally, the nonlinear least squares adjustment method is used to obtain the results with higher accuracy and stronger consistency.
To evaluate our method and compare with other approaches, we designed a series of experiments to compare the performance of NDT-LOAM and LI-SLAM in positioning and mapping. 3.1.
We use mobile robots for data acquisition. The mobile machine is loaded with a variety of sensors, including Azure Kinect DK camera and RS-LiDAR-16, as shown in Figure 3. Through these sensors, we can obtain data such as LiDAR and infrared images.
As shown in Figure 4. Experimental scene 1 was an underground parking lot of the Zhi Teng Building at Shenzhen University, covering a total distance of 203.245 m. The feature points of the scene are obvious, and there are no dramatic changes in the scene. Generally speaking, LiDAR has high accuracy in this scene. distance of 323.552m. In Figure 6, the red arrow indicates the approximate walking route of the robot. In the process of walking, the scenes that the robot passes through include railing scene, hall scene and narrow corridor scene. Generally speaking, there are few geometric features in the corridor, and the location accuracy of the laser radar is very poor.
We tested the data of these three experiments. Some important threshold parameters used in the experiment are set as follows:
Parameter 1, 2, 3 are 10s, 25s and 3s; Parameters L1, L2, L3 and L4 are 5m, 7m, 20m and 0.2m. 3.2.
At night, we conducted experiment 1, experiment 2 and experiment 3 in the above experimental scene 1, experimental scene 2 and experimental scene 3 respectively, and all experiments have loops. We compare the results with loops (LI-SLAM) and without loops (NDT-LOAM). The positioning accuracy results are shown in Table 1.
Table 1 Comparison of error results
Group #01 #02 #03
Distance(m) Amount of Detected Loop Closure 203.245 8 422.863 6 323.552 10
Accuracy (%) NDT-LOAM 100% 0.107 100% 1.735 100% 4.575
LI-SLAM 0.024 0.245 1.176
Experiment 1 is an underground parking lot scene, including many columns, beams and other conventional buildings, with good environmental perception structure. It can be seen from Table I. and Figure 7 that in this experiment, the positioning accuracy of NDT-LOAM has been relatively high, and the positioning error is 0.1072m. In this experiment, LI-SLAM detects 8 loop closures, with an accuracy of 100% and a positioning error of 0.0244m. Since the positioning accuracy of NDT-LOAM is relatively high, point cloud comparison is not done here. The experiment shows that the positioning accuracy of LI-SLAM is much higher than that of NDT-LOAM in the environment with simple scene and good LiDAR positioning and mapping effect. space is relatively wide, the feature points are relatively few, and there are several speed bumps. Figure 8 shows the positioning trajectory obtained by NDT-LOAM and LI-SLAM, in which the red circle represents the starting point of the trajectory. In Table 1 and Figure 7, it can be seen that in this experiment the positioning error of NDT-LOAM is 1.735m. The LI-SLAM algorithm proposed by us has a positioning error of 0.245m. Compared with the algorithm without loop closure optimization, the positioning accuracy of LI-SLAM has been greatly improved. Figure 9 (a) and Figure 9 (b) are the point cloud images obtained by NDT-LOAM and LI-SLAM respectively. It can be clearly seen that the mapping accuracy of LI-SLAM is higher than that of NDT-LOAM.
(a) (b) Figure 11: Point cloud maps of experimental scene3.
The scene of experiment 3 is complex. The scene includes narrow corridors, halls and railings. Especially in narrow corridors, the structure of environmental perception is poor. in Table 1 and Figure 8, It can be seen that the positioning accuracy of LI-SLAM is significantly higher than that of NDTLOAM. The positioning accuracy of NDT-LOAM is relatively poor, and the positioning error is 4.575m. The LI-SLAM proposed in this paper detects 8 loop closures, with an accuracy of 100% and a positioning error of 1.1758m. The experiment shows that the positioning accuracy of LI-SLAM is much higher than that of NDT-LOAM in the environment of complex scene and poor effect of LiDAR positioning and mapping.
Figure 11 is the point cloud diagram of experiment 3, in which figure 11 (a) is the point cloud diagram obtained by NDT-LOAM and Figure 11 (b) is the point cloud diagram obtained by LI-SLAM. From the point cloud map, it can be clearly seen that the point cloud map obtained by NDT-LOAM has wall ghosting and bending, which does not appear in LI-SLAM. This shows that, the mapping accuracy of LI-SLAM is significantly higher than that of NDT-LOAM in this environment.
In this paper, we propose a LiDAR SLAM back-end used at night. This method combines LiDAR and infrared image. First, we use NDT-LOAM as the front-end to get the front-end odometer data. Then we use BOW to score the similarity of infrared images. Then we combine the infrared image, front-end odometer and LIDAR point cloud data to determine the loop closure. Finally, we optimize the frontend results and build the global map.
The main innovation of this paper is a new method of loop closure detection based on infrared image and LiDAR. This method is a good fusion of image and LiDAR information, and it can work normally in dark conditions. It judges the loop closure points in time and space and identifies the final loop closure. We selected several different scenes at night to verify our method. The results show that compared with the SLAM Based on LiDAR, our LiDAR/Infrared image SLAM with loop closure detection and global graphics optimization has achieved better performance, including better mapping and point location results. However, in some cases, the information extracted from infrared images is too little. Therefore, there is still room for improvement in the enhancement of infrared image information. In the future, we plan to use deep learning and other methods to enhance the infrared image information. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003)(Cat. No. 03CH37453). IEEE, 2003, 1: 206-211. [18] Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE transactions on robotics, 2015, 31(5): 1147-1163. [19] Latif Y, Cadena C, Neira J. Robust loop closing over time for pose graph SLAM[J]. The
International Journal of Robotics Research, 2013, 32(14): 1611-1626. [20] Ulrich I, Nourbakhsh I. Appearance-based place recognition for topological localization[C]//Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065). Ieee, 2000, 2: 10231029.