Automatic sensor calibration is ubiquitous for robots and autonomous vehicles. Traditional target-based calibration methods typically require multiple registrations of an object in the field of view. However, by moving the object continuously while recording, the 3D positions of the target generate a path that may improve the calibration quality. In this work, we propose a novel curve-based trajectory registration method to enhance calibration accuracy with spherical and checkerboard targets. This approach supplements the existing target position point set with estimated intermediate points after applying spline fitting techniques to the measurements. The curvature of the movement is aligned between the point sets resulting in reduced error between the registered point clouds and more accurate extrinsic calibration parameters. We validate these properties in indoor and outdoor real-world scenarios.
We provide an overview on the synchronization system between the camera and the LiDAR as shown in Fig. 4. A Velodyne VLP-16 LiDAR device is spinning with a 1200 RPM (rotation per minute) while an Arduino Triggerbox triggers the camera with 4 FPS. If one trigger sign arrives, the camera mounted on the same vehicle exposes for a varying duration, at most for 0.04 seconds. Then the picture is taken by a global shutter, and the system triggers the LiDAR to save a synchronized turnaround. The
LiDAR saves the data packages of every 15° rotation slice. When it gets the trigger, it finishes the current chunk of data, labels it as the end of the scan to be saved, and identifies the last 360° rotation as one complete synchronized turnaround to the current image. However, in the case of a moving object, the worst case is when the object appears at the beginning of the point cloud, which may cause an error 0.05 at maximum. If the target object moves with a typical 0.2–0.4 / velocity, it causes 1–2 error. Therefore, the point pairwise registration may be inaccurate, while the proposed solution tries to deal with this particular problem.
This section introduces the evaluated variations of the proposed pose estimation described in Fig. 1 using interpolated points on the estimated trajectory and point cloud trajectory registration. Let ℒ and denote the point sets of estimated sphere centers from LiDAR data and camera images, respectively. The final task is to find the rigid transformation matrix T = [︀ R|t︀] constructed by a rotation matrix R and a translation vector t. In this paper, Rℒ and tℒ describe the south rotation and translation from the camera frame to the LiDAR frame. In the particular case of sphere-based estimation, ≥ 4 point pairs are required.
First, let us consider the point pair solution as shown
in Alg.1, based on the method of Arun et al. [
Rp − tℒ ⃦⃦ 2 , ⃦ 2 (1)
0.05 s (full turnaround)
Saved Synchronized Turnaround 1
sure1 xpo
E max. 0.04 s Trigger Sign 1 where pℒ and p denotes the -th point pair of the Li- interpolation method and the sampling frequency to add DAR and the camera data. The translation is eliminated new synthetic points to the point sets. by choosing the point set barycenters as the coordinate To find the best method, the following requirements system origins in the two images. The rotation can be must be met: the internal control points have to be inderived using singular value decomposition (SVD) on terpolated while the segments tightly follow the knots the matrix H constructed from the point sets. If the without self-intersection between the points. Moreover, point pairs are perfectly synchronized and noiseless, this the final segment sequence has to be continuous and easy method guarantees optimal results. However, in the pres- to evaluate at any point. ence of noise, such as due to synchronization errors as we Linear interpolation (LI). A naive solution is the described in Sec. 1.1, Eq. (1) no longer provides optimal linear interpolation between the neighboring points of solutions. the measurement to reconstruct the path with line seg
The improvements we propose are described in Alg. 2. ments. Derivatives with a large magnitude of the curve
The initial step is the pre-filtering of the point sets be- are not expected. As rapid changes in the trajectory are
cause noisy data may occur. This is based on scaled not expected, the line-based approach is a good initial
median absolute deviation: step; however, its precision depends on the target object
movement speed and complexity. If two adjacent control
= · median (| − median(x)|) , (2) points are denoted by p and p+1 where ≤ − 1, is
the number of the points, and ∈ [
Catmull–Rom spline (CRS). The second approach
is a curve-based solution applying cubic centripetal
Catmull–Rom spline [
(3)
Algorithm 1 Pose estimation with point pair registration – PPR [
+1 − − 1 Based on the points p and the derivatives m calculated by (4), the Hermite curve is fitted data pair-wise. The equation of the spline is written as c() = 2 − 2 − 1 b1 + − 1 b2, 2 − 1 b1 = b2 = a1 = a2 = a3 = 2 − 2 − 0 3 − 3 − 1 1 − 1 − 0 2 − 2 − 1 3 − 3 − 2 a1 + a2 + p0 + p1 + p2 + − 0 a2, 2 − 0 − 1 a3, 3 − 1 − 0 p1, 1 − 0 − 1 p2, 2 − 1 − 2 p3.
3 − 2 The parameter ∈ [1, 2] interpolates between p1 and p2. If the point coordinates are denoted by p = ︀[ , , ]︀ , then the knot parameterization of CRS is
Kochanek–Bartels spline (KBS). The third trajec(4) tory approximation curve we evaluate is the Kochanek–
Bartels spline [
2 (p+1 − p), (1 − )(1 + ℬ)(1 − ) 2 (p+1 − p)+
(8) (1 − )(1 − ℬ )(1 + ) (p+2 − p+1). (9)
2
The final curve segment can be defined as a cubic
Hermite spline between the internal control points p
and p+1, for any ∈ [
(10) where and the scale = 0.5 describes the centripetal spline.
Point clouds after registration
To this end, we evaluate the curve segment at every step distance of /, and insert those into the trajectory path. This, however, yields trajectory point sets with diferent sizes; hence, point pair registration cannot be applied between them in the previous form.
After the spline fitting, the point pairs lost importance due to the quasi-continuous point set and varying sampling between the control points. Classical point cloud registration methods can be applied to dense 3D paths.
One of them is the iterative closest point (ICP)
algorithm [
We examined whether the registration results could be more precise using the new approach, and what method is the most suitable among the alternate versions. Furthermore, we tested the parameter setting for the sampling interval. The tests were implemented in MATLAB. To compare the results, we computed the root mean square error (ℛℳℰ ) of the Euclidean distance between the aligned point clouds. The sensors of the vehicle (see in Fig. 3 and the synchronization in Sec. 1.1) captured camera-LiDAR video sequences to test real-world scenarios.
The input of the algorithms is the 3D point sequences
of sphere centers. The test input for illustration is
visualized in Fig. 6 containing 30 positions per sensor which
means a 7.5 sec. long video sequence in an outdoor
environment in a parking lot. We processed this data feed as
written in [
At first, we compared the trajectory registration methods discussed in Sec. 2.2 without estimating the moveTrajectory estimation 0.1 0.5 1 ment and any interpolated points. In Fig 5, the results for the test case in Fig. 6 are plotted. The ℛℳℰ of the ICP algorithm is 0.0155 with 0.0114 standard deviation while the PPR and CPD algorithms gave the same results, 0.0110 with 0.0083 standard deviation. Based on the experiments and the fact that applying PPR with point clouds of diferent sizes is not feasible, we recommend using the CPD.
We analyzed the interpolation methods and whether the -based sampling gives the sought ideal density. We examined several setups in the same example in Fig. 7 and Fig. 1. In conclusion, one position per 0.25 is a suficient sampling parameter. In this particular case, linear interpolation was suitable for the problem due to the stretched curvatures in the movement. Nonetheless, other scenarios require more adaptive approaches like CRS and KBS. Furthermore, the compared calibration results in Fig. 8 on the right suggest that the translation of LI is more accurate, but the rotation is expected as in the case of CRS and KBS.
Based on the test scenarios, the proposed settings of proposed Alg. 2 are the following. Tuning the sampling interval to 0.25 gives suficiently accurate results; besides, more dense sampling has no significant efect. Due to a linear-like input scenario, the linear approach performs the best in the exhibited use case, but the results are similarly acceptable with CRS and KBS. Finally, trajectory registration using CPD is the most efective choice. Applying this settings, the ℛℳℰ of the analyzed case was reduced from 0.0110 (PPR, no interpolated trajectory) to 0.0053 (CPD, = 0.25 , LI).
Synthetic tests provide the opportunity for direct comparison of the estimated calibration parameters to ground truth data. Based on early-stage results, the main open question is the best interpolation method for trajectory estimation which largely depends on the specific scenario. Paths with greater curvature give a better result for the rigid transformation with CRS and KBS. During the further refinement of the paper, we will analyze in more depth how the synthetized movement afects the precision of the proposed methods.
This paper argued how trajectory registration could improve extrinsic sensor calibration algorithms if we can extract 3D positions from the input data, e.g., based on a specific target object. The novelty of the method is the controlled path of the object and the generated dense point set by spline fitting and interpolation followed by point cloud registration. The algorithm guarantees sample continuity and speeds up the measurement, as one video sequence is suficient. The method handles the noise in the input images or the LiDAR scans independently owing to the point cloud-based approach which makes it more robust to the outliers.
There are some open questions regarding to the object path and the setup of the measurements. We would like to validate the better accuracy of the calibration via sythetic tests and analyze whether any ideal trajectory exist which makes the results more accurate. Moreover, we examine the efect of the noise in the 3D positions. Besides the spherical setup, chessboard-based approach could be also analyzed where an additional problem is the interpolation of the rotation, as not only one position but the four corner points can be detected in every frame. In the future, another application of the project can be the trajectory estimation of autonomous vehicles using multi-sensor systems.