Point clouds registration based on the point-to-plane approach for orthogonal transformations AMakovetskii Chelyabinsk State University
Bratiev Kashirinykh str. 129 454001 Chelyabinsk Russia
SVoronin Chelyabinsk State University
Bratiev Kashirinykh str. 129 454001 Chelyabinsk Russia
VKober Department of Computer Science CICESE
Carretera Ensenada-Tijuana 3918 22860 Ensenada B.C., Mexico
AVoronin Chelyabinsk State University
Bratiev Kashirinykh str. 129 454001 Chelyabinsk Russia
DTihonkih Chelyabinsk State University
Bratiev Kashirinykh str. 129 454001 Chelyabinsk Russia
IV International Conference on "Information Technology and Nanotechnology" (ITNT
2018
Point clouds registration based on the point-to-plane approach for orthogonal transformations F4033D4148EE427F5F720B60B84F31F3 GROBID - A machine learning software for extracting information from scholarly documents

The most popular algorithm for aligning of 3D point data is the Iterative Closest Point (ICP). This paper proposes a new algorithm for orthogonal registration of point clouds based on the point-to-plane ICP algorithm for affine transformation. At each iterative step of the algorithm, an approximation of the closed-form solution for the orthogonal transformation is derived.
Introduction

The Iterative Closest Point (ICP) algorithm [1][2][3][4][5] has become the dominant method for aligning three dimensional models based purely on the geometry. For alignment it is necessary to find a geometric transformation that connects two point clouds in ℝ 3 by the best way with respect to the 𝐿 2 norm. The ICP algorithm consists of two main stages:

1. Searching of corresponding points (pairs) in two clouds; 2. Minimizing the error metric (variational subproblem of the ICP).

There are two basic approaches to choosing the error metric for pairs of points. Within the point-topoint approach [1], the distance between the elements of the pair in ℝ 3 is used. Within the point-toplane approach [2] the distance between the point of the first cloud and the tangent plane to the corresponding point of the second cloud is used.

The key point [6] of the ICP algorithm is the search of either an orthogonal or affine transformations, best in the sense of a quadratic metric that combines two point clouds with a given correspondence between points (the variational subproblem of the ICP algorithm).

For the point-to-point metric in the case of orthogonal transformations, the solution in a closedform was obtained by Horn [7,8]. The solution [7] is based on the use of quaternions, whereas the solution [8] uses orthogonal matrices. The solutions are linear in time with respect to the number of point pairs. The original ICP algorithm is widely used for the rigid objects registration, but it does not work well for the case of the non-rigid objects. An extension of the ICP algorithm is proposed [9], using scaling in addition to rotation and translation. A generalization of this algorithm to the case of an arbitrary affine transformation was done [10,11]. A closed-form solution to the point-to-point problem was derived [12][13][14].

The above mentioned approaches for solving the variational subproblem of the ICP algorithm are based on the point-to-point metric. The point-to-plane metric has been shown to perform better than the point-point metric in terms of accuracy and convergence rate [15]. A closed-form solution to the point-to-plane case for orthogonal transformations is an open problem. Instead, iterative methods based on the linear least-squares optimization or closed-form methods for small angles only are often used [12]. Iterative solutions require an initial approximate estimate of the transformation parameters, and the iterations might converge slowly, converge to a local optimum or not converge at all.

In [16,17] a closed-form solution to the point-to-plane problem for an arbitrary affine transformation is proposed. The affine approach works well when the correspondence between point clouds is good. In this case, the affine point-to-plane method precisely reconstructs original geometric transformation for arbitrary affine transformations, in particular for orthogonal transformations [16,17]. When a correspondence between clouds is not sufficiently good, the affine approach cannot reconstructs an original orthogonal transformation.

In this paper, we propose an approximation of a closed-form solution to the point-to-plane problem for orthogonal transformation. The method is based on the closed-form solution for the affine point-toplane problem [16,17], matrix polar decomposition and the Horn's method for calculating the nearest orthonormal matrix [8]. The proposed method does not require an initial approximate estimate. Computer simulation results are provided to illustrate the performance of the proposed method of solving the minimization problem.

Closed-form solution for affine point-to-plane problem

Let 𝑃 = {𝑝 1 , … , 𝑝 𝑛 } be a source point cloud, and 𝑄 = {π‘ž 1 , … , π‘ž 𝑛 } be a destination point cloud in ℝ 3 . Suppose that the relationship between points in 𝑃 and 𝑄 is given in such a manner that for each point 𝑝 𝑖 exists a corresponding point π‘ž 𝑖 . The ICP algorithm is commonly considered as a geometrical transformation for rigid objects mapping 𝑃 to 𝑄 𝑅𝑝 𝑖 + 𝑑,

where 𝑅 is a rotation matrix, 𝑑 is a translation vector, 𝑖 = 1, … , 𝑛.

The group of affine transformations in the dimension of three has 12 generators. It means that the affine transformation in the dimension of three is a function of 12 variables. Let us consider the ICP variational problem for an arbitrary affine transformation in the point-to-plane case. Denote by 𝑆(𝑄) a surface constructed from the cloud 𝑄, by 𝑇(π‘ž 𝑖 ) denote a tangent plane of 𝑆(𝑄) at point π‘ž 𝑖 . Let 𝐽(𝐴, 𝑇) be the following function:

𝐽(𝐴) = βˆ‘ (< 𝐴 𝑝 𝑖 βˆ’ π‘ž 𝑖 , 𝑛 𝑖 > ) 2 𝑛 𝑖=1 ,

where <β€’,β€’> denotes the inner product, 𝐴 is a matrix of an affine transformation in the homogenous coordinates:

𝐴 = ( π‘Ž 11 π‘Ž 12 π‘Ž 13 𝑑 1 π‘Ž 21 π‘Ž 22 π‘Ž 23 𝑑 2 π‘Ž 31 π‘Ž 32 π‘Ž 33 𝑑 3 0 0 0 1 ),

𝑝 𝑖 is a point from the cloud 𝑃, 𝑛 𝑖 is the unitary normal for 𝑇(π‘ž 𝑖 )

𝑝 𝑖 = ( 𝑝 1 𝑖 𝑝 2 𝑖 𝑝 3 𝑖 1 )

,

𝑛 𝑖 = ( 𝑛 1 𝑖 𝑛 2 𝑖 𝑛 3 𝑖 0 ) . ()

The ICP variational problem can be stated as follows: arg π‘šπ‘–π‘› 𝐴 𝐽(𝐴) .

(5) The solution of the problem ( 5) is given by the following way [16,17]: The reconstructed affine transform is done by the following formula:

𝑀𝐴 = 𝐢. (𝐴 = 𝑀 βˆ’1 𝐢.
Polar decomposition and orthogonal transformations

A square matrix M can be decomposed into the product of an orthonormal matrix R and a positive semi-definite matrix S [5]. The matrix S is always uniquely determined. The matrix R is uniquely determined when M is nonsingular. When M is nonsingular, we can actually write directly [8] 𝑀 = 𝑅𝑆,

𝑅 = 𝑀(𝑀 𝑑 𝑀) βˆ’ 1 2 .

(18) The matrix 𝑀 𝑑 𝑀 is positive semi-definite and symmetric. The orthogonal matrix 𝑅 in (18) can be computed by the following way [8]:

𝑅 = 𝑀𝐢 ( 1 βˆšπœ† 1 0 0 0 1 βˆšπœ† 2 0 0 0 1 βˆšπœ† 3 ) 𝐢 𝑑 , ()

where 𝐢 is orthogonal matrix consisting of columns, that are eigenvectors of the matrix 𝑀 𝑑 𝑀.

Numbers πœ† 𝑖 , 𝑖 = 1, … ,3, are eigenvalues of the matrix 𝑀 𝑑 𝑀. The formula (18) also defines [8] a nearest orthogonal matrix 𝑅 for the nonsingular matrix 𝑀. It means that the formula (18) describes the projection from the group 𝑆𝐿(3) to the subgroup 𝑆𝑂(3).

Projection on 𝑺𝑢(πŸ‘)

For approximation of the exact solution of the problem (5) we propose the following method. At each step of the ICP algorithm, we project a top-left submatrix 𝐴′ (size of 3 Γ— 3) of a matrix 𝐴 of an affine transform, computed by the formula ( 16), to 𝑆𝑂(3) by the formula (19). After that it is necessary to recalculate a translation 𝑑 = (𝑑 1 , 𝑑 2 , 𝑑 3 ) 𝑑 . Denote by 𝑅 a result of projection of a top-left submatrix 3 Γ— 3 of a matrix 𝐴 to 𝑆𝑂(3). Denote by 𝑁 the following matrix 𝑛 Γ— 3:

𝑁 = ( 𝑛 1 1 𝑛 2 1 𝑛 3 1 … 𝑛 1 𝑛 𝑛 2 𝑛 𝑛 1 𝑛 ),
Computer simulation
We consider two variants of the ICP algorithm here

The first is point-to-point ICP based on Horn algorithm. The second is point-to-plane ICP based on the proposed approximation of an exact solution of the variational problem. Other elements of ICP algorithm are same.

Let 𝑃 be the cloud consisting of 34817 points, see figure 1 (blue colour)

The cloud 𝑄 (green colour) is obtained from 𝑃 by the orthogonal transformation 𝑄 = 𝑇 1 * 𝑃, where 𝑇 1 is given by 𝑇 1 = ( 1.00000 0.00000 0.00000 3.10000 0.00000 0.83867 βˆ’0.54464 1.13270 0.00000 0.54464 0.83867 1.92795 0.00000 0.00000 0.00000 1.00000

).

Computed by the proposed method transformation 𝑀 1 is given as

𝑀 1 = (

1.00000 0.00000 0.00000 3.10000 0.00000 0.83867 βˆ’0.54464 1.13270 0.00000 0.54464 0.83867 1.92795 0.00000 0.00000 0.00000 1.00000

).

The reconstructed by the point-to-point ICP geometrical transformation has the same matrix. The point-to-point ICP method converges in 31 iterations, processing time 1745 milliseconds. The proposed ICP method converges in 10 iterations, processing time 913 milliseconds.

Figure 3 shows the clouds 𝑃 (blue) and 𝑄 (green), figure 4 shows the clouds 𝑃′ = 𝑀 1 β€’ 𝑃 (blue) and 𝑄 (green) together. ).

Computed by the proposed method transformation 𝑀 2 is given as ).

The reconstructed by the point-to-point ICP geometrical transformation has the same matrix. The point-to-point ICP method converges in 41 iterations, processing time 2458 milliseconds. The proposed ICP method converges in 16 iterations, processing time 1491 milliseconds. Figure 5 shows the clouds 𝑃 (blue) and 𝑄 (green), figure 6 shows the clouds 𝑃′ = 𝑀 2 β€’ 𝑃 (blue) and 𝑄 (green) together. ).

Computed by the proposed method transformation 𝑀 1 is given as ).

The reconstructed by the point-to-point ICP geometrical transformation has the same matrix. The point-to-point ICP method converges in 19 iterations, processing time 984 milliseconds. The proposed ICP method converges in 9 iterations, processing time 747 milliseconds. Figure 7 shows the clouds 𝑃 (blue) and 𝑄 (green), figure 8 shows the clouds 𝑃′ = 𝑀 3 β€’ 𝑃 (blue) and 𝑄 (green) together. ).

Computed by the proposed method transformation 𝑀 4 is given as ).

The reconstructed by the point-to-point ICP geometrical transformation has the same matrix. The point-to-point ICP method converges in 24 iterations, processing time 6316 milliseconds. The proposed ICP method converges in 16 iterations, processing time 5792 milliseconds. Figure 9 shows the clouds 𝑃 (blue) and 𝑄 (green), figure 10 shows the clouds 𝑃′ = 𝑀 4 β€’ 𝑃 (blue) and 𝑄 (green) together.

Conclusion

In this paper, we revised error minimizing steps of the ICP algorithm. A new algorithm for orthogonal registration of point clouds based on the point-to-plane ICP algorithm for affine transformation is proposed. At each iterative step of the algorithm, an approximation of the closed-form solution for the orthogonal transformation is derived.

Figure 1 .Figure 1. Illustration of projection of the top-left submatrix 𝐴′ onto 𝑆𝑂 3 .
Figure 2 .Figure 2. Block-diagram of the proposed algorithm.
Figure 1Figure 1 illustrates projection of the top-left submatrix 𝐴′ of the matrix 𝐴 of the affine transform onto submanifold 𝑆𝑂 3 . The block-diagram of the proposed algorithm as part of the ICP algorithm is shown in Figure 2.
Figure 3 .Figure 3. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 4. Cloud 𝑃′ = 𝑀 1 β€’ 𝑃 (blue), cloud 𝑄 (green).
Figure 4 .Figure 3. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 4. Cloud 𝑃′ = 𝑀 1 β€’ 𝑃 (blue), cloud 𝑄 (green). 5.1.2. Let 𝑃 be the cloud consisting of 34817 points, see figure 3 (blue colour) The cloud 𝑄 (green colour) is obtained from 𝑃 by the orthogonal transformation 𝑄 = 𝑇 2 * 𝑃, where 𝑇 1 is given by
Figure 5 .Figure 5. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 6. Cloud 𝑃′ = 𝑀 2 β€’ 𝑃 (blue), cloud 𝑄 (green).
Figure 6 .Figure 5. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 6. Cloud 𝑃′ = 𝑀 2 β€’ 𝑃 (blue), cloud 𝑄 (green).
5. 1 . 3 .Let 𝑃 be the cloud consisting of 34817 points, see figure5(blue colour)The cloud 𝑄 (green colour) is obtained from 𝑃 by the orthogonal transformation 𝑄 = 𝑇 3 * 𝑃, where 𝑇 3 is given by 𝑇 3
Figure 7 .Figure 7. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 8. Cloud 𝑃′ = 𝑀 3 β€’ 𝑃 (blue), cloud 𝑄 (green).
Figure 8 .Figure 7. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 8. Cloud 𝑃′ = 𝑀 3 β€’ 𝑃 (blue), cloud 𝑄 (green).
5. 1 . 4 .Let 𝑃 be the cloud consisting of 106289 points, see figure7(blue colour)The cloud 𝑄 (green colour) is obtained from 𝑃 by the orthogonal transformation 𝑄 = 𝑇 4 * 𝑃, where 𝑇 4 is given by 𝑇 4
Figure 9 .Figure 9. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 10. Cloud 𝑃′ = 𝑀 4 β€’ 𝑃 (blue), cloud 𝑄 (green).
Figure 10 .Figure 9. Cloud 𝑃 (blue), cloud 𝑄 (green).Figure 10. Cloud 𝑃′ = 𝑀 4 β€’ 𝑃 (blue), cloud 𝑄 (green).
𝐴 is the column of variables with 12 elements 𝐴 = (π‘Ž 11 π‘Ž 12 π‘Ž 14 π‘Ž 14 = 𝑑 1 π‘Ž 21 π‘Ž 22 π‘Ž 23 π‘Ž 24 = 𝑑 2 π‘Ž 31 π‘Ž 32 π‘Ž 33 π‘Ž 34 = 𝑑 3 ) 𝑑 . (15)𝑝 1 𝑖 𝑛 1 𝑖 𝑛 𝑗 𝑖𝑝 1 𝑖 𝑛 2 𝑖 𝑛 𝑗 𝑖 𝑝 1 𝑖 𝑛 3 𝑖 𝑛 𝑗 𝑖 0(𝑛 𝑗 𝑃𝑁) 𝑖 =𝑝 2 𝑖 𝑛 1 𝑖 𝑛 𝑗 𝑖 𝑝 2 𝑖 𝑛 2 𝑖 𝑛 𝑗 𝑖 𝑝 2 𝑖 𝑛 3 𝑖 𝑛 𝑗 𝑖 0 𝑝 3 𝑖 𝑛 1 𝑖 𝑛 𝑗 𝑖 𝑝 3 𝑖 𝑛 2 𝑖 𝑛 𝑗 𝑖 𝑝 3 𝑖 𝑛 3 𝑖 𝑛 𝑗 𝑖 0, 𝑖 = 1, … , 𝑛, 𝑗 = 1, … ,3,(9)(𝑛 1 𝑖 𝑛 𝑗 𝑖𝑛 2 𝑖 𝑛 𝑗 𝑖𝑛 3 𝑖 𝑛 𝑗 𝑖0 )𝑀 3𝑖+𝑗 = (π‘š 11 π‘–π‘—π‘š 21 π‘–π‘—π‘š 31 π‘–π‘—π‘š 41 π‘–π‘—π‘š 12 π‘–π‘–π‘š 22 π‘–π‘—π‘š 32 π‘–π‘—π‘š 42 π‘–π‘—π‘š 13 π‘–π‘—π‘š 23 π‘–π‘—π‘š 33 π‘–π‘—π‘š 43 𝑖𝑗 ),𝑖, 𝑗 = 1, … ,3,(10)π‘š π‘˜π‘™ 𝑖𝑗 = βˆ‘ (𝑝 𝑗 𝑛 𝑖 𝑃𝑁) π‘˜π‘™ 𝑖 𝑛 𝑖=1 𝑝 1 π‘˜ 𝑛 1 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 𝑝 1 π‘˜ 𝑛 2 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 , π‘˜, 𝑙 = 1, … ,4, 𝑖, 𝑗 = 1, … ,3, π‘˜ 𝑝 1 π‘˜ 𝑛 3 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 0(11)(𝑝 𝑗 𝑛 𝑖 𝑃𝑁) π‘˜ =𝑝 2 π‘˜ 𝑛 1 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 𝑝 2 π‘˜ 𝑛 2 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 𝑝 2 π‘˜ 𝑛 3 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 0 𝑝 3 π‘˜ 𝑛 1 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 𝑝 3 π‘˜ 𝑛 2 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 𝑝 3 π‘˜ 𝑛 3 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ 0, π‘˜ = 1, … , 𝑛, 𝑖, 𝑗 = 1, … ,3.(12)(𝑛 1 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜π‘› 2 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜π‘› 3 π‘˜ 𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜0 )𝐢 is the coefficients column with 12 elements𝑐 𝑗 = βˆ‘ 𝑛 𝑗 𝑖 < π‘ž 𝑖 , 𝑛 𝑖 > 𝑛 𝑖=1 , 𝑗 = 1, … ,3,(13)𝑐 3𝑖+𝑗 = βˆ‘π‘› π‘˜=1𝑝 𝑗 π‘˜ 𝑛 𝑖 π‘˜ < π‘ž π‘˜ , 𝑛 π‘˜ >, 𝑖, 𝑗 = 1, … ,3.(14))𝑀 is the coefficients matrix 12 Γ— 12𝑀 𝑗 = (π‘š 11 π‘—π‘š 21 π‘—π‘š 31 π‘—π‘š 41 π‘—π‘š 12 π‘–π‘š 22 π‘—π‘š 32 π‘—π‘š 42 π‘—π‘š 13 π‘—π‘š 23 π‘—π‘š 33 π‘—π‘š 43 𝑗 ),𝑗 = 1, … ,3,(7)π‘š π‘˜π‘™ 𝑗 = βˆ‘ (𝑛 𝑗 𝑃𝑁) π‘˜π‘™ 𝑖 𝑛 𝑖=1, π‘˜, 𝑙 = 1, … ,4, 𝑗 = 1, … ,3,(8)
Acknowledgments

The work was supported by the Ministry of Education and Science of Russian Federation (grant β„– 2.1743.2017) and by the RFBR (grant β„– 18-07-00963).

A Method for Registration of 3-D Shapes PBesl NMckay IEEE Transactions on Pattern Analysis and Machine Intelligence 14 1992 Object Modeling by Registration of Multiple Range Images Image and YChen GMedioni Vision Computing 10 1992 Real-time analysis of parameters of multiple object detection systems VIProtsenko NKazanskiy P GSerafimovich 10.18287/0134-2452-2015-39-4-582-591 Computer Optics 39 4 2015 A local order transform of digital images VMyasnikov 10.18287/0134-2452-2015-39-3-397-405 Computer Optics 39 3 2015 3D scene reconstruction from stereo images with unknown extrinsic parameters GoshinYe VFursov V A 10.18287/0134-2452-2015-39-5-770-775 Computer Optics 39 5 2015 Zippered Polygon Meshes from Range Images Computer Graphics Proceedings GTurk MLevoy Annual Conference Series 1994 Closed-Form Solution of Absolute Orientation Using Unit Quaternions BHorn Journal of the Optical Society of America A 4 1987 Closed-form Solution of Absolute Orientation Using Orthonormal Matrices Journal of the Optical BHorn HHilden SNegahdaripour Society of America Series A 5 1988 An extension of the ICP algorithm considering scale factor SDu NZheng SYing QYou YWu Proc. 14th IEEE Internat. Conf. on Image Processing (ICIP) 14th IEEE Internat. Conf. on Image essing (ICIP) 2007 Affine iterative closest point algorithm for point set registration Pattern Recognition SDu NZheng YingSLiu J Letters 31 2010 Affine Registration of Point Sets Using ICP and ICA SDu NZheng GMeng ZYuan IEEE Signal Processing Letters 15 2008 A modified iterative closest point algorithm for shape registration DTihonkih AMakovetskii VKuznetsov Proc. SPIE Appl. of Digital Image 99712D 2016. 9971 The iterative closest points algorithm and affine transformations DTihonkih MakovetskiiKuznetsov V Proc. Int. Conference of Analysis of Images, Social Networks, and Texts Int. Conference of Analysis of Images, Social Networks, and Texts 2016 A fusion algorithm for building three-dimensional maps AVokhmintsev AMakovetskii VKober ISochenkov VKuznetsov Proc. SPIE's 60 Annual Meeting: Applications of Digital Image Processing XXXVIII 9599 SPIE's 60 Annual Meeting: Applications of Digital Image essing XXXVIII 9599 2015 Efficient Variants of the ICP Algorithm Proceedings of the International Conference on 3-D Digital Imaging and Modeling SRusinkiewicz MLevoy 2001 An efficient point-to-plane registration algorithm for affine transformations Applications of Digital Image Processing AMakovetskii SVoronin VKober DTihonkih 2017 10396 103962J Affine registration of point clouds based on point-to-plane approach AMakovetskii SVoronin VKober DTihonkih Procedia Engineering 201 2017