The aim of the research is a 3D point matching which uses models from di erent sources to union them in the general coordinate system to create a high-accuracy scene from the 3D scanner. There are several images from the stereo camera (dual optical camera) at the device output to construct a 3D high-resolution model with 360 degrees horizontal eld of view. There is a high-accuracy point cloud from the LIDAR module (laser range nder) to correct the full eld view from the stereo camera. The research process is called point cloud registration and will help to create an output 3D representation with advantages of both sources. The presented algorithm has low computational complexity and,in the future, will be implemented on the scanner embedded system. The results of the data processing are presented in this paper.
The object of the research is the multi-sensor mobile 3D scanner with 360 degrees
range, which uses the stereo camera and LIDAR module. Calculations are made
on the embedded heterogeneous computing system with GPU NVidia Tegra TK1
and FPGA Xilinx Zynq-7000 soc [
The resolution of a 360 degrees model from the stereo camera is up to 10 millions colored points on the target system. However, the representation has low accuracy and includes unrecognized parts of the scene with solid-color objects without shadows.
The point cloud from the laser range nder Sweep V1 has high accuracy up
to 1 centimeter for 40 meters distance [
Both of clouds in the Cartesian coordinate system are turned versus each
other, they have di erent angles of view and shifted origin points.
Point cloud registration is used to build a new representation, which contains
source models from the stereo camera and LIDAR and will be used to correct
the output scene. The general approach to the point cloud registration is using
key points for shifting and rotating input models to the scene construction. It
uses geometric features of source clouds and makes a high-quality matching with
data that has not any distortions [
In practice, there are many distortions in the stereo camera model, and we
cannot use this method without signi cant modi cations [
Therefore, we chose an approach, which uses some scanner geometry features in lieu model key points. We decided to divide the data fusion task into two subtasks.
The rst task is the 3D scene construction from the scanner with 360 degrees range. The second one is the data fusion of combined stereo camera model with LIDAR point cloud. Both of subtasks could be solved by key point and geometric approaches. The decomposition allows to understand the task clearly and to get the estimated result, which could help to evaluate the matching quality. 2 2.1
The stereo camera is consisted of two calibrated coaxial oriented camera modules,
which give a couple of calibrated images for disparity map calculation to create
the 3D scene. We use StereoBM and StereoSGBM methods for disparity map
calculation with the use the default calibration module from OpenCV library
[
>8y <
x >:z = f = xl = yl b=disp y=f y=f (1)
x, y, z - point coordinates in the 3D model in mm. xl, yl - point coordinates in the disparity map in pixels. f - focal length of the camera in mm. disp - disparity value in pixels. b - baseline, the distance between two camera modules in mm. Laser range nder Sweep V1 creates a 2D scanning in the horizontal axis of rotation. This module has been installed on the scanner rotating part and does vertical scanning. After nishing, the 2D vertical scanner part is turned for a few degrees to perform the next 2D scanning. Figure 5 shows the point cloud from LIDAR. 2.3
Stereo camera has been installed on the horizontal rotated part of the scanner to create a 360-degrees range model. The scanner turns by view angle from the previous state to take a couple of photos. Figure 6 shows the top view of the scanner which makes new a couple of images every angle.
We use equation 2 to calculate view angle in degrees.
= binning dispRes=maxRes (2)
- camera horizontal view angle in degrees.
dispres - horizontal resolution of the disparity map.
maxRes - maximum horizontal resolution of the camera matrix.
binning - camera binning factor, which depends on the camera operation mode.
Values , maxRes and binning are got from camera speci cations [
The equation 3 presents the calculation of degrees for the system.
360-degree model construction from the stereo camera
For 360-degrees scene construction we turn every model by degrees from the
previous state. For rotation, we obtain spherical coordinates from the cartesian
coordinate system using the ordinary equation [
The rotation equation 4 in the spherical coordinate system is used to construct the scene. (4) (5) 8>r < > : = rnModel = nModel = nModel + (n 1) Where n 2 [1...N], N is the quantity of sources models, which equals (5) 360=42; 5 = 8; 4 = 9: This approach with using some geometric features (view angle and rotation angle) has advantages over the key points method.
Firstly, the deformation of input models does not a ect to 360 degrees model construction. Secondly, fewer models are used for construction because it is not necessary to cross them with each other. Finally, there is a simple implementation, which does not require high-performance hardware resources. The method also has a disadvantage. Angle errors and vibrations in the mechanical part of the device will in uence on the 360 degrees range stereo model and won't be founded using this method. 2.5
Steps below provide matching of LIDAR and stereo camera point clouds. 1. O set the laser range nder model down in the Cartesian coordinate system(6). 2. Convert the shifted scene to the spherical coordinate system. 3. Rotate the model in spherical coordinates system (7).
>8xshift <
yshift >:zshift = xlidar = ylidar = zlidar >8r < > : = rshift = shift = shift n =2 (6) (7) For improving calculation performance we shift LIDAR cloud because there are fewer points than in the stereo camera model. 3 3.1
We developed the software in C++ which creates a union point cloud from the stereo camera and laser range nder using geometrical method described above. Both representations are kept in di erent organized structures to implement fast calculations. Figure 8 shows the LIDAR point cloud structure. The main feature of the LIDAR model is that every point of the 2D range in a plane has equal coordinate. There are one-dimensional arrays with point structure which consists of angle and radius. We can get an array through indexing. This structure is used to keep fewer values and to get simple indexation. In addition, for rotating we change only the index value of the array. So, there is no need to change every point.
The stereo camera model has other features. Unfortunately, we cannot
assume that every point in a line has equal in column case and in row space.
Equation [
We scanned the next scene to evaluate the quality of the model combination: 9 colored clouds from the stereo camera with 42 degrees view angle; LIDAR point cloud where every 3 degrees a 500 points plane has been built. As a result of this research, the algorithm and software have been developed and tested with a real data from the 3D scanner. The future work is the construction of a laboratory scene with some metrical objects to evaluate algorithm accuracy. The other way is a developing a method for point cloud correcting using presented data structures and scanner characteristics. In the future, we are going to improve the application performance using a multi-threaded implementation.