=Paper=
{{Paper
|id=Vol-2744/short14
|storemode=property
|title=Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation (short paper)
|pdfUrl=https://ceur-ws.org/Vol-2744/short14.pdf
|volume=Vol-2744
|authors=Anton Poroykov,Pavel Kalugin,Sergey Shitov,Irina Lapitskaya
}}
==Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation (short paper)==
https://ceur-ws.org/Vol-2744/short14.pdf
Modeling ArUco Markers Images for Accuracy Analysis
of Their 3D Pose Estimation*
Anton Poroykov[0000-0002-9284-1397], Pavel Kalugin[0000-0001-6796-4486],
Sergey Shitov[0000-0002-5723-2529] and Irina Lapitskaya[0000-0001-6007-0252]
National Research University "Moscow Power Engineering Institute",
Krasnokazarmennaya 14, Moscow, 111250, Russia
poroykovay@gmail.com
Abstract. Fiducial markers are used in vision systems to determine the position
of objects in space, reconstruct movement and create augmented reality. Despite
the abundance of work on analysis of the accuracy of the estimation of the fidu-
cial markers spatial position, this question remains open. In this paper, we pro-
pose the computer modeling of images with ArUco markers for this purpose. The
paper presents a modeling algorithm, which was implemented in the form of soft-
ware based on the OpenCV library. Algorithm is based on projection of three-
dimensional points of the marker corners into two-dimensional points using the
camera parameters and rendering the marker image in the new two-dimensional
coordinates on the modeled image with the use of the perspective transformation
obtained from these points. A number of dependencies were obtained by which
it is possible to evaluate the error in determining the position depending on mark-
ers size. Including the probability of detecting a marker depending on its area on
an image.
Keywords: Computer Modeling, ArUco Markers, Pose Error Estimation.
1 Introduction
Fiducial markers are used in vision systems to determine the position of objects in
space, reconstruct movement and create augmented reality. The main directions of their
application are the estimation of the position of objects in space and the technology of
augmented reality. Particularly relevant is the use of markers in systems where it is
required to manipulate objects in a closed limited space. This is due to the fact that
markers are a relatively cheap and affordable way to estimate the position of objects in
space. It has acceptable accuracy with the ability to increase it depending on the equip-
ment used and the processing algorithm. Also, to improve accuracy, this method can be
combined with other methods of estimation the position in space. Examples of the use
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
* Publication financially supported by RFBR grant №18-08-01484
2 A. Poroykov, P. Kalugin, S. Shitov, I. Lapitskaya
of tags are the creation of autonomous robotic systems [1-2], industrial large-scale tasks
[3] and even their use for medical purposes in surgery [4].
Fiducial markers combine a special mark that is robastly detected on images and a
small amount of data that can be interpreted as a marker number. The mark and data
are determined using digital image processing methods. In this paper, we consider one
of the most modern markers varieties – ArUco markers proposed in [5].
A number of papers have been devoted to analysis of the accuracy of the estimation
of the fiducial markers spatial position. In [6], one of the earliest and most comprehen-
sive works, several types of markers were compared. However, they are considered
from the point of view of the application for augmented reality, since an application for
determining the position in space for robotic systems has not yet been widely devel-
oped. Around the same time, similar results were obtained in [7–9]. The work [10]
should also be mentioned as it contains the evaluation of artificial fiducial markers for
underwater applications.
One of the most modern works is [11]. In it computer modeling of the influence of
the position of ArUco markers relative to the camera on the error of estimation of their
spatial position was carried out. The paper presents a large number of obtained depend-
encies. However, they were obtained for specific parameters preassigned in the model-
ing, and therefore cannot be used in practice.
Despite the abundance of work on analysis of the accuracy of the estimation of the
fiducial markers spatial position, this question remains open. Therefore, this work is
devoted to computer modeling of images with ArUco markers with the ability to pre-
assigned their position both in distance and in rotation angles in space.
2 Images modeling technique
OpenCV [12] – the library for digital image processing was chosen as the basis for
modeling. It is the de facto standard in the computer and technical vision industry. The
library is widely used and is constantly being improved. OpenCV is written in C ++,
however there are a large number of projects that allow to use it with other program-
ming languages, such as Python and C#. The C# was chosen for the implementation of
a software for ArUco markers modeling. There are several wrapper libraries for using
OpenCV in C#. The main ones are EmguCV [13] and OpenCVSharp [14]. The last was
used in this work, since it allows to use the OpenCV with the latest version of
.NET Core and has a syntax closer to the original library.
The simulation algorithm consists of the following sequential steps (see Fig. 1):
1. Assign the intrinsic and extrinsic parameters of the camera (the size of the sensor
in pixels and millimeters, the focal length of the lens, distortion coefficients, camera
location);
2. Assign the two-dimensional coordinates of the corners of the marker in accordance
with its original image;
3. Assign the three-dimensional coordinates of the corners of the marker in accord-
ance with its specified physical dimensions;
Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation 3
4. Assign the modeled 3D position of the marker using the translation and rotation
vector;
5. Calculation of new three-dimensional coordinates of the corners of the marker by
multiplying the initial coordinates by the rotation and translation matrixes;
6. Projection of three-dimensional points of the marker corners into two-dimensional
points on the modeled image using the camera parameters;
7. Calculation of the function of the perspective transformation from two-dimensional
origin coordinates into projected coordinates of the corners;
8. Rendering the marker image in the new two-dimensional coordinates on the mod-
eled image with the use of the perspective transformation;
9. Estimation of the 3D marker position with markers detector on the modeled image;
10. Evaluation of the error by comparing the estimated position of the marker with a
preassigned position in step 4;
11. Repeat steps 4-10 for other marker positions.
Fig. 1. Scheme of modeling images with ArUco markers.
After drawing the modeled marker image, the background was filled with black, but
there is also the possibility of using any other background image. The outer perimeter
4 A. Poroykov, P. Kalugin, S. Shitov, I. Lapitskaya
of the marker is white; therefore, adding a background image with a complex structure
should not affect the results of determining the spatial position of the modeled marker,
although it can cause false markers to be found. Fig. 2 shows an example of modeled
images with different marker tilt angles.
Fig. 2. An example of modeled images with different marker tilt angles.
3 Modeling results
For modeling, the following parameters were used. The dimensions of the camera sen-
sor are 4.8 by 3.6 mm (1/3”). The lens focal length is 4 mm. These parameters corre-
spond to a typical modern webcam. The size of the marker was 10 by 10 mm. To detect
marks in the image, we used the standard algorithm for recognizing ArUco markers in
OpenCV with standard parameters.
Fig. 3. The dependence of the influence of the marker area size on the error in detecting its po-
sition for marker position perpendicular to camera.
Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation 5
We give two examples of the results obtained by modeling. One of the important
parameters that may be of interest when constructing measuring systems with fiducial
markers is the accuracy of finding their position in space, both in tilt angles and in
distance. The dependence of the influence of the area of the marker on the error in
finding its position obtained in the modeling is presented on Fig. 3. The dependence
shows that reducing the size of the marker in the image leads to an increase in the error.
Fig. 4. The probability of detecting a marker depending on its area on image for 3 different tilt
angels: 0°, 45°, 70°.
Fig. 4 shows the probability of finding a marker depending on its area on image for
3 different tilt angels: 0°, 45°, 70°. From the dependence it can be seen that the marker
recognition probability for tilt angle 0° differs significantly from 100% in two regions:
with the area of 300 pixels and with the area in the range 350÷450 pixels. This result is
consistently repeated for different series of modeling with different initial parameters.
Moreover, for markers with some numbers in the dictionary, the second region (with
the area in the range 350÷450 pixels) disappears. Explaining this behavior is one of our
tasks for our further work.
4 Conclusions
The paper presents an approach to computer modeling of images with fiducial markers
based on the use of the OpenCV library. It has several advantages. Firstly, it simplifies
the writing of modeling software, since it does not require the use of graphic engines,
which may require studying the principles of their work and API for its integration.
Secondly, markers modeling, detection and pose estimation is carried out by standard
6 A. Poroykov, P. Kalugin, S. Shitov, I. Lapitskaya
library functions. In the case of using graphic engines, it might be necessary to calculate
the transformations of the modeling results due to the different coordinate systems used
in the graphic engine and the OpenCV.
The developed software for modeling digital images with ArUco markers allows to
evaluate the error in estimation the position of the markers when digitally processing
modeled images, taking into account both intrinsic and extrinsic camera parameters.
We gave two examples of the results obtained by modeling. With their help it is able to
estimate accuracy of future measurement system during its developing. And evaluate
probability of detecting markers on image depending on it area size in pixels.
In order to verify the results of computer modeling in the future, it is planned to carry
out a physical laboratory modeling of the accuracy of estimating the position of markers
in space. The position of the markers will be set and monitored using precision dis-
placement modules. The results obtained will be compared with computer modeling,
which will confirm the efficiency of the proposed methodology.
References
1. Bacik, J., Durovsky, F., Fedor, P., Perdukova, D.: Autonomous flying with quadrocopter
using fuzzy control and ArUco markers. Intel Serv Robotics. 10, 185–194 (2017).
https://doi.org/10.1007/s11370-017-0219-8.
2. Shabalina, K., Sagitov, A., Svinin, M., Magid, E.: Comparing Fiducial Markers Performance
for a Task of a Humanoid Robot Self-calibration of Manipulators: A Pilot Experimental
Study. In: Ronzhin, A., Rigoll, G., and Meshcheryakov, R. (eds.) Interactive Collaborative
Robotics. pp. 249–258. Springer International Publishing, Cham (2018).
https://doi.org/10.1007/978-3-319-99582-3_26.
3. Ujkani, E., Dybedal, J., Aalerud, A., Kaldestad, K.B., Hovland, G.: Visual Marker Guided
Point Cloud Registration in a Large Multi-Sensor Industrial Robot Cell. In: 2018 14th
IEEE/ASME International Conference on Mechatronic and Embedded Systems and Appli-
cations (MESA). pp. 1–6 (2018). https://doi.org/10.1109/MESA.2018.8449195.
4. Pflugi, S., Vasireddy, R., Lerch, T., Ecker, T.M., Tannast, M., Boemke, N., Siebenrock, K.,
Zheng, G.: Augmented marker tracking for peri-acetabular osteotomy surgery. Int J CARS.
13, 291–304 (2018). https://doi.org/10.1007/s11548-017-1690-6.
5. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Auto-
matic generation and detection of highly reliable fiducial markers under occlusion. Pattern
Recognition. 47, 2280–2292 (2014). https://doi.org/10.1016/j.patcog.2014.01.005.
6. Xiang Zhang, Fronz, S., Navab, N.: Visual marker detection and decoding in AR systems: a
comparative study. In: Proceedings. International Symposium on Mixed and Augmented
Reality. pp. 97–106 (2002). https://doi.org/10.1109/ISMAR.2002.1115078.
7. Malbezin, P., Piekarski, W., Thomas, B.H.: Measuring ARTootKit accuracy in long distance
tracking experiments. In: The First IEEE International Workshop Agumented Reality
Toolkit,. p. 2 pp.- (2002). https://doi.org/10.1109/ART.2002.1107000.
8. Abawi, D.F., Bienwald, J., Dorner, R.: Accuracy in optical tracking with fiducial markers:
an accuracy function for ARToolKit. In: Third IEEE and ACM International Symposium
on Mixed and Augmented Reality. pp. 260–261 (2004).
https://doi.org/10.1109/ISMAR.2004.8.
Modeling ArUco Markers Images for Accuracy Analysis of Their 3D Pose Estimation 7
9. Pentenrieder, K., Meier, P., Klinker, G.: Analysis of Tracking Accuracy for Single-Camera
Square-Marker-Based Tracking. In: Proc. Dritter Workshop Virtuelle und Erweiterte Realitt
der GIFachgruppe VR/AR, Koblenz, Germany (2006).
10. dos Santos Cesar, D.B., Gaudig, C., Fritsche, M., dos Reis, M.A., Kirchner, F.: An evalua-
tion of artificial fiducial markers in underwater environments. In: OCEANS 2015 - Genova.
pp. 1–6 (2015). https://doi.org/10.1109/OCEANS-Genova.2015.7271491.
11. López-Cerón, A., M. Canas, J.: Accuracy analysis of marker-based 3 D visual localization.
In: XXXVII Jornadas de Automatica Workshop. p. 8 (2016).
12. OpenCV, https://opencv.org/, last accessed 2020/07/05.
13. Emgu CV: OpenCV in .NET (C#, VB, C++ and more), http://www.emgu.com/wiki/in-
dex.php/Main_Page, last accessed 2020/07/05.
14. OpenCVSharp, https://github.com/shimat/opencvsharp, last accessed 2020/06/15.