=Paper=
{{Paper
|id=Vol-2744/short6
|storemode=property
|title=Modeling the Deformation of the Aerodynamic Surface to Estimate the Error of the Photogrammetric Algorithm (short paper)
|pdfUrl=https://ceur-ws.org/Vol-2744/short6.pdf
|volume=Vol-2744
|authors=Anton Poroykov,Nataliya Ilina,Ekaterina Shmatko
}}
==Modeling the Deformation of the Aerodynamic Surface to Estimate the Error of the Photogrammetric Algorithm (short paper)==
https://ceur-ws.org/Vol-2744/short6.pdf
Modeling the Deformation of the Aerodynamic Surface to
Estimate the Error of the Photogrammetric Algorithm*
Anton Poroykov1 [0000-0002-9284-1397], Nataliya Ilina1 [0000-0002-1793-4606]
and Ekaterina Shmatko1 [0000-0003-3565-9502]
National Research University "Moscow Power Engineering Institute",
Krasnokazarmennaya 14, Moscow, 111250, Russia
poroykovay@gmail.com
nataliya48452@gmail.com
shmatko.97@bk.ru
Abstract. In-flight measurements are one of the directions of aircraft testing
which has been developed actively recently. Creation of a flight laboratory based
on an unmanned vehicle to reduce the cost of developing and analysis techniques
for in-flight measurements is proposed. For the laboratory an onboard system for
recording experimental images during a flight was developed. It was decided to
conduct computer modeling of images that can be obtained using the developed
system for its verification. Unity a cross-platform photorealistic graphic engine
with visual integrated development environment was chosen for this purpose. For
modeling both external and internal real camera parameters, including distortion
coefficients, and parameters of the measured surface could be preassigned. The
developed methodology for computer modeling of images of the deformable sur-
face allows to evaluate the accuracy of various algorithms for processing exper-
imental images, as well as the influence of the parameters of the experimental
setup on the error of the measurements results.
Keywords: Computer Modeling, Shape Measurement, Measurement Errors
Image Pattern, Correlation Technique.
1 Introduction
In-flight measurements are one of the directions of aircraft testing which has been de-
veloped actively recently. These measurements provide the ability to measure various
parameters directly in flight [1]. The development of these type of methods and tech-
niques is carried out by the international scientific and technical collaborations, for ex-
ample AIM and AIM2 [2]. Examples of practical tests include in-flight tests of the
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
* Publication financially supported by RFBR grant №20-01-00547
2 A. Poroykov, N. Ilina, E. Shmatko
VUT100 Cobra [3], Airbus A320 [4] and even Airbus A380 [5]. The practical imple-
mentation of this approach allows to accelerate both the development of new types of
aircraft, and the in-flight tests of serial samples, and their commissioning.
Carrying out such tests is associated with high cost. Therefore, the creation of a flight
laboratory, based on an unmanned vehicle, was proposed [6]. On the one hand, this will
significantly reduce the cost of testing, and on the other, the test conditions will be close
to real in-flight tests: an increased level of vibration, the influence of aerodynamic loads
and flight parameters on the deformation of the airframe, strong exposure to the Sun.
The application of this approach will allow to improve existing and develop new meth-
ods and techniques for in-flight tests.
The first method to be studied at the proposed laboratory will be the method of meas-
uring deformations (surface shape) – Image Pattern Correlation Technique (IPCT) [7-
8]. It is based on digital processing of stereo images to restore an object surface shape.
Processing is based on correlation methods that are widely used in flow diagnostic
methods: PIV (Particle Image Velocimetry) [9] and BOS (Background Oriented Schlie-
ren) [10]. For this purpose, an electronic optical system for recording stereo images in
flight was developed. A drone wing was selected as the object of the study. The system
allows to receive images on the aircraft board and transmit them wirelessly to the
ground to the operator’s computer. The system also allows to obtain flight parameters
to estimate their impact on the magnitude and type of deformations.
To conduct bench tests of the developed system, it was proposed to use the previ-
ously created experimental setup for estimating the error of photogrammetric methods
[11-12]. However, due to a number of circumstances, laboratory testing was not possi-
ble. Therefore, it was decided to conduct computer modeling of images that can be
obtained using the developed system.
The main idea of modeling is to set the shape of the surface that simulate the wing
during deformation, and to obtain its images with changes caused by this deformation.
After processing the images and restoring the surface shape using the IPCT, this result
can be compared with the wing surface coordinates predefined in the modeling. The
deviation between the points of the modeled and the calculated surfaces can serve as an
estimate of the error of the developed system and the selected processing algorithm. By
changing the modeling parameters: the position of the camera stereo system, the back-
ground pattern parameters and the deformation amplitude, experimental dependences
can be obtained to estimate the measurement error for various parameters of the in-
flight experiment.
2 Surface modeling technique
A cross-platform environment for the development of computer games - Unity [12] was
chosen as the basis for modeling. Its main advantages are the presence of a visual inte-
grated development environment, a photorealistic graphics engine, support for solid
state physics and the ability to write scripts in C#. The latter fact allows to use all the
features of the language in the modeling and connect additional libraries.
Modeling the Deformation of the Aerodynamic Surface to Estimate the Error… 3
In computer graphics, the smallest flat figure is a triangle and all complex surfaces
are broken into them. This is due to the fact that the surface of the triangle remains flat
during its points moving. Therefore, to simulate the deformation of the wing surface, it
is necessary to break it into triangles. Moreover, one of the assumptions used for mod-
eling is the absence of elastic deformations of the surface. In other words, the area of
the wing surface remains constant under all deformations. Therefore, the sides of the
triangle should not change.
To implement such triangles, one of the types of objects built into Unity – rigid bod-
ies can be used. These objects do not change their shape over time, while they are taken
into account in the work of the physical engine. They can be affected by gravity, forces
from other rigid bodies, and their speed and acceleration will be determined by the
magnitude, direction and time of application of the resultant force. Rigid bodies in
Unity can be interconnected using joints that simulate connections. This method was
chosen to create the surface of the simulated wing.
At the beginning of the modeling, the size of the wing and the number of partitions
of its surface into triangles vertically and horizontally are set. The wing is defined by a
rectangular surface. It, in turn, is divided into rectangles, and then each rectangle into
two triangles (see Fig. 1a). Neighboring triangles are connected using joints at two an-
gles (see Fig. 1b). Four triangles in the corners of the surface are fixed in space on soft
links to maintain the surface in a horizontal position. The texture of the background
pattern is applied to the surface. When moving a triangle, the corresponding part of the
texture of the background screen moves with it.
Fig. 1. The scheme of formation of the wing surface from triangles and their connection: a) the
partition of the wing surface into triangles, b) the formation of two adjacent triangles of four
vertices, c) horizontal and vertical connections between the vertices of the triangles.
4 A. Poroykov, N. Ilina, E. Shmatko
To specify the surface, the coordinates of the vertices of the rectangles are first calcu-
lated. The vertex coordinates are stored in a separate Verticies property of the object,
which is the wing surface in Unity. Next, each rectangle is divided into two triangles.
To record the vertices of triangles, not their coordinates are indicated, but the coordinate
numbers in the Verticies property. Therefore, four vertices are used to define two trian-
gles, not six (see Fig. 1b). Coordinate numbers are stored in the Triangles property. The
vertices must be specified in clockwise order. This condition defines the side on which
the texture will be displayed.
Each triangle is created as a rigid body. To simulate surface deformation, triangles
are interconnected. Two triangles in one rectangle are interconnected using two hori-
zontal joints (see Fig. 1c) having coordinates that coincide with the coordinates of two
common vertices of the triangles. The adjacent horizontal rectangles are interconnected
using two joints, the coordinates of which coincide with the coordinates of two common
vertices of the two outside triangles. Neighboring rectangles vertically are connected
using two joints, the coordinates of which coincide with the coordinates of two common
vertices of four triangles (see Fig. 1c).
Thus, the simulated wing surface consists of a large number of interconnected rigid
bodies, which are triangles. Their joints have a large number of settings, with the help
of which the interaction of the triangles: the limiting angles of rotation in two axes, the
forces acting when they are exceeded, etc., are configured. This allows to set the rigidity
of the simulated wing. If necessary, force can be applied to individual triangles with a
given direction and magnitude. This allow simulating deformations caused by aerody-
namic loads.
The software for modeling is a compiled program for the 32-bit or 64-bit Windows
operating system (it is possible to create executable files for other platforms). The view
of the graphical user interface of the program is presented in Fig. 2.
Fig. 2. Graphical interface of the program for modeling wing surface deformation.
Modeling the Deformation of the Aerodynamic Surface to Estimate the Error… 5
The control panel is located on the left side of the window. There are buttons on it for
setting a random position of the surface, for capturing images from the camera stereo
system, turning on the calibration mode, restarting the modeling and exiting the pro-
gram. In the window itself, the view of the wing is presented from the third camera,
which is not included in the stereo system. It is located at a large angle to the surface
for visual inspection. Two small windows are situated at the bottom of the panel. They
display the image from the cameras included in the stereo system. The text fields above
the buttons in the control panel are for indicating and setting the positions of eight tri-
angles on the surface. These triangles simulate the operation of servos in a laboratory
setup [11-12].
On click on the “Image Capture” button, the images received from the stereo system
cameras are saved to the folder with the modeling program. At the same time, a json
file is saved with the coordinates of the triangles. These coordinates are the exact value
of the position of the surface and can serve to estimate the accuracy of the results ob-
tained by restoring the shape of the surface during image processing.
To obtain calibration data of the stereo system, a calibration mode is provided. In
this mode, the wing surface is not displayed, but instead a calibration target appears
with the structure of a chessboard. The target randomly changes its position in space to
simulate the calibration procedure on a real setup. In each position, images are captured
from the stereo system, which are then used to calculate the external and internal pa-
rameters of the cameras, as well as their relative position in stereo system.
3 Modeling results and their processing
The scheme for reconstructing the surface shape using the IPCT which was used in the
work consists of:
─ application of a background pattern on the surface under study (on the model of the
deformable wing);
─ registering images of the background pattern from two cameras of the stereo system;
─ detecting marks on the images, according to their coordinates a preliminary compen-
sation of the displacement occurs using the perspective transformation;
─ splitting the converted images into small parts (interrogating windows) for which the
correlation function is calculated, the maximum of the function is the displacement
of the parts relative to each other in images from different cameras;
─ preliminary compensation of displacements and final calculation using cross-corre-
lation gives two pairs of two-dimensional coordinates of one three-dimensional point
in the images, triangulation by these coordinates allows to determine three-dimen-
sional coordinates of surface points, in other words, restore the desired shape.
Image processing was carried out using the OpenCV library in the Python program-
ming language. The cross-correlation processing algorithm is implemented in Python
and allows iterative processing with a decrease of an interrogating window and post-
filtering to increase the accuracy of the determined displacement.
6 A. Poroykov, N. Ilina, E. Shmatko
Fig. 3 shows an example of modeled images of a simulated aircraft wing. The surface
size is 400×200 mm2, it was divided into 1600 triangles. The parameters of the cameras
corresponded to the real cameras used in the created system [6]. And their location
relative to the wing corresponded to the approximate location of the system on an un-
manned aerial vehicle. As a texture for the surface, we used a background pattern with
randomly located dots and additional marks for preliminary image processing.
Fig. 3. Modeled images of a deformed surface of a simulated aircraft wing.
Fig. 4 shows an example of a vector field obtained by calculating the cross-correlation
function. Processing was performed with the initial size of the interrogation window
32x32 pixels to a final size of 8x8 pixels in 3 iterations. The minimum value of the
normalized correlation function at the maxima (the color of the arrows in Fig. 4) in the
interrogation windows was at least 0.6, which indicates the reliability of the calculation.
Fig. 4. The vector field obtained by calculating the cross-correlation function between two im-
ages in Fig. 3.
Modeling the Deformation of the Aerodynamic Surface to Estimate the Error… 7
After processing the modeled images, the results were compared with the given values
of the surface shape during modeling. At the beginning, the coordinates of the surface
points from Unity were transformed into the coordinate system of the camera stereo
system. The coordinate system in Unity is left-handed, and in OpenCV right-handed.
The cameras were also located at a certain distance from the origin and at an angle of
35°. These parameters also had to be taken into account when comparing the results.
Further, the three-dimensional surface was interpolated for points specified in the mod-
eling for comparison with points obtained during processing. The x and y coordinates
were equal, the error along the z coordinate was calculated.
Fig. 5 shows an example of comparing processing results with set of points prede-
fined during modeling. The standard deviation was less than 2 mm on the surface of
160x80 mm2, which indicates a high measurement error.
Fig. 5. Comparison of processing results with set of points predefined during modeling.
4 Conclusions
In the work, software was created for computer modeling of images of a deformed sur-
face obtained using a stereo system. When modeling, it is possible to set both real cam-
era parameters (external and internal parameters, including distortion coefficients), and
parameters of the measured surface. A surface is defined using many connected trian-
gles. Changing the parameters of their connection allows to bring the surface behavior
closer to real physical objects, for example, the wing of an aircraft. Also, adding an
additional random error into the predefined values during modeling allows to estimate
the influence of the calibration accuracy of the stereo system of cameras on the meas-
urement results.
8 A. Poroykov, N. Ilina, E. Shmatko
The developed methodology for computer modeling of images of the deformable
surface allows to evaluate the accuracy of various algorithms for processing experi-
mental images, as well as the influence of the parameters of the experimental setup on
the error of the results.
References
1. Boden, F., Lawson, N., Jentink, H.W., Kompenhans, J. eds: Advanced In-Flight Measure-
ment Techniques. Springer Berlin Heidelberg, Berlin, Heidelberg (2013).
https://doi.org/10.1007/978-3-642-34738-2.
2. AIM, http://aim.dlr.de/site/index.php, last accessed 2020/06/29.
3. Kirmse, T.: Recalibration of a stereoscopic camera system for in-flight wing deformation
measurements. Meas. Sci. Technol. 27, 054001 (2016). https://doi.org/10.1088/0957-
0233/27/5/054001.
4. Kirmse, T., Maring, S., Ebel, P.-B., Schröder, A.: Fan blade deformation measurements on
the DLR Airbus A320-ATRA by means of IPCT as part of the ground test campaign in the
frame of the DLR-project SAMURAI. In: Dillmann, A., Heller, G., Krämer, E., Wagner, C.,
and Breitsamter, C. (eds.) New Results in Numerical and Experimental Fluid Mechanics X.
pp. 629–638. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-
3-319-27279-5_55.
5. Boden, F., Jentink, H., Petit, C.: IPCT wing deformation measurements on a large transport
aircraft. In: Boden, F., Lawson, N., Jentink, H.W., and Kompenhans, J. (eds.) Advanced in-
flight measurement techniques. pp. 93–115. Springer, Berlin, Heidelberg (2013).
https://doi.org/10.1007/978-3-642-34738-2_7.
6. Poroykov A.Y., Surkov D.A., Ilina N.S., Lebedev S.V., Ul’yanov D.V., Lapitskiy K.M.:
Development of the flight laboratory for research of aerodynamic surfaces deformation. J.
Phys.: Conf. Ser. (2020). In print.
7. Petit C., Jentink H., Boden F., Kannemans H., Veermann H., Kirmse T.: Introducing a new
measurement method for wing twist and bending. In: European Test & Telemetry Confer-
ence 2009, pp. 1–6. (2009).
8. Poroikov, A.Yu.: Reconstruction of 3D Profile of a Deformed Metallic Plate by Means of
the Image Pattern Correlation Technique. Meas Tech. 57, 390–395 (2014).
https://doi.org/10.1007/s11018-014-0466-4.
9. Raffel, M., Willert, C.E., Scarano, F., Kähler, C.J., Wereley, S.T., Kompenhans, J.: Particle
Image Velocimetry: A Practical Guide. Springer (2018).
10. Meier, G.: Computerized background-oriented schlieren. Exp Fluids. 33, 181–187 (2002).
https://doi.org/10.1007/s00348-002-0450-7.
11. Poroykov A.Yu.: A system for determining the error in measuring the shape of a flexible
deformable surface by means of the Image Pattern Correlation Technique. Herald of the
Bauman Moscow State Tech. Univ., Instrum. Eng. 6, 28–39 (2017).
https://doi.org/10.18698/0236-3933-2017-6-28-39.
12. Ivanova, Y.V., Poroykov, A.Yu.: Estimation of the measurement error of photogrammetric
techniques by controlled flexible deformable surface. In: 2019 International Youth Confer-
ence on Radio Electronics, Electrical and Power Engineering (REEPE). pp. 1–4. IEEE, Mos-
cow, Russia (2019). https://doi.org/10.1109/REEPE.2019.8708779.
13. Technologies, U.: Unity Real-Time Development Platform | 3D, 2D VR & AR Visualiza-
tions, https://unity.com/, last accessed 2020/06/29.