In-flight measurements are one of the directions of aircraft testing which has been developed actively recently. Creation of a flight laboratory based on an unmanned vehicle to reduce the cost of developing and analysis techniques for in-flight measurements is proposed. For the laboratory an onboard system for recording experimental images during a flight was developed. It was decided to conduct computer modeling of images that can be obtained using the developed system for its verification. Unity a cross-platform photorealistic graphic engine with visual integrated development environment was chosen for this purpose. For modeling both external and internal real camera parameters, including distortion coefficients, and parameters of the measured surface could be preassigned. The developed methodology for computer modeling of images of the deformable surface allows to evaluate the accuracy of various algorithms for processing experimental images, as well as the influence of the parameters of the experimental setup on the error of the measurements results.
In-flight measurements are one of the directions of aircraft testing which has been
developed actively recently. These measurements provide the ability to measure various
parameters directly in flight [
Carrying out such tests is associated with high cost. Therefore, the creation of a flight
laboratory, based on an unmanned vehicle, was proposed [
The first method to be studied at the proposed laboratory will be the method of
measuring deformations (surface shape) – Image Pattern Correlation Technique (IPCT)
[78]. It is based on digital processing of stereo images to restore an object surface shape.
Processing is based on correlation methods that are widely used in flow diagnostic
methods: PIV (Particle Image Velocimetry) [
To conduct bench tests of the developed system, it was proposed to use the
previously created experimental setup for estimating the error of photogrammetric methods
[
The main idea of modeling is to set the shape of the surface that simulate the wing during deformation, and to obtain its images with changes caused by this deformation. After processing the images and restoring the surface shape using the IPCT, this result can be compared with the wing surface coordinates predefined in the modeling. The deviation between the points of the modeled and the calculated surfaces can serve as an estimate of the error of the developed system and the selected processing algorithm. By changing the modeling parameters: the position of the camera stereo system, the background pattern parameters and the deformation amplitude, experimental dependences can be obtained to estimate the measurement error for various parameters of the inflight experiment. 2
A cross-platform environment for the development of computer games - Unity [
Modeling the Deformation of the Aerodynamic Surface to Estimate the Error… 3 In computer graphics, the smallest flat figure is a triangle and all complex surfaces are broken into them. This is due to the fact that the surface of the triangle remains flat during its points moving. Therefore, to simulate the deformation of the wing surface, it is necessary to break it into triangles. Moreover, one of the assumptions used for modeling is the absence of elastic deformations of the surface. In other words, the area of the wing surface remains constant under all deformations. Therefore, the sides of the triangle should not change.
To implement such triangles, one of the types of objects built into Unity – rigid bodies can be used. These objects do not change their shape over time, while they are taken into account in the work of the physical engine. They can be affected by gravity, forces from other rigid bodies, and their speed and acceleration will be determined by the magnitude, direction and time of application of the resultant force. Rigid bodies in Unity can be interconnected using joints that simulate connections. This method was chosen to create the surface of the simulated wing.
At the beginning of the modeling, the size of the wing and the number of partitions of its surface into triangles vertically and horizontally are set. The wing is defined by a rectangular surface. It, in turn, is divided into rectangles, and then each rectangle into two triangles (see Fig. 1a). Neighboring triangles are connected using joints at two angles (see Fig. 1b). Four triangles in the corners of the surface are fixed in space on soft links to maintain the surface in a horizontal position. The texture of the background pattern is applied to the surface. When moving a triangle, the corresponding part of the texture of the background screen moves with it. To specify the surface, the coordinates of the vertices of the rectangles are first calculated. The vertex coordinates are stored in a separate Verticies property of the object, which is the wing surface in Unity. Next, each rectangle is divided into two triangles. To record the vertices of triangles, not their coordinates are indicated, but the coordinate numbers in the Verticies property. Therefore, four vertices are used to define two triangles, not six (see Fig. 1b). Coordinate numbers are stored in the Triangles property. The vertices must be specified in clockwise order. This condition defines the side on which the texture will be displayed.
Each triangle is created as a rigid body. To simulate surface deformation, triangles are interconnected. Two triangles in one rectangle are interconnected using two horizontal joints (see Fig. 1c) having coordinates that coincide with the coordinates of two common vertices of the triangles. The adjacent horizontal rectangles are interconnected using two joints, the coordinates of which coincide with the coordinates of two common vertices of the two outside triangles. Neighboring rectangles vertically are connected using two joints, the coordinates of which coincide with the coordinates of two common vertices of four triangles (see Fig. 1c).
Thus, the simulated wing surface consists of a large number of interconnected rigid bodies, which are triangles. Their joints have a large number of settings, with the help of which the interaction of the triangles: the limiting angles of rotation in two axes, the forces acting when they are exceeded, etc., are configured. This allows to set the rigidity of the simulated wing. If necessary, force can be applied to individual triangles with a given direction and magnitude. This allow simulating deformations caused by aerodynamic loads.
The software for modeling is a compiled program for the 32-bit or 64-bit Windows
operating system (it is possible to create executable files for other platforms). The view
of the graphical user interface of the program is presented in Fig. 2.
The control panel is located on the left side of the window. There are buttons on it for
setting a random position of the surface, for capturing images from the camera stereo
system, turning on the calibration mode, restarting the modeling and exiting the
program. In the window itself, the view of the wing is presented from the third camera,
which is not included in the stereo system. It is located at a large angle to the surface
for visual inspection. Two small windows are situated at the bottom of the panel. They
display the image from the cameras included in the stereo system. The text fields above
the buttons in the control panel are for indicating and setting the positions of eight
triangles on the surface. These triangles simulate the operation of servos in a laboratory
setup [
On click on the “Image Capture” button, the images received from the stereo system cameras are saved to the folder with the modeling program. At the same time, a json file is saved with the coordinates of the triangles. These coordinates are the exact value of the position of the surface and can serve to estimate the accuracy of the results obtained by restoring the shape of the surface during image processing.
To obtain calibration data of the stereo system, a calibration mode is provided. In this mode, the wing surface is not displayed, but instead a calibration target appears with the structure of a chessboard. The target randomly changes its position in space to simulate the calibration procedure on a real setup. In each position, images are captured from the stereo system, which are then used to calculate the external and internal parameters of the cameras, as well as their relative position in stereo system. 3
The scheme for reconstructing the surface shape using the IPCT which was used in the work consists of: ─ application of a background pattern on the surface under study (on the model of the deformable wing); ─ registering images of the background pattern from two cameras of the stereo system; ─ detecting marks on the images, according to their coordinates a preliminary compensation of the displacement occurs using the perspective transformation; ─ splitting the converted images into small parts (interrogating windows) for which the correlation function is calculated, the maximum of the function is the displacement of the parts relative to each other in images from different cameras; ─ preliminary compensation of displacements and final calculation using cross-correlation gives two pairs of two-dimensional coordinates of one three-dimensional point in the images, triangulation by these coordinates allows to determine three-dimensional coordinates of surface points, in other words, restore the desired shape.
Image processing was carried out using the OpenCV library in the Python programming language. The cross-correlation processing algorithm is implemented in Python and allows iterative processing with a decrease of an interrogating window and postfiltering to increase the accuracy of the determined displacement.
Fig. 3 shows an example of modeled images of a simulated aircraft wing. The surface
size is 400×200 mm2, it was divided into 1600 triangles. The parameters of the cameras
corresponded to the real cameras used in the created system [
Fig. 4 shows an example of a vector field obtained by calculating the cross-correlation function. Processing was performed with the initial size of the interrogation window 32x32 pixels to a final size of 8x8 pixels in 3 iterations. The minimum value of the normalized correlation function at the maxima (the color of the arrows in Fig. 4) in the interrogation windows was at least 0.6, which indicates the reliability of the calculation. After processing the modeled images, the results were compared with the given values of the surface shape during modeling. At the beginning, the coordinates of the surface points from Unity were transformed into the coordinate system of the camera stereo system. The coordinate system in Unity is left-handed, and in OpenCV right-handed. The cameras were also located at a certain distance from the origin and at an angle of 35°. These parameters also had to be taken into account when comparing the results. Further, the three-dimensional surface was interpolated for points specified in the modeling for comparison with points obtained during processing. The x and y coordinates were equal, the error along the z coordinate was calculated.
Fig. 5 shows an example of comparing processing results with set of points predefined during modeling. The standard deviation was less than 2 mm on the surface of 160x80 mm2, which indicates a high measurement error. In the work, software was created for computer modeling of images of a deformed surface obtained using a stereo system. When modeling, it is possible to set both real camera parameters (external and internal parameters, including distortion coefficients), and parameters of the measured surface. A surface is defined using many connected triangles. Changing the parameters of their connection allows to bring the surface behavior closer to real physical objects, for example, the wing of an aircraft. Also, adding an additional random error into the predefined values during modeling allows to estimate the influence of the calibration accuracy of the stereo system of cameras on the measurement results.
The developed methodology for computer modeling of images of the deformable surface allows to evaluate the accuracy of various algorithms for processing experimental images, as well as the influence of the parameters of the experimental setup on the error of the results.