Paper presents a mathematical support for solving the problem of navigation along the optical channel of an aircraft. The stages were formalized and the main notations were introduced, a formal procedure for obtaining coordinates of special points of a digital image from aircraft cameras was presented. It is proposed to determine the coordinates of the aircraft based on the method that uses the estimation of the density function of the distribution of the coordinates of special points of the digital image. A method of verifying the reliability of the aircraft position assessment based on consistency measures has been formulated. The results of testing the proposed algorithm are given. Conclusions are formulated.
The relevance of this topic is determined by the increasing significance of unmanned aerial vehicles
(UAVs) for various applications, including surveillance, reconnaissance, monitoring, and data
collection. With the advancement of UAV technologies, the importance of accurate and reliable
methods for assessing the vehicle's location becomes critical to ensure the effectiveness and safety
of their operations [
Today, navigation systems for aircraft are the object of many studies. Both scientific institutions and
commercial business organizations around the world are actively engaged in this issue. Special
attention is paid to aircraft navigation in the absence of GPS signals [
This paper [
The study proposes a novel integration scheme combining GNSS, INS, and LiDAR data for
UAVbased navigation, particularly in areas where GNSS signals are obstructed or unreliable [
It is proposed to compensate for the work of satellite navigation due to the introduction of
additional optical navigation sensors. Sensors are integrated using the Kalman filter and its
modifications, for example [
Much attention is paid to the application of AI in navigation systems. In work [
This research [
The article [
This study [
The works of Ukrainian authors [
It is assumed that the flight mission can be performed in the area where relevant data has been previously collected. This requirement is put forward due to the fact that a set of descriptors of special landmark points, together with their associated coordinates, must be stored on board the UAV during the flight. So, the first step should be to conduct aerial photography and link the images to the digital map.
Linking pictures to a digital map is a separate task [
The second stage is the direct selection of special points and the determination of their
descriptors, it is described in detail in [
Thus, pre-flight preparation should ensure the availability of a text (or special format) file with descriptors of special points and their corresponding coordinates on the on-board computer during the flight mission = , , , ; = −3,3, = −3,3 , ; = 1, where , is magnitude; , is angle; is the number of special points.
The third stage is a flight mission. If it is necessary to position the aircraft, the on-board computer processes received picture from the target camera in order to find features and compare their descriptors with the available descriptors of the Orient array. The purpose of the comparison is to form a new array containing a list of coordinates corresponding to the matching descriptors: = , ; = 1, , where is the number of singular points in the image, or the actual number of descriptor matches, if some distance metric threshold is introduced.
In the following, we will analyze the set of coordinates of the array as a sample of realizations of some random variable. The purpose of such an analysis is to establish the most likely location of the aircraft, depending on the probability distribution of data.
Let's consider the approach to the selection of special points on the digital image from the camera of
the target load of the UAV directly during the flight mission. After detecting singular points using
the operators discussed in [
Let be
= , , , ; = −3,3, = −3,3 , , ; = 1, – a set of descriptors of control singular points with their corresponding coordinates, ! = , , , ; = −3,3, = −3,3 , _# $ , _# $ , = 1, – a set of descriptors of special points of the test image and corresponding pixel indices of the digital image,
_# $ ∈ &0; ( ) ℎ − 1*; _# $ ∈ &0; + , ℎ − 1*, where ( ) ℎ, + , ℎ - height and width of the picture from the LA camera.
If the values of the magnitude and angles are normalized, then as a measure of the distance between the descriptors can be used a metric based on the Euclidean distance ,
= . , / + , 1 , where
4 4 , / = ∑8674 ∑5674 , − , 3
4 4 ; , 1 = ∑8674 ∑5674 , −
3; = 1, , = 1, ,
For each k-point of the test picture the nearest l-point of the array condition:
/89/:89; <:=> , thereby defining a two-dimensional array of coordinates , = , ; = 1, , corresponding to the positions of special points _# $ , _# $ , = 1, on the current digital picture from UAV cameras. Therefore, an array can be entered into consideration
?9<@A = BCD=, BCD=, , ; = 1, , in which the indices of special points on the digital terrain image are matched with some coordinates from the array . (1) (2)
Let the _E , coordinate array be obtained as a result of comparing the descriptors of special points on the picture and the data of the array. It is necessary to determine the point _ F , , which contains the most likely coordinates near which the aircraft is located.
There is always a possibility that the obtained points (2), which are assigned the coordinates of
the array, may belong to the image of the landmark object in the test image, and may just
can be found from the
coincidentally be similar to such special points. If the two-dimensional density distribution [
As an estimate of the two-dimensional distribution density by the array, we can choose construction and analysis of a histogram of relative frequencies to localize rectangular areas of the probable location of landmarks, and already within the specified areas, we can determine a typical value among and , = I1II,III. For example, taking about the central point of the corresponding rectangle.
To construct a two-dimensional histogram of the relative frequencies of the coordinate distribution, the following is proposed. Let some (in general, arbitrary) ℎA>0, ℎJ>0.
Let's set the division KLM,LN of the area of implementation of coordinates , , = 1, 0, − 1, = 0, − 1 , where
, upper left corners of which are determined by points with indices KLP,LQ: /89/89 , - the number of areas in the respective directions , the = 6S,T ; >/89,
E = W1, U = XA: 7LAMYZ[\ F , ]J: 7LJNYZ[^ ,
0, ℎ _ $ .
-and the area of division KLP,LQ, where the condition is true
Some , 88,F55 contain the geometric location of the landmark object, and the central point of taken as the position of the aircraft , _ F , : a = + 0,5 A + 0,5 J/89/89c.
If we assume that the search object on the test image can be located within several areas, then for their localization it is enough to select the areas where the condition is fulfilled
U88,55 ≥ Ueℎf@DℎC <, where UeLf@DLC < is some threshold value.
Then the position of the aircraft can be chosen as the weighted average of the coordinates of the centers of the rectangles, where the condition (3) is fulfilled, i.e.:
_ F , : ; = ̄ ; = ̄ >, where ̄ = ∑8V8697iS ∑5T5/6i7S 85 ⋅ ∑ZrZp[qno ∑jkmZkkpY⋅lqZnko jZk⋅lZk ; ̄ = ∑8V8697iS ∑5T5/6i7S 85 ⋅ ∑ZrZp[qno ∑jkmZkkpY⋅lqZnko jZk⋅lZk ; and if the object occupies several areas, then:
= ∑8V8697iS ∑5T5/67iS U88,55 ⋅ s88,55.
That in the last case, the estimation of the location of the aircraft based on weighted averages (4) is robust, because improbable coordinates are simply taken into account.
We formalize the criterion for verifying that the received coordinates of the features of the current picture really correspond in most cases to the area where the aircraft is located.
For example, if a complete inconsistency is observed for the components of the array (2), then this is evidence that the determined coordinates of the features , do change ordinarily due to the change in the positions _# $ , _# $ = 1, of special points in the picture.
As an estimate of the degree of consistency, it is possible to propose the use of Spearman or Kendall rank correlation coefficients between array vectors: , _# $ ; = 1, , , _# $ ; = 1, , (5) (6) , _# $ ; = 1, та , _# $ ; = 1, .
Let the corresponding rank array be obtained for any of the specified arrays , where ?, , l, – ranks, i.e. serial numbers of the variant in variation series by ?, , l, ; or = = 1, 1,
and _# $ or _# $ = 1, ; index E = ; , >; index s = ; _# $, _# $>.
The value of Spearman's rank correlation coefficient v̂x is calculated according to the formula y v̂x = 1 − T Tz7S ∑T6S , 3, де , = ?, − l, .
Significance v̂x is determined on the basis of a hypothesis {i: vx = 0, for the verification of which a statistical characteristic is it has t -distribution with the number of degrees of freedom = − 2 .
And if = |}~√T73,
. S7|}~z | | ≤ S73, , S7z, - quantile of the t-distribution; - the probability of an error of the first kind when accepting a hypothesis {i (level of significance), when here is no consistency between the determined coordinates of special points and their location on the digital image, and the obtained coordinates of the location of the UAV should raise doubts.
If (6) is true for each of arrays (5), when the position of the aircraft _ F , can be considered statistically justified.
For a more reliable assessment of the consistency of the coordinates of the selected features with their positions on the digital image, it is advisable to check the corresponding hypotheses for all data arrays (5), but only for those that fall into the partition areas for which condition (3) is fulfilled. In this case, it should be expected that random "coincidences" of descriptors will be characteristic of points that are in areas with a relatively low level of relative frequencies. In further research, it is worth paying attention to the possibility of evaluating the consistency of elements of arrays (5) separately for each of the elementary areas of division and, perhaps, taking into account for further use only those points where such a correlation occurs.
Let the analysis be subject to the video from the observed camera, the field of vision of which is directed vertically downwards. For two consecutively received frames with a frequency of, for example, 5 seconds, it is necessary to calculate the displacement, namely the real distance that was covered in the interval between the frames and the direction of movement.
For more accurate calculations, you can add a check of the value of the angle of inclination of the machine (roll and pitch), and use only those frames that were recorded at the time when the inclination of the machine was released. Provided that the image is obtained too much due to long, large angles of inclination, which allow when turning or caused by weather factors, it is possible to stop the movement, until the moment when it is recommended that the leveling takes place. At a sufficiently high altitude, or a low speed of movement, an increase in the interval between receiving frames for processing is possible.
A special factor that can affect the quality of determining the distance traveled by an aircraft is the case when the image is subjected to various deformations such as fish-eye or distortion.
Distortion is primarily able when using zoom lenses, and the higher the zoom ratio, the more able it is. In addition, the level of changes may vary depending on the distance to the object, in some cases a close object may be distorted, while a distant one will be without deformations.
Thus, only the central area of the image, which is approximately 50-70% of the area of the entire image, can be recommended to be used to determine special points. The specified approach should contribute to reducing the calculation error caused by deformations.
Before starting processing, it is also a desirable procedure to scale the received image by reducing it, which helps to reduce the load on the image processing microcontroller. For example, you can recommend reducing the image to 600x480.
Taking into account the comments made about the image received from the cameras of the target load of the aircraft, it is possible to proceed to its processing in order to determine the displacement.
Let's consider the algorithm (Figure 1) for calculating the displacement of the aircraft in more detail with a description of each step. Let B1 and B2 be a set of singular point descriptors and corresponding pixel indices of two consecutive images !S =
S , , S , ; = −3,3, = −3,3 , _# $S, _# $S , = 1, S , !3 = 3 , , 3 , ; = −3,3, = −3,3 , _# $3 , _# $3 , = 1, 3 , and array B1 was received earlier than B2.
Algorithm. Determining the displacement of the aircraft based on the analysis of two consecutive images from the target load cameras of the unmanned aircraft during movement.
Step 1. The first step is to get a zoomed image from the camera.
Step 2. We define arrays of special points on the image and their descriptors.
Step 3. We check whether there are points and descriptors for comparison.
Step 4. If there are no points for comparison (possible only when processing the first frame), remember the points found (see Step 2) and their descriptors as points for comparison. We get a new image and go to point 1.
Step 5. If there are special points for comparison, we compare their descriptors in order to detect points that are present in the images.
Step 6. We subtract the pixel indices of pairs of points that coincide on images, thus obtaining a set of displacements along the width and height of the image for each of the key points.
Step 7. We average the values obtained in point 6 of the value, thereby obtaining an estimate of the average displacement of local features: _# $ = ∑Zpo 8_BCDZo78_BCDZz , _# $ = ∑Zpo 5_BCDZo75_BCDZz ,
where T – the number of coincidences of singular points in two images.
Step 8. Based on expression = < , we calculate the value of the distance per 1 pixel of the Y image (to reduce the number of calculations, we check for a change in height, and if the height has nspecial points changed or has changed insignificantly, we leave the previous value of the distance).
Step 9. Multiplying the values _# $ and _# $ obtained (see clauses 8 and 7) and we get an estimate of the average displacement of the camera relative to the earth's surface for the time that has passed between the moments of receiving frames.
A conclusion about the direction of movement of an unmanned aerial vehicle can be made based on the sign of the obtained estimates _# $ and _# $
Step 10. We save the found (see Step 2) points and their descriptors as points for comparison. Let's go to Step 1.
Thus, using the given algorithm, it is possible only to estimate the distance the aircraft has moved, but also to obtain the direction of such movement. Therefore, by fixing these indicators during the flight task, it is possible to obtain an estimate of the location with reference to the coordinates. However, the latter is possible only after the orientation of the initial image has been performed. In addition, the given algorithm can acquire refinements for the case of performing various types of maneuvers by an aircraft, so practical studies of this issue go beyond the scope of the purely theoretical material of the current section.
optical channel
The essence of the simulation is to replace the input video stream: instead of a real flight, test images are used with dimensions that significantly exceed the resolution of the video camera.
Accordingly, the real flight is replaced by a sliding window (simulation of the field of vision of
the UAV camera [
In the selected zone 1 of Figure 3 there are parameters for the filtering method of special points.
To set the signal loss point, press "Set GPS signal loss point", and to set the base - "Set base". After that, we get the result, which can be seen in Figure 4.
After that, if the user presses the "Next step" button, the process of positioning the drone and returning it to the base is described in the previous part. Namely, first the drone searches for its position (Figure 5), after which it flies straight to the base (Figure 6).
The result of the first test can be seen in fig. 5, 6. They show that the drone will return to the base if there is enough information on SPECIAL POINTS from the map and a clear enough picture from the drone itself. It can be seen that due to the small size of the UAV camera, it took a long time to find the position, as the drone flew in a spiral for a long time.
When the size of the image from the camera increases, the speed of finding increases, and the possibility of correcting the location increases, as can be seen in zone 1 of Figure 7.
When the drone flies for a long time without determining the position during the flight, the position search takes place again (Figure 8). It can be seen that the developed algorithm works correctly, but position search is required.
Testing was also conducted on different images with different image quality (Figure 9). It has been confirmed that the method works on different images, and although it depends on the quality, it can handle low-quality images.
Thus, as a result of performing a simulation test of the method proposed in this section, the expected results were obtained, which confirms its correctness.
Mathematical support for the navigation of an unmanned aircraft along an optical channel must take into account the objectively existing limitations associated with the hardware capabilities of the onboard computer and a number of assumptions. The formal stages of determining the position of the aircraft and the conditions for their implementation have been formulated.
The procedure for determining the coordinates of special points during a flight mission and preparing data for further analysis in order to determine the position of the aircraft is presented in detail.
For the first time, a method of preliminary determination of the coordinates of the aircraft based on the non-parametric estimation of the two-dimensional density function of the distribution of the coordinates of special points is proposed.
A method of verifying the reliability of the previously determined position of the aircraft based on the evaluation of the consistency coefficients of the determined coordinates of special points of the image and their actual position on the image is proposed.
Approaches to solving the problem of estimating the location of an unmanned aircraft in cases where it is not possible to reliably determine the coordinates based on the proposed method are defined. In particular, the procedure for recording changes in the position of the aircraft based on the analysis of consecutive frames of aerial photography has been formalized.
The software was developed to perform a simulation test of the proposed aircraft navigation technology using an optical channel. A description of the structure of the developed software is given.
Simulation testing was performed using the developed software. Correctness of operation of the proposed technology on different images and in different situations has been confirmed.
Taking into account the confirmed correctness of the proposed method and the proven technology of the automatic control of the aircraft, further research should be directed to their unification and the creation of a suitable unmanned aircraft with the possibility of autonomous navigation along the optical channel.
The author(s) have not employed any Generative AI tools. [15] T. Hubanova, R. Shchokin, O. Hubanov, V. Antonov, P. Slobodianiuk, S. Podolyaka, Information technologies in improving crime prevention mechanisms in the border regions of southern Ukraine, Journal of Information Technology Management 13 (2021) 75-90. [16] V. Kortunov, I. Dybska, G. Proskura, A. Kravchuk, Integrated mini INS based on MEMS sensors for UAV control, IEEE Aerospace and Electronic Systems Magazine 24 (1) (2009) 41–43. doi: 10.1109/MAES.2009.4772754. [17] F. M. Zakharin, S. A. Ponomarenko, Unmanned Aerial Vehicle integrated navigation complex with adaptive tuning, in: Proceedings of 4th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), IEEE, Kyiv, Ukraine, 2017, pp. 23–26, doi: 10.1109/APUAVD.2017.8308768. [18] S.O. Ponomarenko, F.M. Zakharin, Features of modeling the process of complex processing of navigation information in on-board complexes of unmanned aerial vehicles, in: Proceedings of XIX International Conference Dynamical System Modeling And Stability Investigation, Kyiv, Ukraine, 2019, pp. 189–192. [19] P. Prystavka, O. Cholyshkina, T. Sorokopud Experimental study of distributions differential invariants based on spline image models, CEUR Workshop Proceedings 3530 (2022) 163-172.
URL: https://ceur-ws.org/Vol-3530/paper16.pdf. [20] P. Prystavka, O. Cholyshkina, Comparative analysis of differential invariants based on the spline model for various image distortion, Advanced Information System 4(4) (2020) 70–76. [21] P. Prystavka, A.V. Chirkov, V.I. Sorokopud, D.V. Zdota, Simulation testing of the information technology of aircraft navigation by optical channel, Science-Based Technologies 46(3) (2020) 370–377. doi: 10.18372/2310-5461.47.14935. [22] Y. Averyanova, et al., UAS cyber security hazards analysis and approach to qualitative assessment, In: S. Shukla, A. Unal, J. Varghese Kureethara, D.K. Mishra, D.S. Han (Eds.), Data science and security, volume 290 of Lecture Notes in Networks and Systems, Springer, Singapore, 2021, pp. 258–265. doi: 10.1007/978-981-16-4486-3_28. [23] N. S. Kuzmenko, I. V. Ostroumov, K. Marais, An accuracy and availability estimation of aircraft positioning by navigational aids, in: Proceedings of 5th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC), IEEE, Kiev, Ukraine, 2018, pp. 36–40. doi: 10.1109/MSNMC.2018.8576276. [24] V. Kharchenko, I. Chyrka, Detection of airplanes on the ground using YOLO neural network, in: Proceedings of 17th International Conference on Mathematical Methods in Electromagnetic Theory (MMET), IEEE, Kyiv, Ukraine, 2018, pp. 294–297. doi: 10.1109/MMET.2018.8460392. [25] R. S. Odarchenko, S. O. Gnatyuk, T. O. Zhmurko, O. P. Tkalich, Improved method of routing in UAV network, in: Proceedings of International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), IEEE, Kyiv, Ukraine, 2015, pp. 294–297. doi: 10.1109/APUAVD.2015.7346624.