Enhanced UAV Tracking through Multi-Sensor Fusion and Extended Kalman Filtering

Abstract

This research introduces a sophisticated method for UAV tracking utilizing multi-sensor fusion and Extended Kalman Filter (EKF) techniques. We model a sophisticated 3D UAV trajectory alongside diverse sensor information, encompassing RF signal strength, radar range and velocity, audio signals, and optical location data. The EKF technique is employed to estimate the drone's position and velocity, exhibiting enhanced tracking precision relative to individual sensor observations. The simulation integrates variable noise levels and adaptive estimation of measurement noise covariance to improve the resilience of the EKF. Real-time visualization of sensor data and EKF estimates facilitates instant evaluation of the algorithm's eficacy. The results indicate that the EKF eficiently eliminates sensor noise, yielding a more refined and precise trajectory estimation. The analysis of sensor data indicates the average signal strengths and standard deviations for each sensor category. The EKF exhibits a 12.5% enhancement in tracking accuracy relative to unprocessed optical sensor data, with mean tracking errors of 0.07 m for the EKF compared to 0.08 m for the optical sensor independently. This study emphasizes the eficacy of multi-sensor fusion and EKF implementation in improving drone tracking capabilities across diverse applications, such as surveillance, search and rescue, and air trafic management.

Keywords

• eol>Multi-Sensor Fusion
• Extended Kalman Filter
• UAV Tracking
• Adaptive Filtering
• Real-time Visualization

1. Introduction

Unmanned Aerial Vehicles (UAVs), commonly referred to as drones, are extensively utilized throughout military, commercial, and recreational domains. With the rise of their presence in airspace, precise tracking systems are essential for air trafic control, surveillance, and environmental monitoring. Singlesensor UAV tracking methodologies frequently exhibit deficiencies in accuracy and robustness. Every sensor possesses advantages and disadvantages, influenced by external variables. Optical sensors are inefective in poor visibility conditions, whereas radar systems encounter clutter or interference. The issues have resulted in the advancement of enhanced tracking approaches. Multi-sensor fusion improves UAV tracking by integrating data from RF transmissions, radar, acoustic sensors, and optical systems. This method enhances capabilities and reduces shortcomings, yielding a more precise representation of UAV position and motion, even under harsh conditions or sensor malfunctions. Integrating sensor inputs poses issues in data synchronization, noise management, and real-time processing. Advanced ifltering and estimating techniques are required to address diverse sensor properties. The Extended Kalman Filter (EKF) proficiently resolves these challenges. It addresses non-linear systems, rendering it appropriate for UAV tracking, where sensor data and UAV states frequently exhibit non-linearity. EKF integrates system model predictions with sensor data to perpetually enhance UAV state estimation. This research assesses multi-sensor fusion utilizing the Extended Kalman Filter for unmanned aerial vehicle tracking. A simulation-based methodology illustrates a sophisticated 3D drone route alongside diverse sensor measurements. The EKF approach assesses the location and velocity of UAVs, hence improving tracking precision. The research advances by:(i)Establishing a realistic simulation environment incorporating many sensor kinds. (ii) Formulating an Extended Kalman Filter algorithm to address noise and errors. (iii)Real-time monitoring of sensor data and Extended Kalman Filter forecasts. (iv) Evaluating enhanced tracking accuracy by comprehensive analysis.

This study seeks to showcase sophisticated UAV tracking systems and highlight the advantages of multi-sensor fusion and Extended Kalman Filter techniques. The study suggests strategies to improve

UAV safety and eficacy in urban air transport, border monitoring, and emergency response. With the proliferation of UAV usage, dependable and accurate tracking solutions are imperative. This research provides a framework for practical implementation, potentially enhancing UAV tracking and multisensor fusion to improve the resilience and efectiveness of UAV management systems.

2. Literature Reviews

In recent years, substantial progress has been made in UAV tracking and localization, with multi-sensor fusion and Extended Kalman Filter (EKF)[1] algorithms proving to be useful in enhancing accuracy and robustness. This paper examines significant advancements in these domains, emphasizing their applications in UAV tracking and navigation[2].

Multi-sensor fusion[3] has become significant because it utilizes the complimentary benefits of

several sensor kinds. Adaptive multi-object tracking methodologies employing sensor fusion, which amalgamates data from cameras and LiDAR, have demonstrated improved tracking efectiveness in intricate urban settings[4]. The integration of Global Navigation Satellite System (GNSS)[5] and Inertial

Measurement Unit (IMU) data is essential for UAV navigation.

Deep learning has significantly impacted multi-sensor fusion techniques[ 6]. A federated fusion

architecture integrating GNSS, IMU, monocular camera, and barometer data exhibited sub-meter location precision across diverse situations. In indoor UAV localization[7], where GNSS signals are frequently inaccessible, research has investigated alternate sensor combinations, attaining centimeterlevel precision in intricate indoor settings[8].

Comparative evaluations of various filtering methods, including the EKF, Unscented Kalman Filter

(UKF), and Particle Filter (PF), have elucidated the trade-ofs among accuracy[ 9], processing eficiency, and resistance to non-linearities. The amalgamation of optical odometry[10] with supplementary sensor data has demonstrated the capacity to improve UAV navigational accuracy, especially in environments lacking GPS.

Adaptive filtering[ 11] methods have garnered interest for their capacity to manage fluctuating sensor precisions and environmental circumstances. An Adaptive Extended Kalman Filter (AEKF) [12]for determining ground target locations with UAV-based observations exhibited enhanced performance in situations characterized by fluctuating sensor noise levels[13].

Recent research have investigated the amalgamation of machine learning techniques with traditional

ifltering methods[ 14], demonstrating improved accuracy and convergence velocity relative to standard

EKF implementations, particularly in dificult urban canyon environments. The integration of semantic information in UAV tracking has emerged as a viable avenue, exhibiting enhanced resilience[15] in intricate urban settings by combining semantic segmentation of camera images with traditional sensor data.

These achievements underscore the continuous development of UAV tracking and localization methods, highlighting the significance of adaptive, multi-sensor strategies in tackling the problems posed by various operational contexts.

3. Mathematical Model

3.1. State Space Model for UAV Tracking

3.1.1. State Vector: The state vector ∈ R6 is defined as: = [, , , , , ]

Let (, , ) ∈ R3 denote the three-dimensional position of the UAV in spherical coordinates, and ( , , ) ∈ R3 signify its three-dimensional velocity components in the respective directions. 3.1.2. State Transition Model:

The discrete-time state transition model is given by:

Where:

( + 1) = Ψ( ) ( ) + ( ) • Ψ( ) ∈ R6× 6 is the state transition matrix at time step • ( ) ∼ (0, Ω( )) is the process noise, assumed to be zero-mean Gaussian with covariance Ω( )

The state transition matrix Ψ( ) is defined as:

Ψ( ) = ︂[ Λ 3 Λ 3]︂

O3 Λ 3

Where Λ 3 is the 3 × 3 identity matrix, O3 is the 3 × 3 zero matrix, and is the time step. 3.1.3. Measurement Model:

The measurement model is given by:

Where:

( ) = Γ( ) ( ) + ( ) Γ( ) = [Λ 3

O3] • ( ) ∈ R3 is the measurement vector • Γ( ) ∈ R3× 6 is the measurement matrix • ( ) ∼ (0, Σ( )) is the measurement noise, assumed to be zero-mean Gaussian with covariance Σ( )

The measurement matrix Γ( ) is defined as:

3.1.4. Process Noise Covariance: The process noise covariance matrix Ω( ) ∈ R6× 6 is defined as:

Ω( ) = diag([ , , , , , ])

Where , , , , , and are the variance components for position and velocity in spherical coordinates. ( 1 ) ( 2 ) ( 3 ) ( 4 ) ( 5 ) ( 6 ) 3.1.5. Measurement Noise Covariance: The initial measurement noise covariance matrix Σ( ) ∈ R3× 3 is defined as:

An adaptive estimation of Σ( ) is implemented:

Σ( ) = diag([ , , ]) Σ adapted( ) = (1 − )Σ( ) + ( ( ) ( )) • is the adaptation factor • ( ) = ( ) − Γ( )ˆ( | − 1) is the measurement residual 3.1.6. Extended Kalman Filter Algorithm

The EKF algorithm is implemented with the following steps:

Prediction Step: Update Step: ˆ( | − 1) = (Ψ − 1)ˆ( − 1| − 1) Π( | − 1) = (Ψ − 1)Π( − 1| − 1)Ψ( − 1) + Ω( − 1) ( ) = ( ) − Γ( )ˆ( | − 1) Σ( ) = Γ( )Π( | − 1)Γ( ) + Λ( )

K( ) = Π( | − 1)Γ( )Σ( )− 1 ˆ( | ) = ˆ( | − 1) + K( ) ( ) Π( | ) = ( − K( )Γ( ))Π( | − 1) 3.1.7. Sensor Models: Multiple sensor models are implemented to simulate diverse measurement sources: d) Acoustic Sensor: ( ) = sin(2 ( )) + ( )

e) Optical Sensor: ( ) = ( ) + ( )

Where * ( ) represents sensor-specific noise terms and ( ) represents the UAV’s position at time step . This relationship inherently introduces high sensitivity to distance changes, contributing to signal variability. 3.1.8. Dynamic Noise Modeling:

The implementation incorporates a dynamic noise level:

Where base = 0.05 and ( ) represents time. level( ) = base · (1 + 0.5 · sin(0.05 · ( ))) ( 7 ) ( 8 ) ( 9 ) ( 10 ) ( 11 ) ( 12 ) ( 13 ) ( 14 ) ( 15 ) (16) (17) (18) (19) (20) (21) 3.1.9. Complex UAV Trajectory:

The UAV’s position ( ) follows a non-linear path:

( ) = ⎣cos(0.05 ( ))⎦ ⎡ sin(0.1 ( )) ⎤ 0.1 ( )

This trajectory combines sinusoidal motion in the x-y plane with linear motion in z, resulting in continuous and complex changes in the UAV’s distance from the RF sensor. The time-varying nature of this path contributes significantly to the variability of the RF signal strength.

4. Performance analysis

4.1. RF Sensor

The RF Sensor model replicates the inverse square law correlation between signal strength and distance, integrating variable Gaussian noise. The average signal intensity of 0.39 suggests regular UAV activity in proximity to the sensor. A standard deviation of 0.09 indicates considerable variability resulting from the inverse square relationship, fluctuating noise levels, and intricate UAV route. The illustration in Fig 1 depicts swift variations in signal intensity, a general sinusoidal pattern reflecting the UAV’s trajectory, and strength fluctuations aligned with proximity alterations. These attributes illustrate the model’s capacity to replicate authentic RF sensor performance in UAV tracking contexts.

4.2. Radar Sensor Analysis:

4.2.1. Distance-Dependent Noise: The noise range is represented as: where: • ( ) is the range noise at time step • level( ) is the dynamic noise level • ( ) is the true range to the target ( ) = level( ) · ︂( 1 + ( ) ︂) 100 · ( 0, 1 ) (22) (23) • ( 0, 1 ) represents a standard normal distribution

This noise model escalates with distance, emulating real-world behavior wherein radar precision

generally diminishes for targets at greater distances. The term (1 + 1(00) ) ensures that the noise standard deviation increases linearly with range, with a 1% increase per meter of distance. 4.2.2. Occasional Dropouts: The code incorporates a 5% probability of radar range measurement failure, resulting in the value being set to NaN. This emulates sporadic sensor malfunctions or signal obstructions, enhancing the authenticity of the scene. 4.2.3. Relative Precision: 4.2.4. Visualization Notwithstanding the increased noise and dropouts, the radar measurements exhibit reasonably low standard deviations (0.41 m for range, 0.06 m/s for velocity), signifying commendable precision. (a)In Fig2 The radar range plot (top-right) displays steady data interspersed with occasional gaps (dropouts). (b) In Fig 3 The radar velocity plot (middle-left) exhibits a more consistent pattern than the range measurements, probably owing to the lack of simulated dropouts in the velocity data. 4.3. Acoustic Sensor 4.3.1. Statistical Results: (a)Mean Amplitude: -0.01. (b)Standard Deviation: 0.71. 4.3.2. Visualization (a)In Fig 4 illustrates the acoustic sensor plot (middle-right), which exhibits a distinct sinusoidal pattern accompanied by noise. (b) It illustrates that the acoustic signal (magenta line) exhibits a uniform periodic pattern throughout the simulation. 4.3.3. Interpretation of Results: (a)Near-Zero Mean (-0.01): (i)This signifies that the signal oscillates symmetrically about zero, as anticipated for a sine wave with no DC ofset. (ii)The minor variation from precisely zero is probably attributable to the introduced noise and limited sampling. (b) Standard Deviation (0.71):(i)This value denotes the dispersion of the signal amplitudes. (ii) For a pure sine wave with amplitude 1, the standard deviation would be √12 ≈ 0.707. (iii)The measured value of 0.71 indicates a relatively low noise level, allowing the underlying sinusoidal pattern to prevail. 4.3.4. Implications for UAV Tracking: (a)Periodic Pattern Detection:(i) The distinct sinusoidal characteristics of the signal can facilitate the identification of cyclic motion patterns of the UAV. (ii)It may assist in recognizing recurrent actions or environmental factors afecting the UAV’s movement. (b)Relative Change Detection: (i)Although unsuitable for exact positioning, the auditory signal may be beneficial for identifying relative alterations in the condition of the UAV. (ii)Sudden fluctuations in amplitude or frequency may signify abrupt maneuvers or environmental disturbances.

4.4. Comparison with Other Sensors:

The multi-sensor methodology amalgamates RF for proximity, radar for range and velocity, and auditory signals. The RF changes considerably, confounding the location, but facilitating proximity detection. Radar data reveals an expanding range and consistent velocity, indicating fluid UAV motion. Acoustic data indicates periodic rhythms, which may be beneficial for detecting cyclic motion. This combination improves tracking accuracy, with EKF decreasing the mean error to 0.07 m, in contrast to 0.08 m for optical sensing alone.

4.5. EKF Performance

4.5.1. Tracking Accuracy EKF demonstrates improved performance in UAV tracking compared to raw optical sensor data. Key ifndings include: • Mean Tracking Error (Optical): 0.08 m • Mean Tracking Error (EKF): 0.07 m • EKF shows a 12.5% improvement in tracking precision 4.5.2. Key Features 4.5.3. Adaptive Measurement Noise Covariance

The EKF employs an adaptive estimation of the measurement noise covariance:

Radapted = (1 − )R + (yy ) (24) where = 0.1 is the adaptation factor. 4.5.4. Multi-Sensor Fusion

The EKF efectively integrates data from optical, RF, radar, and acoustic sensors.

4.5.5. Robustness The system maintains accurate tracking despite simulated radar dropouts (5% probability of NaN readings). 4.5.6. State Transition and Measurement Models

Linear models are used for both state transition and measurement:

State transition: ( + 1) = Ψ( ) ( )

Measurement: ( ) = Γ( ) ( ) (25) (26) 4.5.7. Noise Modeling • Process noise covariance Q = diag([0.1, 0.1, 0.1, 0.01, 0.01, 0.01]) • Initial measurement noise covariance R = diag([0.1, 0.1, 0.1])

The EKF’s capability to manage various sensor inputs, adjust to varying noise circumstances, and deliver seamless state estimates renders it highly appropriate for UAV tracking in intricate situations. Although the 12.5% accuracy boost is considerable, there exists potential for additional improvements via sophisticated sensor fusion methodologies or non-linear filtering techniques.

4.6. Comparison with Other Filtering Models

The analysis of the filtering models in Fig. 9 highlights the advantages and drawbacks of each method in the tracking of UAVs. The EKF demonstrated superior performance, achieving a tracking error of 0.07m, which surpassed optical tracking of 0.08m by 12.5%. EKF’s capacity to manage adaptive noise and assimilate multi-sensor data proficiently enhanced its accuracy and resilience. The Unscented Kalman Filter (UKF) exhibited a notably elevated inaccuracy of 1.33m and encountered challenges related to computational complexity and nonlinearities in this implementation. Likewise, the Particle

Filter (PF), with an inaccuracy of 1.24m, faced dificulties in handling noise and preserving accuracy

despite its probabilistic structure. Optical tracking functioned as a baseline but was lacking in the sophistication ofered by advanced filtering methods. The Fig 10 graph compares trajectory estimations from various filtering methods with the actual UAV path. The actual trajectory (shown by the solid black line) is monitored using several approaches, with the EKF estimation (depicted by the blue dashed line) demonstrating the closest alignment. The UKF (green dotted line) and PF (magenta dash-dot line) implementations exhibit greater deviations from the correct path, particularly in the curved segments of the trajectory. The positional accuracy is preserved within approximately ±0.5m on both the X and Y axes, with the most considerable variances occurring during intricate maneuvers.

5. Future Studies

5.1. Advanced Machine Learning Integration

Future research should investigate deep learning architectures to enhance multi-sensor data fusion processes. The development of neural network-based systems for real-time noise prediction and ifltering would improve tracking accuracy. The implementation of reinforcement learning algorithms may facilitate automated sensor calibration that adjusts to changing environmental conditions. The examination of recurrent neural networks for trajectory forecasting could enhance predictive capabilities in complex scenarios.

5.2. Enhanced Environmental Adaptability

The integration of environmental context awareness systems facilitates dynamic adaptation to varying atmospheric and terrain conditions. The implementation of advanced sensor weighting algorithms informed by real-time environmental factors would enhance performance. The creation of efective solutions for complex urban settings and areas lacking GPS would enhance operational capabilities.

5.3. Multi-UAV Tracking Systems

Future research should explore collaborative tracking algorithms for the coordination of multiple

UAVs within complex airspace. The development of eficient inter-UAV communication protocols

would improve overall tracking accuracy by facilitating shared positional awareness. The development of distributed sensor fusion architectures enhances system scalability and redundancy. The integration of advanced collision avoidance systems will ensure safe operations in densely populated UAV environments.

5.4. Real-World Implementation

Extensive field testing in various operational scenarios would confirm system performance in real

world conditions. Research on hardware optimization must prioritize eficient real-time processing and the reduction of power consumption. Analyzing power requirements will result in optimized energy management strategies. Integration studies with existing air trafic management systems will facilitate seamless adoption into the current aviation infrastructure.

6. Conclusion

The current research demonstrates significant advancements in UAV tracking via the amalgamation of multi-sensor fusion and EKF methodologies. The research integrates various sensor modalities—RF signal strength, radar measurements, acoustic signals, and optical data—resulting in a 12.5% enhancement in tracking precision relative to single-sensor systems, with the EKF minimizing average tracking errors to 0.07 meters. The adaptive EKF implementation efectively handles dynamic noise conditions and sensor uncertainties, exhibiting enhanced performance relative to UKF and PF techniques, which shown limitations with mean errors of 1.33m and 1.24m, respectively. The simulation outcomes confirm the system’s capacity to address real-world problems, such as radar dropouts and fluctuating noise conditions while preserving tracking precision within ±0.5 meters on both the X and Y axes. As UAV applications proliferate across diverse industries, the approaches established in this study provides a viable basis for sophisticated tracking systems, possibly transforming areas like urban air mobility, search and rescue operations, and autonomous delivery systems.

Declaration on Generative AI

The author(s) have not employed any Generative AI tools.

References

[1] G. A. Einicke , L. B. White , Robust extended kalman filtering , IEEE transactions on signal processing 47 ( 1999 ) 2596 - 2599 .

[2] L. Gao , H. Xiang , X. Xia , J. Ma, Multi-sensor fusion for vehicle-to-vehicle cooperative localization with object detection and point cloud matching , IEEE Sensors Journal ( 2024 ).

[3] J. K. Hackett , M. Shah, Multi-sensor fusion: a perspective , in: Proceedings., IEEE International Conference on Robotics and Automation , IEEE, 1990 , pp. 1324 - 1330 .

[4] L. Liu , J. Tian , Z. Shi , J. Fan , Y. Rui , Multi-sensor fusion for multi-target detection and tracking , in: Autonomous Vehicles and Systems, River Publishers, 2024 , pp. 175 - 217 .

[5] B. Hofmann-Wellenhof , H. Lichtenegger , E. Wasle, GNSS-global navigation satellite systems: GPS, GLONASS , Galileo, and more, Springer Science & Business Media , 2007 .

[6] H. MAILKA , M. Abouzahir , M. Ramzi , An eficient end-to-end ekf-slam architecture based on lidar, gnss, and imu data sensor fusion for autonomous ground vehicles , Multimedia Tools and Applications 83 ( 2024 ) 56183 - 56206 .

[7] K. Wang , L. Kooistra , R. Pan , W. Wang , J. Valente , Uav-based simultaneous localization and mapping in outdoor environments: A systematic scoping review , Journal of Field Robotics ( 2024 ).

[8] O. J. Montañez , M. J. Suarez , E. A. Fernandez , Application of data sensor fusion using extended kalman filter algorithm for identification and tracking of moving targets from lidar-radar data , Remote Sensing 15 ( 2023 ) 3396 .

[9] S. A. Negru , P. Geragersian , I. Petrunin, W. Guo, Resilient multi-sensor uav navigation with a hybrid federated fusion architecture , Sensors 24 ( 2024 ) 981 .

[10] C. Xiang , C. Feng , X. Xie , B. Shi , H. Lu , Y. Lv , M. Yang , Z. Niu, Multi-sensor fusion and cooperative perception for autonomous driving: A review, IEEE Intelligent Transportation Systems Magazine ( 2023 ).

[11] A. H. Jazwinski , Adaptive filtering, Automatica 5 ( 1969 ) 475 - 485 .

[12] C.-W. Peng , C.-C. Hsu , W.-Y. Wang, High-accuracy lane geodetic coordinates extraction based on camera-lidar fusion , in: 2023 IEEE International Conference on Consumer Electronics (ICCE) , IEEE, 2023 , pp. 1 - 2 .

[13] K. Tian , M. Radovnikovich , K. Cheok , Comparing ekf, ukf, and pf performance for autonomous vehicle multi-sensor fusion and tracking in highway scenario , in: 2022 IEEE International Systems Conference (SysCon) , IEEE, 2022 , pp. 1 - 6 .

[14] S. Syamal , C. Huang , I. Petrunin , Enhancing object detection and localization through multi-sensor fusion for smart city infrastructure , in: 2024 IEEE International Workshop on Metrology for Automotive (MetroAutomotive) , IEEE, 2024 , pp. 41 - 46 .

[15] X. Wang , K. Li , A. Chehri , Multi-sensor fusion technology for 3d object detection in autonomous driving: A review , IEEE Transactions on Intelligent Transportation Systems ( 2023 ).