Object detection using Unmanned Aerial Vehicles (UAVs) introduces unique challenges compared to traditional methods, primarily due to the varying angles and altitudes from which images are captured. Conventional Convolutional Neural Network (CNN) implementations, while being the state-of-the-art in object detection, demand substantial memory and computational resources, posing dificulties for deployment on UAVs with limited onboard resources. Furthermore, these models typically generalize well within a limited distance range by training on images captured from various distances. However, UAVs capture images at a wide range of distances, complicating the model's ability to generalize efectively. In this work therefore, we present a design space exploration which aims to identify the efect of CNN parameters for UAV-based object detection at various altitudes by examining two critical parameters: input image resolution and network width (number of channels). We conduct extensive experiments to evaluate the efect of these parameters in terms of accuracy and computational eficiency across multiple altitudes. Lower resolutions reduce computational load but may compromise detection accuracy, while higher resolutions enhance accuracy at the expense of increased processing requirements. Similarly, varying the network width influences the balance between model complexity and detection performance. We showcase that the requirements vary significantly across diferent altitudes, demonstrating the potential of dynamic network structures that adjust parameters according to the altitude. Our findings provide insights into the optimal configuration of CNN parameters for UAV object detection across diferent altitudes, contributing to the development of more eficient and adaptable UAV vision systems. This research paves the way for more efective deployment of UAVs in various applications, from surveillance and search-and-rescue to environmental monitoring and beyond.
Unmanned Aerial Vehicles (UAVs), commonly known as drones, are increasingly employed across a
broad spectrum of fields, including but not limited to search and rescue operations [
Object detection on UAVs is a critical task due to its role in enhancing situational awareness, which is pivotal for safe navigation, obstacle avoidance, and the execution of missions that necessitate the identification of subjects of interest. However, several challenges are inherent to object detection using UAVs. Firstly, due to their nature, most UAV applications require the object detection task to be performed in real-time. On-board computational and power resources are limited, thereby requiring the deployment of models that are computationally eficient while maintaining the required accuracy for the task at hand. Secondly, unlike in traditional object detection tasks, UAV-based tasks encounter images from a wide variety of angles and altitudes, potentially impacting the generalization capabilities of the model. Furthermore, diverse environmental conditions such as reflections, smoke, and adverse weather can obscure objects and degrade image clarity, further complicating the detection task.
Existing research in UAV-based object detection demonstrates significant advancements in handling
various challenges posed by UAV imagery. These studies collectively emphasize the importance of
eficient detection methods that satisfy the constraints of UAV platforms. Techniques such as adaptive
feature extraction [
In this study, we examine the efects of two critical parameters–input image resolution and network width–on the performance and eficiency of CNN models for UAV-based object detection. We hypothesize that higher input resolutions and wider networks can capture more details and features, respectively; however, these advantages come with an increased computational load. Our objective is to identify the optimal values for these parameters across diferent altitude ranges to optimize the accuracy-to-performance trade-of. We aim to use these findings to develop a dynamic network structure capable of adjusting these parameters based on a learnable dynamic decision point (rather than a static threshold) tailored to each input image. This will be implemented using reconfigurable network structures (either hardware or software), enabling us to enhance the accuracy-to-performance balance.
Specifically, the contributions of this research are twofold: • We perform extensive evaluation of the efect of changing the two parameters, input resolution and network width (number of channels), on the accuracy, performance and memory requirements of the object detection CNN (tiny YOLOv7) at various altitude ranges. • We demonstrate that the optimal values of these parameters varies significantly with altitude, highlighting the need for dynamic adjustments based on real-time altitude data.
The remainder of this paper is structured as follows: Section 2 reviews the existing related work. In Section 3, we detail our methodology. Section 4 presents the experimental results. Finally, Section 5 discusses some of the challenges and potential future work and Section 6 concludes our study.
Over the years, numerous advancements have been made in the field of UAV-based object detection, particularly focusing on improving accuracy and eficiency. Several studies have aimed to address the unique challenges posed by UAV imagery, such as small object size, high density, and varying viewpoints.
A significant portion of the literature has focused on improving detection algorithms and network
architectures. Tan et al. [
Comprehensive reviews of deep learning techniques applied to UAV-based object detection have
highlighted the importance of lightweight models for deployment on UAVs with limited computational
resources. Wu et al. [
Real-time object detection in UAV imagery has also gathered significant attention due to its importance
in scenarios like emergency rescue and precision agriculture. Cao et al. [
Several innovative methodologies have been introduced to enhance UAV object detection. Bazi et
al. [
Despite the extensive research in UAV object detection, our work is the first to examine dynamic parameters for UAV object detection, exploring the need for adaptable and flexible detection methods. By focusing on dynamic parameter adjustment, our approach aims to address the limitations of static models in varying UAV operational conditions, enhancing detection accuracy and eficiency across diverse environments and scenarios.
The parameter values explored in this study are shown in Table 1. The input image resolutions were varied from 1088x1088 pixels to 256x256 pixels, allowing us to investigate the impact of diferent levels of detail and computational demands on the object detection performance. Additionally, the network width was systematically adjusted using multipliers ranging from 1x, which represents the original model structure, to 0,01x. This adjustment helped us evaluate how the model’s number of channels in the convolutional layers influenced its detection capabilities and eficiency.
For evaluation, we employed several metrics to provide a comprehensive assessment of the model’s performance. Mean average precision (mAP) was utilized to quantify the accuracy of object detection across the diferent parameter settings. To evaluate computational eficiency, we measured the multiplyaccumulate (MAC) operations, which reflect the computational load required for processing, and the network size in megabytes (MB), which indicates the model’s memory requirements.
For the object detector, we selected the tiny YOLOv7 model [
As the dataset for our study, we utilized the Multi-Altitude Aerial Vehicles Dataset [
To comprehensively analyze the impact of varying parameters at diferent altitudes, we segmented the original dataset into five distinct subsets, each corresponding to specific altitude ranges: • 50–100 meters • 150–200 meters • 250–300 meters • 350–400 meters • 450–500 meters
In addition to these altitude-specific subsets, we also utilized the entire dataset, which includes images from all altitude ranges. This dataset, referred to as mix_alt, serves as a baseline for comparison against the individual altitude-specific subsets.
For each of these six datasets (the five altitude-specific subsets and the mix_alt dataset), we conducted extensive training experiments. We systematically varied the input image resolution and network width across all possible combinations while maintaining other parameters at their default values. This approach allowed us to evaluate the influence of the two parameters on the performance metrics.
The results in terms of mean Average Percision (mAP) by setting the threshold of Intersection over Union (IoU) to 0.5 for each dataset and parameter combination are illustrated in Figure 1. As expected, lower altitude datasets require less computationally intensive models, with less parameters, to achieve high accuracy. This is due to the larger size, greater clarity and detail available in lower altitude images, which simplifies the task.
Conversely, as the altitude increases, the detection accuracy tends to diminish unless higher parameter values are utilized. This can be attributed to the increased complexity of identifying objects from higher vantage points, where objects appear smaller and less distinct. Thus, higher altitudes necessitate models with greater capacity to maintain comparable levels of accuracy, reflecting the need for more detailed feature extraction and processing capabilities.
Interestingly, mix_alt demonstrates an intermediate performance, performing better than the higher altitude-specific models. This is because mix_alt is trained on the whole dataset, including the easier, lower-altitude images, whereas the higher altitude-specific models are trained only on the hardest images of the dataset.
MAC operations are a key indicator of computational eficiency in neural network models, representing the total number of multiplications and additions needed to process an image. By reducing the number of MAC operations, we can lower the computational load on the UAV, extending its operational time and enabling real-time processing.
In addition to MAC operations, the size of the model is another important factor afecting computational eficiency. Model size refers to the amount of memory required to store the model parameters, including weights, biases, and other configurations. This size directly influences the amount of onboard storage needed for the UAV to operate.
Figure 2 presents the computational eficiency results for each model configuration. As anticipated, both MAC operations and model size increase with larger parameter values. Notably, MAC operations scale uniformly with both parameters, whereas model size is more significantly afected by the width multiplier compared to the resolution. This suggests that for prioritizing a smaller model size, a lower width combined with a higher resolution is more efective than the reverse.
The most significant challenge is the seamless transition between models during inference. Currently, switching between models that are optimized for diferent altitude ranges can be computationally intensive and may introduce latency, which is detrimental to real-time processing requirements. To address this, we will explore the potential of dynamic networks that can reconfigure themselves on-the-fly to adapt to changing altitudes.
Reconfigurable hardware presents a promising solution to this challenge. By utilizing hardware
that can adapt its configuration based on the UAV’s altitude, it is possible to optimize the processing
pipeline for speed and eficiency. Field Programmable Gate Arrays (FPGAs) that support dynamic
reconfiguration could be investigated for this purpose [
Additionally, exploring algorithmic, non-manual methods for selecting the optimal parameters for each altitude range will streamline the process and reduce the large computational time and resources typically required for manual exploration, which scales exponentially as the network size and number of parameters increase. By employing automated approaches, the system could eficiently determine the best parameters based on the specific UAV’s capabilities and the task’s required accuracy. These methods would minimize human intervention, allowing UAVs to dynamically adjust to varying conditions and mission requirements while achieving an optimal accuracy-performance trade-of. This would enable the UAVs to adapt in real time without requiring labor-intensive manual tuning, enhancing operational eficiency.
In addition to technical advancements in model adaptability, there is a critical need to expand the datasets used for training and evaluation. Our current dataset has fixed altitudes and limited environmental diversity, which does not fully capture the complexities encountered in real-world scenarios. Therefore, we aim to create a comprehensive dataset that includes images captured at varying altitudes and under diverse environmental conditions such as diferent weather patterns, times of day, locations etc. This dataset would not only improve the robustness of the detection models but also provide a more rigorous benchmark for future research in UAV-based object detection.
Addressing the challenges of parameter optimization, seamless model transition, and dataset diversity will be crucial for advancing the field of UAV-based object detection. Through a combination of innovative algorithms, adaptable hardware solutions, and comprehensive data collection, we aim to significantly enhance the performance and applicability of UAV object detection systems.
Our study confirms that diferent altitude ranges necessitate distinct parameters to achieve optimal accuracy levels in UAV-based object detection. We have demonstrated that input image resolution and network width are critical factors that must be tuned according to the altitude from which images are captured.
The findings indicate that dynamic network structures, which adjust their parameters based on realtime altitude data, can substantially enhance both the eficiency and performance of object detection systems deployed on UAVs. Such an approach would not only optimize the accuracy-to-performance trade-of but also ensure that the computational resources of UAVs are utilized more efectively. This is particularly important given the limited processing power and battery life of most UAVs.
In summary, our research highlights the importance of considering altitude-specific parameter optimization in the design of UAV object detection systems. The use of dynamic network structures that can adapt to altitude variations presents a promising avenue for developing more flexible, eficient, and efective UAV vision systems.
This approach paves the way for advancements in UAV technology, enabling more accurate and eficient object detection, and ultimately enhancing the capabilities and applications of UAVs across various fields.
This work is supported by the European Union (Horizon 2020 Teaming, KIOS CoE, No. 739551) and from the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation, and Digital Policy.
Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.