Continuous technological progress has led to great changes in our society. UAVs (Unnamed Aerial Vehicles), commonly known as drones, are one of the most significant technological advancements of the last decade. UAVs offer a wide variety of new possibilities and have become a tool that is used on an everyday basis. UAVs can be used in fire control due to their ability to manoeuvre rapidly and their wide range of operation. This article presents a review of the main uses of UAVs in combatting fire. Special emphasis is placed on fire detection techniques using computer and infrared computer techniques, as well as the hardware systems that drones must incorporate to perform this task. This article also presents a simple proposal for fire detection and alert using UAVs and computer vision.
has become the main way to detect and monitor fires. UAVs and drones are a low-cost option for monitoring, detecting and even fighting forest fires.
UAVs are not new; they have been in existence for dozens of years, however only recently have they become
popular. With the development of technologies, modern UAVs have also become more advanced and our
possibilities for using them have become greater, which poses new legal and regulatory questions [
UAVs have been widely used for forestry, agriculture and livestock. For example, it has been used to obtain
scans of large areas of the livestock system [
In [
The objective of this paper is to review the state of the art about the use of UAVs in fire detection in large forested areas. Specifically, there will be a review of aircraft fire detection techniques.
The rest of the article is structured as follows: Section 2 describes diverse state of the art proposals in the area of UAVs and fire detection and vision computer technologies. Section 3 provides a full description of the system proposed in this work, including the functionality of each of its components. Section 4 details the case study and discusses the results obtained from this work. Finally, Section 5 outlines the conclusions drawn from this research. 2
This section studies the main techniques to be incorporated into a UAV for the detection and communication of fire alerts to the people responsible for the forest area. This will allow them to take the corresponding actions. 2.1
One of the main problems detected in the application of computer vision techniques is that most of these
techniques employ classifiers that must be trained. These classifiers need a large number of images of forest fires
for their correct classification, such as Eigenfaces, Fisherfaces, LBP [
The previous section began with a study of the existing remote-control technologies in each of the three parts into which the proposal can be divided: aircraft, communication and control. The initial objective was to offer a complete system that would improve these existing technologies in the field of fire detection.
As far as the aircraft is concerned, there are two distinct parts that have an influence when it comes to flying: its assembly or chassis and its electronics. The analysis of these parts leads to the conclusion that the improvements that can be made in the available resources are minimal if not nil for the purposes of this project. This is because there are large international companies that have years of experience and invest millions of euros in the development of both chassis and electronic stabilization systems, as well as open source projects conducted by independent developers collaborating on do-it-yourself (do it yourself) platforms whose systems are much less stable than the above-mentioned international companies, so competing against them would make no sense.
On the other hand, the communication and control blocks are very similar in all the existing projects, they do not reflect the great advances made in the flight system. All of them use radio station systems for communication which are very stable and offer a long communication range but do not allow data transmission in digital form, so communication is limited to flight orders using the previously explained PPM transmission. This type of transmission requires the presence of a second communication module for telemetry transfer and a third communication system for the transmission of video in real time.
As for the control system used to control the UAV systems, they all use radio stations and are very sophisticated what makes them easy to integrate with PPM radio systems, which can even adapt telemetry reception modules and view them on a digital display. From our analysis we propose to design a system capable of simulating the connection of radio systems to control any UAV stabilizer (if possible the best on the market) and capable of transmitting telemetry digitally together with flight orders and video using only Wi-Fi connection and remotely controlled from the ground with a gamepad connected to a computer instead of using radio stations, displaying telemetry information and video transmitted from the multirotor on the computer screen. With this Wi-Fi communication system which is easily accessible to consumers, has high-powered access points which allow for connections of up to 50km and its cost is much lower than long-range radio systems. Although it does not exceed the distance of the aforementioned radio links, but given the limited flight time of the UAV due to the battery life (common element for all systems), which today barely exceeds 30 minutes, this distance range is more than sufficient for the system to be developed.
In addition, the possible control carried out with a computer is not comparable today to that carried out by a radio station, so designing a control software capable of controlling and displaying the telemetry transmitted by the UAV through a computer would substantially improve the leveraged systems. In addition, it could be seamlessly integrated with the above-mentioned Wi-Fi communication system, which would receive the data measured by the sensors and the video and, finally, could transmit the flight orders from the control software to the UAV. The hardware required to control any of the existing stabilizers on the market must be capable of transmitting the information to the stabilizer through a series of outputs (at least 8) as if it were a radio receiver.
Radio receivers use manufacturer-specific settings to order the data to be transmitted, but maintain a standard for the connection, using three-pin connectors for each of the channels, through which the data associated with each is sent. The only exception is the case of the manufacturer Futaba, which uses a proprietary protocol (S-BUS) with which a single three-pin connector is able to send information from all radio channels. However, all controllers are compatible with the above-mentioned standard, while only a few are compatible with Futaba’s S-BUS protocol.
We chose a Raspberry PI 3 microcontroller as hardware for performing this task, it is capable of running a Linux operating system and it allows to connect the APM 2.5 through one of its USB ports and connects a Wi-Fi adapter to the other port. All these features made us opt for this hardware. Therefore, the controller consists of a Raspberry PI 3 and an APM 2.5 connected via the USB port.
This hardware has to interact with at least the stabilizers of the multirotor. There are many multi-rotor stabilizers on the market, so the DJI Wookong-M stabilizer has been chosen. It provides very precise stabilization and control. 3.1.1
The LiPo battery communicates directly with the ammeter of the APM 2.5, which performs consumption measurements. The ammeter data cable is connected to the APM 2.5 itself (the result of the measurement is sent) and the continued power supply is connected to the PMU of the DJI Wookong-M, to the multi-turner drives and to the UBEC voltage regulator. PMU connects to the other components of the DJI Wookong-M in the manner specified by the manufacturer, DJI Innovations, as shown in Figure 2.
The drives will connect their data cable to the DJI Wookong-M’s MC inputs in the order specified by DJI Innovations and its banana connectors to the corresponding motor connectors regardless of the order (a test without propellers is required to check the direction of rotation before the first flight). The UBEC will be connected to the Raspberry PI model B GPIO pins (pin 2 for power and pin 6 for ground) and power. It will also do this with the flight camera and the high-resolution camera stabilizer. The Raspberry PI connects through one of its USB 2.0 ports with the USB Wi-Fi antenna to establish communication with the access point and with the other USB port it connects to the APM 2.5 microUSB to exchange data (flight orders in one direction and sensor information in the other). The flight IP camera will be connected to the Ethernet port of the Raspberry PI. The APM 2.5 will connect each of its first 6 outputs (marked as outputs) to the DJI Wookong-M’s MC with 3-pin connectors so that the corresponding signal will be sent to each of the DJI’s channels through each of the outputs. If a camera stabilizer is available, it will connect to outputs 7 (pitch movement) and 8 (roll movement) of the APM 2.5, Figure 3.
With the present connection and the type of each link between components shown above, the wiring diagram of the electronic components that control and monitor the multirotor is as shown in Figure 4. The main role of the base station’s control software is to offer the necessary functionality to perform remotely controlled flights manually thanks to the use of the previously mentioned gamepad or to perform autonomous control through the application of an autonomous flight algorithm designed for this project. This algorithm follows a series of points introduced by the software with the help of the UAV status received through telemetry. The control software offers a configurator that allows you to set the parameters needed to perform a new flight or to schedule a future flight without having to connect the UAV. In addition, it is possible to export the configuration into files so that they can be used as flight profiles, selectable according to the type of flight to be performed or the specific configuration of the multirotor, as well as other parameters such as the software language.
The route programming mode is designed so that whoever programs the routes that will later be travelled by the UAV, will be able to load these routes on the UAV at the time of flight and will be able to repeat the same routes without having to plan them again, thus this mode helps save time. Scheduled routes are exported in files with extension .route and can be loaded from the main configuration or even with the software started in flight mode, from where the route can also be configured without the need to have previously programmed it in route programming mode. Another of the features of the control base station software is that it will be executed from the places where the flights are performed, which will be in most cases areas where there is no Internet connection (or at least not at an acceptable speed), so you cannot depend on the connection during the flight to show the information regarding the area where the flight is performed, so the map of the area must be previously obtained and geo-positioned. The image can be obtained from anywhere as long as it is perpendicular to the terrain, this allows for photographs taken by the UAV to be used, but the coordinates of at least three of the four corners must be known in order to assign a coordinate (longitude and latitude) to each of the pixels in the image. This is called image geolocation, which can be done through the control software if an image is included for the first time without associated geolocation data or using the developed software to facilitate the acquisition of terrain images using Google Maps, accessible online and free of charge, currently published on http://servidor-online.com/hawk-geoposition/ (a) View of the route for sale.
(b) View of the collected video
The system used for the geopositioning stores in the metadata of the image the information of the coordinates of the corners in a transparent way to the user without the user even realizing it, since no external files are used, but rather, as metadata, they are included in the image. The metadata format is formed with the coordinates of each corner, in the following format: GEO − Inf ormation, LatitudeA : LongitudeA, LatitudeB : LongitudeB, LatitudeC : LongitudeC : LongitudeC , LatitudeD : LongitudeD, 0.00000000Example : GEO − Inf ormation, 40.967755287228385 : −5.62173769696952938, 40.96775528287228385 : −5.628174998588699, 40.96410968463514 : −5.621737696952938, 40.96410968463514 : −5.628174998588699, 0.0000000000 Another of the main features of the software is the control of the UAV in two ways: manual (with a pilot on the ground via the gamepad) and autonomous or automatic (without a pilot, performing a calculation of the movement automatically and based on the introduction of a series of route points). An algorithm has been developed to detect fires using the Python script language. The OpenCV and Numpy APIs have been used for this purpose. The video taken by the UAV camera is processed frame by frame. The colour frame of each frame is changed to HSV. The HSV color space is very similar to the way we humans perceive the images of the environment, more so than even the RGB space. A mask with the upper and lower color values is then defined. A mask is applied to the frame and only the colors in the range we have defined are visible. In figure 6, we can see how the developed algorithm that is deployed in the Raspberry PI of the UAV works. This paper has made an overview of the main computer vision techniques for fire detection in forested areas. Unlike ground or space systems, the cost of deploying these techniques in UAVs is low and no humans are put at risk when performing this activity.
Since the density of trees in woodland areas impedes the detection of small fires through simple monitoring. The combination of sensorization and artificial vision techniques in a UAV is the best choice for fire detection in forests.
One of the drawbacks encountered in carrying out this work is the possibility that the smoke may block the images of the fire, although before the smoke becomes dense, the fire should have already been detected. Sunlight, for example, can cause false positives, so the combination of infrared images and the proposed algorithm on the collected video can contribute to robust detection of forest fires, including high probability of detection, low false alarm rates and improved adaptive capabilities in various environmental conditions.
The use of infrared images, which are easier to analyze, often causes hot gases to also be detected as fire. Since these areas are similar to those of fire, they can produce a difference between the fire areas that appear in the visible domain and those in the infrared domain.
This work was developed as part of “MOVIURBAN: M´aquina social para la gesti´on sostenible de ciudades inteligentes: movilidad urbana, datos abiertos, sensores m´oviles”. ID: SA070U 16. Project co-financed with Junta Castilla y Le´on, Consejer´ıa de Educaci´on and ERDF funds. [21] Forest health, natural resources, fire, trees, wildfire, silviculture photos. http : //www.f orestryimages.org/.
(Accessed: 10 March 2018). [22] J. Ramiro Mart´ınez-de Dios, Luis Merino, An´ıbal Ollero. Fire detection using autonomous aerial vehicles with infrared and visual cameras. Proceedings of the 16th IFAC World Congress 2005