=Paper=
{{Paper
|id=Vol-2590/short1
|storemode=property
|title=Algorithm for search and recognition of fire by means of unmanned aerial vehicles
|pdfUrl=https://ceur-ws.org/Vol-2590/short1.pdf
|volume=Vol-2590
|authors=Alexander Mezhenin,Vera Izvozchikova,Vladimir Shardakov,Veronika Zaporozhko,Vladimir Borisov
|dblpUrl=https://dblp.org/rec/conf/micsecs/MezheninISZB19
}}
==Algorithm for search and recognition of fire by means of unmanned aerial vehicles==
https://ceur-ws.org/Vol-2590/short1.pdf
Algorithm for search and recognition of fire by
means of unmanned aerial vehicles
Alexander Mezhenin1[0000−0002−7150−9811] , Vera
2[0000−0002−8707−9510]
Izvozchikova , Vladimir Shardakov2[0000−0001−6151−6236] ,
2[0000−0002−2193−9389]
Veronika Zaporozhko , and Vladimir
Borisov3[0000−0002−6312−4620]
1
ITMO University, Kronverksky Ave. 49, St. Petersburg, Russia
mejenin@mail.ru
2
Orenburg State University, Ave. Pobedy 13, Orenburg, Russia
werovulv@inbox.ru
3
Orenburg branch of the Russian State University of Oil and Gas named after I. M.
Gubkin, st, of Young Leninists 20, 460047, Orenburg, Russia
borisov-vps@mail.ru
Abstract. The paper considers the possibility of providing early detec-
tion of fires in oil and gas wells through the use of the RGB palette,
as well as intelligent decision-making to eliminate the consequences of
fire using the possibility of remote dynamic monitoring based on robotic
platforms. The result was a simulated prototype of a hardware-software
complex (APC), which allows monitoring of the territory based on the
distributed processing of large stream of graphical information in real
time (including the exact coordinates of the fire, its size, direction of
fire, proximity to settlements). For the most effective detection and pre-
vention of growth of natural and man-made fires. The proposed agro-
industrial complex will be implemented on the basis of modern technolo-
gies: robotics, GPS-technologies, GIS-technologies, client-server Internet
technologies, video surveillance, intelligent information technologies. Ac-
curate automated determination of coordinates with the help of GPS-fire
signal makes it possible to timely proceed to the localization and elimi-
nation of the fire, thereby preventing and reducing the negative impact
on people, nature, wildlife, reducing the damage caused by the fire. The
created software and hardware complex will allow to quickly develop and
make the most optimal decisions on the direction of fire brigades and fire
equipment to the places of fire, especially in particularly remote areas.
Keywords: monitoring of territories · robotic platforms · detection of
natural and man-made fires · unmanned aerial vehicles · fire coordinates.
Copyright c 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
2 A. Mezhenin et al.
1 Introduction
Fires at gas and oil facilities are a disaster that causes incalculable material,
moral and environmental losses.
Analysis of fire statistics shows that every fourth fire is accompanied by the
explosion and the subsequent development of the combustion in the area of 5000
m2 . If a fire occurs without an explosion, the fire area in most cases is 500 m2 ,
and the maximum area reaches 3000 m2 , so its temporary detection of natural
and man-made fires is an important factor in ensuring their rapid extinguishing.
Currently, this function is performed by drones and unmanned aerial vehicles.
In recent years they have become increasingly popular in the detection of fires
in forests and fields. They are especially necessary in hard-to-reach places where
the definition of fire without the use of aviation is impossible. This task will
effectively manage the unmanned aerial vehicle of the type ”copter”. The main
tasks of using quadrocopters in emergency situations are to determine the lo-
calization of areas affected by natural disasters, conduct cartographic surveys,
Search for missing people.
Centralized control of the unmanned aerial vehicle is carried out by a human
operator (one to one). The structure of the control system:
- human operator;
- remote terminal;
- aircraft with on-Board intelligent system of information processing.
The entire process of controlling the aircraft is carried out by the operator
through the terminal, the on-Board information processing system displays the
image on the operator’s screen, and he, based on his experience, makes the
appropriate decision.
The paper proposes to develop a prototype of an automated hardware and
software complex, which in addition to performing the function of dynamic mon-
itoring of the territory in real time provides a continuous analysis of the data
coming from the sensors, and gives the operator an operational emergency alert
when the parameters deviate from the specified norm, predicts possible emer-
gencies, is able to calculate and form the optimal route to the source of the fire,
i.e. is an expert intelligent system.
The advantages of using a quadcopter for the monitoring of gas and oil wells:
- ability to continuously monitor even the most remote and wild areas, in-
cluding in any windy weather;
- objectivity of the obtained data (absence of human factor-fatigue, falling
asleep during visual observation of fires from specialized towers);
- reducing the cost of one hour flight;
- speed of data collection;
- accuracy of data collection;
- optimization of frames.
Development of software and hardware complex 3
2 Related Work
The analysis of modern applications and publications in the field of fire moni-
toring in Russia and abroad has shown that the use of unmanned aerial vehicles
(UAVs) for these purposes is in demand. However, this trend has not yet become
widespread due to imperfect implementation approaches and the complexity of
processing large amounts of data, as well as the following disadvantages:
- not all systems have flexible integration to adapt the system to different
subject areas;
- high technical requirements for equipment;
- have a closed program code.
The relevance of this scientific problem is confirmed by theoretical and prac-
tical works of foreign and Russian researchers. The General direction studying
the problems of data mining is defined as ”Monitoring Wildfire UAVs”.
The authors, led by Sharon Rabinovich [1], proposed a coordination strategy
with a new methodology for estimating the periphery of a spreading fire using
limited observations. The developed information-logical model of the information
system displays the data of the subject area in the form of a set of information
objects and links between them. A group of researchers from the Ivanovo fire
and rescue Academy of the Ministry of emergency situations of Russia under
the leadership of V. A. Smirnov considered a set of positive effects from the
introduction of unmanned aerial vehicles into the monitoring system of forest
fire situation, and also showed all the advantages that are possible when using
unmanned aerial vehicles [2].
Scientists from America [3] have developed methods of planning the trajec-
tory of unmanned air transport to monitor the boundaries of fires. The disad-
vantage of the work is that the algorithm was not tested by the authors, but
only an imitation of fire was proposed.
The researchers of the University of Toronto [4] proposed a method for
monitoring the occurrence based on the Kalman filter with the use of unmanned
aerial vehicles. The authors conducted a fire simulation on the basis of which a
method for assessing the behavior of forest fire propagation and the contour of
the fire front was developed.
Scientists Zhongjie Lin and Hugh H. T. Liu proposed a model for distributed
topology-based optimization for joint monitoring of fires through the use of mul-
tiple UAVs. The problem of interaction of several UAVs in a distributed network
is formulated. A method is proposed that allows cooperation between UAVs at
the group level [5].
The actual research is presented by the authors under the leadership of A. S.
Vasilyev [6]. In their article, the scientists considered the key issues arising in
the development of software and hardware complex of detection and monitoring
of fires on the basis of an unmanned aerial vehicle by combining images, and
also presented the architecture of the proposed software.
In work [7] the model of the simulator on the basis of CAVE-technology
is presented. The disadvantage of this model is the fact that the movement
of the unmanned aerial vehicle must be monitored by the operator and place
4 A. Mezhenin et al.
marks in the system where the fire is detected and describe the procedure for its
suppression.
Researchers Connie Phan and Hugh H. T. Liu describe a technique for con-
trolling UAVs through mobile devices. The authors propose a methodology for
the interaction of several aircraft through the use of a hierarchical platform [8].
Scientists from China presented the work on fire identification based on deep
learning. The authors in their study took into account unmanned aerial vehicles
equipped with global positioning systems (GPS), through which it is possible
to provide direct georeferencing images, mapping the area with high resolution.
The paper presents a 15-level self-learning architecture DCNN called ”F ireN et”.
The disadvantage of such a system is the speed of processing a large amount of
data [9].
In his scientific work V. Vipin proposed a method of classification of fire
pixels using rule-based models in the RGB and YCbCr color space [10].
Researchers K. Angayarkkani and N. Radhakrishnan developed fuzzy rules
for the use Of YCbCr color space for segmentation and fire image detection [11].
Scientist C. Yuan developed a set of algorithms for tracking fires, including
median filtering, color space transformation [12]. These works are based on
methods of image processing using elements of manual work, the results of which
are highly dependent on the accuracy of the manually selected parameters.
Statistical and machine learning methods are not actively used in this di-
rection. The Gaussian mixture model (GMM) is used for flame detection [13].
However, using an empirical value for the number of mixtures may not lead
to better results. The SVM classifier is used in [14]. It should be noted that
when widely used feature descriptors such as the oriented gradient histogram
(HOG) with space invariant feature transformation (SIFT) are used with these
classifiers, the false alarm rate is not low enough.
Satellite imagery is a common method of fire detection [15], but the long
scanning period and low flexibility make fire detection difficult. Infrared thermo-
graphic cameras are used to obtain thermal images of the terrain [16]. Through
them it is possible to obtain reliable data on the distribution of heat for fire de-
tection. However, most infrared thermographic imaging systems operate in the
wavelength range of 0,75 to 100 microns. They find much less information about
the environment on this strip. This information can also be very important, es-
pecially when flammable and combustible materials are presented. In addition,
according to Nyquist’s theorem, the recorded thermal image has a lower spa-
tial resolution than the visible spectrum cameras. In addition, thermographic
systems are quite expensive with high maintenance costs.
3 Experimental Part
To get the coordinates of the quadcopter you need to have the coordinates of the
UAV, the initial values of which are obtained by GPS. Next, we have a pixel-
by-pixel shift of the x and y values. the shift Equation is shown in formulas (1)
Development of software and hardware complex 5
and (2):
x − Camera
X= , (1)
2
−(y − Camera)
Y = , (2)
2
where x and y are the coordinates of the transmitted images, the X and
Y coordinates of the received image, Camera image produced on the camera
quadracopter.
To detect the source of fire, we use color analysis based on the RGB model.
This model describes each color with a set of the following colors: red, green and
blue.
We describe the possibility of flame pixel detection for RGB color model by
the following system:
R(x, y) > Rred
1
PK
Rred = K · i=1 R(x, y)) (3)
R(x, y) > G(x, y) > B(x, y)
where values Rx,y , Bx,y , Gx,y – value of red, blue and green colors in pixel on
coordinates x, y, K is the total number of pixels, Rred is the average intensity
of the red color.
r, g and b are normalized components of RGB space defined by the following
formulas:
r = R/(R + G + B), (4)
g = G/(R + G + B), (5)
b = B/(R + G + B), (6)
The algorithm for determining the ignition source is shown in fig. 1.
According to the presented algorithm, the quadcopter transmits the image
to a personal computer by scanning the marked GPS point. By using the RGB
palette, the color that is closest to the ignition source (red) is determined. If the
system detects a fire, it automatically notifies the operator, who decides to take
the necessary measures.
The results of the fire detection simulation are shown in fig. 2.
Thus, the process of modeling the ignition of oil wells is carried out and the
algorithm for early fire detection is described. The developed methods and ap-
proaches for analyzing oil well ignition are planned to be used in forecasting and
intellectual verification of upcoming changes in the heterogeneous infrastructure
of the complex model, based on spatial and temporal distribution and proactive
forecasting of the use of computing resources.
6 A. Mezhenin et al.
Fig. 1. Algorithm for determining the source of fire
Fig. 2. Simulation results of the detection of a fire
Development of software and hardware complex 7
4 Conclusions
The paper proposes an algorithm for color processing of RGB palette, which
allows to control the territory on the basis of distributed processing of a large flow
of graphic images in automatic real-time mode (including accurate determination
of the coordinates of the fire, its area, the direction of the flame, the proximity
of settlements, etc.).
The socio-economic effect of the implementation of the proposed intelligent
automated software and hardware complex is as follows:
- remote access via the Internet to an interactive map of operational moni-
toring, which allows you to obtain information about the state of the controlled
territory;
- the use of agriculture will significantly reduce the cost of special services to
monitor fires, through the use of drones;
- collection and analysis of operational data on the state of the fire situation.
References
1. Rabinovich, S., Renwick, E., Gabriel H.: Toward Dynamic Monitoring and Suppress-
ing Uncertainty in Wildfire by Multiple Unmanned Air Vehicle System. Journal of
Robotics, 2018, p. 12 (2018).
2. Smirnov, V., Surovegin, A.: Application of unmanned aerial vehicles for monitoring
forest fires. In: X International Scientific-Practical conference “EurasiaScience”, pp.
59–60, Moscow (2017)
3. Skeele, R., Hollinger, G.: Aerial Vehicle Path Planning for Monitoring Wildfire Fron-
tiers. Field and Service Robotics, pp. 455–467 (2016)
4. Lin, Zhongjie, Liu, Hugh, Wotton, Mike: Kalman Filter-Based Large-Scale Wildfire
Monitoring With a System of UAVs. IEEE Transactions on Industrial Electronics,
66(1), pp. 606–615 (2019)
5. Lin, Zhongjie, Liu, Hugh: Topology based distributed optimization for multi-UAV
cooperative wildfire monitoring. Optim Control Appl Meth, 39, pp. 1530–1548
(2018)
6. Vasiliev, A., Krasnyashchikh, A., Korotaev, V., Lashmanov, O., Lysenko, D., Ne-
narokomov, O., Shirokov, A., Yaryshev, S.: Development of software and hardware
complex for forest fire detection by combining images. Proceedings of higher educa-
tional institutions. Instrumentation, 55(12), pp. 50–55 (2012)
7. Kuznetsov, A., Musalimov, V., Saenko, A., Trambitsky, K.: Application of image
analysis algorithms for detection of fires. Proceedings of higher educational institu-
tions. Instrumentation, 55(6), pp. 51–56 (2012)
8. Connie, Phan, Liu, Hugh: A cooperative UAV/UGV platform for wildfire detection
and fighting. In: 7th International Conference on System Simulation and Scientific
Computing, p.21, Beijing, China (2008)
9. Zhao, Yi, Ma, Jiale, Li, Xiaohui, Zhang, Jie: Saliency Detection and Deep Learning-
Based Wildfire Identification in UAV Imagery. Sensors, 18(3), p. 19, 2018.
10. Vipin, V.: Image processing based forest fire detection. Int. J. Emerg. Technol.
Adv. Eng., 2, pp. 87-–95 (2012)
11. Angayarkkani, K., Radhakrishnan, N.: An Intelligent System for Effective Forest
Fire Detection Using Spatial Data. Int. J. Comput. Sci. Inf. Secur., 7, pp. 202-–208
(2010)
8 A. Mezhenin et al.
12. Yuan, C., Liu, Z., Zhang, Y.: UAV-based forest fire detection and tracking using
image processing techniques. In: Proceedings of the International Conference on
Unmanned Aircraft Systems IEEE, pp. 639-–643, Denver, CO, USA (2015)
13. Töreyin, B., Dedeoglu, Y., Güdükbay, U., Cetin, A.E.: Computer vision based
method for real-time fire and flame detection. Pattern Recognit. Lett, 27, pp. 49-
–58 (2006)
14. Byoung, C., Kwang-Ho, C., Jae-Yeal, N.: Fire detection based on vision sensor and
support vector machines. Fire Saf. J., 44, pp. 322-–329 (2009)
15. Li, Z., Nadon, S., Cihlar, J.: Satellite detection of Canadian boreal forest fires:
Development and application of the algorithm. Int. J. Remote Sens, 21, pp. 3057—
3069 (2000)
16. Katayama, H., Naitoh, M., Suganuma, M., Harada, M., Okamura, Y., Tange, Y.:
Development of the Compact InfraRed Camera (CIRC) for wildfire detection. SPIE
Opt. Eng., 8, pp.745806-–745814 (2009)