=Paper=
{{Paper
|id=Vol-2670/MediaEval_19_paper_49
|storemode=property
|title=Deep
Learning Models for Estimation of Flood Severity Using Multimodal and Satellite Images
|pdfUrl=https://ceur-ws.org/Vol-2670/MediaEval_19_paper_49.pdf
|volume=Vol-2670
|authors=Hariny Ganapathy,Geetika Bandlamudi,Yamini
L,Bhuvana J,T.T. Mirnalinee
|dblpUrl=https://dblp.org/rec/conf/mediaeval/GanapathyBLJM19
}}
==Deep
Learning Models for Estimation of Flood Severity Using Multimodal and Satellite Images==
https://ceur-ws.org/Vol-2670/MediaEval_19_paper_49.pdf
Deep learning models for estimation of flood severity using
Satellite and News Article Images
Hariny Ganapathy, Geetika Bandlamudi, Yamini L,
Bhuvana J, T.T. Mirnalinee
SSN College of Engineering, Tamilnadu, India
(hariny,geetika,yamini)@cse.ssn.edu.in,(bhuvanaj,mirnalineett)@ssn.edu.in
ABSTRACT by measuring its reflected and emitted radiation at a dis-
This paper addresses the Multimedia Satellite Task at Media- tance from the targeted area, this provides an overview of
Eval 2019. We have focussed on the challenge of extracting modeling techniques used for forecasting the vulnerability to
information present in satellite images. Satellite images pro- flooding of an area [7]. Convolutional neural networks have
vide variety of information like weather, how any event on had shown great performance in various fields such as image
land unfolds and hence they play an important role in disaster classification, pattern recognition etc. Pre-trained networks
management. We present our approaches for three subtasks: like DeepSentiBank have also been used to detect floods
(1) Image-based News Topic Disambiguation, (2) Multimodal with the help of social multimedia and satellite imagery and
Flood Level Estimation from news, (3) Classification o f city- are proven to be giving good results to detect floods [1]. A
centered satellite sequences. All three tasks are related to study on detection of floods was carried out using pre-trained
classifying the images as flood r elated o r n ot. We h ave dis- model FCN-16 with 4-fold cross validation and produced
cussed about the performance of proposed CNN models and highly accurate classification [5]. More research is being car-
pre-trained models in the context of binary classification of ried out on other satellite images like TerraSAR-X images
images for flood r elated data. and a rule based classifier on this data produced a consid-
erably low accuracy and Multispectral imagery produced
a high accuracy for detection of floods [8]. An ensemble of
1 INTRODUCTION CNN models trained for retrieving flood relevant tweets had
Disaster Management is very important in order to help vic- their best model trained only on visual information and the
tims survive during natural or man-made disasters. Disasters reason stated was that nowadays people are likely to share
are unpredictable most of the times. But with the technologies photographs to address their current situation, rather than
evolving and increased understanding of those technologies, detailed textual descriptions [2].
the unpredictability of disasters can be reduced [3]. Flood is
one of such major disasters that can be both man-made as 3 APPROACH
well as natural. The general idea behind floods i s overflowing In pre-processing, the images are resized without interpola-
of water into land regions. Floods may be caused due to in- tion, to 256*256 to retain all the features when the images are
cessant rains or dam breakage and also pose serious damage fed to the model. The models are implemented using Keras
to humans, infrastructure and wildlife. It is also important framework and TensorFlow as back-end. The development
to detect floods i n a reas t hat a re n ot p hysically r eachable [2]. data was divided into 2 sets as training and validation. Based
This paper presents flood detection methods for Multi-modal on the validation accuracy, model parameters were altered to
and Satellite images. achieve better results. The tunnable model parameters used
Various classifiers s uch a s v anilla C onvolutional Neural are listed in Table 1. Binary image classifiers were built for
Network (CNN), pre-trained models like VGG19 are used to each task as explained below.
classify the data into flooded or not. The first task is to build
a binary classifier t hat c lassifies wh ether an im age is related Table 1: Tunnable Hyper-parameters of the models used
to flooding e vent o r n ot. T he s econd t ask i nvolves b uilding a
binary classifier that classifies an image into two classes, based Subtask Run No. of Epochs Learning Rate Optimizer
on whether the image has at least one person standing in
knee-length water. The third task is about utilizing Sentinel-2 1 100 0.001 Adam [6]
INTD
satellite images to classify whether the image has the area 2 100 0.1 Adam
that is flooded o r n ot [2]. MFLE 1 30 0.001 Adam
CCSS 1 100 0.1 Adam
2 RELATED WORK
Remote sensing has been proved to be an effective method
to detect and monitor the physical characteristics of an area 3.1 Image-based News Topic Disambiguation
(INTD):
Copyright 2019 for this paper by its authors. Use
permitted under Creative Commons License Attribution The development data had 2673 images with 82% images
4.0 International (CC BY 4.0). falling in the non-flooded category and 18% falling in the
MediaEval‘19, 27-29 October 2019, Sophia Antipolis, France
MediaEval‘19, 27-29 October 2019, Sophia Antipolis, France Hariny et al.
flooded category. A validation set was taken to be having 4 RESULTS AND ANALYSIS
1087 images with 50% of the images in the flooded category Task 1 has 1087 test samples, Task 2 with 1216 and Task 3
and 50% in the non-flooded category. has 60 [2]. The test results have been reported as their micro-
Run 1: In run 1, the proposed architecture has 6 convolu- averaged F1 score for Task1 and Task3 and macro-averaged
tional layers, 7 activation layers, 5 pooling layers and 1 fully F1 score for Task2 and have been reported in Table 2. Among
connected and dense layer along with 2 batch normalisation
layers. Cross-entropy was the loss function used and the final Table 2: F1 Scores obtained for 3 Multimedia Satellite Tasks
activation function as softmax with a dropout rate of 0.35.
We observed that few of the non-flooded images with lakes Micro Averaged Macro Averaged
and rivers were being classified as flooded. To overcome this Subtask Run F1 score F1 score
we modified our architecture by fine tuning a pre-trained
1 0.6312 -
model as Run2. INTD
2 0.8426 -
Run 2: A fine tuned VGG19 model, pre-trained on Imagenet
MFLE 1 - 0.4015
was used, whose top layer is removed in order to customize
CCSS 1 0.7206 -
this classifier for our dataset. Global Average Pooling was
used followed by a dropout of 20% [4] which was found to be
optimal. An additional fully connected layer is added with two runs submitted for Task 1, the second run produced a
softmax activation function that can predict the class labels better result. Task 2 produced a lower macro-average F1
as as flooded or non-flooded. score because of the model’s inability to differentiate between
an actual flooded region and a natural water body or a
3.2 Multimodal Flood Level Estimation from sand/mud coloured area, which led to a lot of non-flooded
images to be classified as flooded. There are flooding images
News (MFLE):
but only a subset of them belong to the class that we are
Run 1: The dataset for the second task included articles and interested in to (persons standing in water above knee level)
images related to flooding events. In this run, we have ana- and image-level descriptor/feature might not have been able
lyzed only the image modality. The model was trained using to learn this. Another reason that contributed to this low
4770 development set and 950 validation samples. Using only score was that only the images were taken into account and
the images, a CNN with 6 conv layers and softmax activation not the articles associated with them. Also the model was
function was implemented with K fold cross validation with poor at differentiating the flooded areas with muddy water
10 folds. The validation accuracy obtained was 66.67%. For and deserted region. The following observation was made
better results, we removed the K fold cross validation and during the validation phase - about 30% of the 950 sampled
modified the CNN to have relu activation function for all the validation images were muddy areas/desert regions and less
layers and softmax in the last layer. than 20% of those images were correctly classified. For Task
3, extracting the RGB bands from the satellite imagery lead
3.3 City-centered satellite sequences (CCSS): to producing good results. This illustrates the importance of
The dataset used for this task includes the satellite images of merging the bands of these satellite images.
sentinel-2 band. It contained 12 bands of each image and the
pre-processing was done to merge the bands 2,3,4 [10] that 5 CONCLUSION
represent the blue, green and red bands respectively to form In this paper, we presented our approach for the Multime-
an RGB image [3]. T he rasterio tool in python was used to dia Satellite Task at MediaEval 2019. We have proposed
merge the bands of the satellite images as shown in Figure 1. approaches to classify images as flood related or not, for all
The best prediction value is obtained by sliding over 16 pixels the 3 subtasks. These images include satellite images as well.
of the merged images. The converted and scaled images were As an extension, this work can be applied on active radar
then trained on a VGG19 [9], with dynamic input sizes. Since data (Synthetic Aperture Radar). We plan to use the results
the first input layer of VGG19 expects an input dimension of this work in the future for the monitoring and prediction
three, we only pass the RGB information of the satellite data of flooding events.
into the network.
6 DISCUSSION AND OUTLOOK
The results of the pre-trained model VGG19 are found to
be promising and adding more layers to the proposed CNN
model may enhance the performance, since additional layers
would extract more features. However the low performance
of the model used for the second subtask is owing to the
fact that only a part of the given data set i.e. images were
used but not the articles. Also, the future work can focus
Figure 1: Sample RGB Image after fusion of selective bands on differentiating muddy/desert regions from actual flooded
areas as the error percentage of this was high.
The Multimedia Satellite Task MediaEval‘19, 27-29 October 2019, Sophia Antipolis, France
REFERENCES
[1] Benjamin Bischke, Prakriti Bhardwaj, Aman Gautam, Patrick
Helber, Damian Borth, and Andreas Dengel. 2017. Detection
of Flooding Events in Social Multimedia and Satellite Imagery
using Deep Neural Networks. In MediaEval (Sept. 13-15, 2017).
Dublin, Ireland.
[2] Benjamin Bischke, Patrick Helber, Simon Brugman, Erkan
Basar, Zhengyu Zhao, and Martha Larson. The Multimedia
Satellite Task at MediaEval 2019 Flood Severity Estimation.
In Proc. of the MediaEval 2019 Workshop (Oct. 27-29, 2019).
Sophia Antipolis, France.
[3] Yu Feng, Sergiy Shebotnov, Claus Brenner, and Monika Sester.
2019. Ensembled convolutional neural network models for
retrieving flood relevant tweets. Image 10 (2019), 1.
[4] Yarin Gal, Jiri Hron, and Alex Kendall. 2017. Concrete
dropout. In Advances in Neural Information Processing Sys-
tems. 3581–3590.
[5] A Gebrehiwot, L Hashemi-Beni, G Thompson, P Kord-
jamshidi, and TE Langan. 2019. Deep Convolutional Neural
Network for Flood Extent Mapping Using Unmanned Aerial
Vehicles Data. Sensors (Basel, Switzerland) 19, 7 (2019).
[6] Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method
for Stochastic Optimization. In 3rd International Conference
on Learning Representations, ICLR 2015, San Diego, CA,
USA, May 7-9, 2015, Conference Track Proceedings.
[7] Victor Klemas. 2014. Remote sensing of floods and flood-
prone areas: an overview. Journal of Coastal Research 31, 4
(2014), 1005–1013.
[8] Biswajeet Pradhan, Mahyat Shafapour Tehrany, and
Mustafa Neamah Jebur. 2016. A new semiautomated de-
tection mapping of flood extent from TerraSAR-X satellite
image using rule-based classification and taguchi optimization
techniques. IEEE Transactions on Geoscience and Remote
Sensing 54, 7 (2016), 4331–4342.
[9] K. Simonyan and A. Zisserman. 2014. Very Deep Convolu-
tional Networks for Large-Scale Image Recognition. CoRR
abs/1409.1556 (2014).
[10] Qunming Wang, Wenzhong Shi, Zhongbin Li, and Peter M
Atkinson. 2016. Fusion of Sentinel-2 images. Remote sensing
of environment 187 (2016), 241–252.