Timely and accurate flash flood predictions are crucial for early warning of this costly and deadly natural hazard. There are many modeling approaches tackling hourly runoff formation mechanisms and utilizing different levels of computational complexity and requirements to an input data. However, to keep an efficient trade-off between speed, data availability, robustness and interpretability, the focus of operational runoff forecasting systems development often shifts to conceptual hydrological models. However, the developing field of deep learning provides us an opportunity to advance flash flood predictions using new data-driven technologies. In the present study, we extensively search for optimal structure and parameters and examine the predictive performance of the Long Short-Term Memory (LSTM) model for continuous runoff predictions at an hourly time step and compare results with the process-based hydrological model. Results highlight that the LSTM model provides reliable hourly runoff predictions. Yet, it demonstrates only comparable efficiency with a conceptual hydrological model, but with a significant computational overhead.
output dependencies. Recent studies show a high potential of LSTM models in hydrological modeling: it was shown
that LSTM models can outperform both conventional feed-forward ANNs and conceptual hydrological models [
The rapid growth of studies that involve modern deep learning architectures in the field of hydrology is ongoing
right now, and many prospective studies are emerging [
We use the Rimbaud River basin at Collobrières as the study site. It is located in the Maures highlands of southeastern
France, close to the Mediterranean Sea coast and drains an area of 1.4 km². For a detailed description of the basin
characteristics, such as elevation, vegetation, geology, climatology, and a hydrologic regime, please see the Supplement
material of Thirel et al. [
Hourly precipitation data (P) was obtained from three different sources: two nearby rain gauges located outside the
basin and the SAFRAN reanalysis [
In the present study we use GR4H – the process-based, lumped, conceptual hydrological model, which was
developed for runoff predictions at an hourly temporal resolution [
In the present study, the chosen LSTM model is the ANN constructed from two types of calculation layers –
LSTM and fully-connected (Dense) – to map meteorological forcing (input layer) to runoff (output layer) data (Fig.
1). For a detailed description of the whole workflow of implementing LSTM networks for runoff predictions, we
recommend referring to Kratzert et al. [
Objective guidelines for designing an appropriate structure of ANN and its hyperparameters for a specific modeling
task remain elusive [
We constrained the hyperparameter space by considering only six important LSTM model hyperparameters, investigated separately for two variants of forcing data (precipitation and potential evapotranspiration; precipitation only) (see Fig. 2): 1. The input history length (hours), which is the length of the input sequence of meteorological forcing to be considered as a predictor for runoff. We use 25 candidates: from 48 to 1200 hours with a step of 48 hours. 2. The number of LSTM layers, which goes from one to three consecutive layers; 3. The number of LSTM units (# neurons), which is the length of hidden/cell states of LSTM layers. We use candidate values of 16, 32, and 64. 4. The regular dropout rate of the LSTM layer. We use two options: 0 and 0.2. 5. The recurrent dropout rate of the LSTM layer. We use two options: 0 and 0.5. 6. The number of Dense layers. We use the following candidates: one layer with one hidden neuron; two consecutive layers with 32 neutrons on the first and with one neuron on the second layer; three consecutive layers with 64, 32, and one neuron, correspondingly.
For every candidate configuration, we trained the LSTM model and evaluate its corresponding performance on two independent periods: training and validation. For the validation period, we randomly reserve 4 years of available data (~12%), and the remaining data of 30 years has been used for the training period (88%). We used the early stopping technique to interrupt the training while there is no improvement for model performance on the validation period for 5 epochs (1 epoch is the time when each training sample has been entirely used for model optimization), and then save only the best model variant in terms of efficiency regarding the validation period. For model training, we use the RMSprop optimization algorithm with default hyperparameters and the mean squared error as a loss function. 2.4
For the benchmark setting, we slightly adapted the standard split-sample test methodology proposed by Klemeš [
The main goal of the proposed benchmark setting is to try to reach at some extent genericity for choosing a hydrological model of any type: for a fair comparison, we have to estimate GR4H and LSTM model performances under the same testing conditions.
Interestingly, deep data-driven models might not be necessarily required. Results of the implemented extensive grid
search procedure (Fig. 2) show that the development of a reliable LSTM model for hourly runoff predictions does not
require deep architectures that utilize many sequential LSTM and Dense layers. In the context of runoff predictions, it
is in our case a better strategy to build a shallow model with one, but wide, LSTM layer (with a high number of neurons)
with consequent one and narrow (with the one neuron) Dense layer, and the high rate of recurrent regularization.
Obtained results are consistent with the earlier study by Zhang et al. [
Results reveal clear differences in efficiency patterns between GR4H and LSTMs (Fig. 4). For calibration and testing
periods (P1, P2, FP) GR4H has a more uniform spread of efficiency metrics disregarding which data was used for
model calibration and validation: NS ranges between 0.69 and 0.77, KGE ranges between 0.71 and 0.84 and Bias ranges
between 0 and 14 percent. For LSTM_PPE, these intervals are 0.62-0.84 (NS), 0.65-0.9 (KGE) and 1.5-12 (Bias). For
LSTM_P, these intervals are 0.55-0.8, 0.61-0.87 and 2.6-22, correspondingly. In contrast to GR4H, identified patterns
in LSTMs efficiencies highlight the strong natural dependency of data-driven models efficiency and robustness to the
training data (as also observed in [
On the independent validation period (VP, 2002-2004) all models show a substantial drop of efficiency disregarding which data was used for model calibration/training. Both LSTM models trained on a full calibration period (FP, 19682001) outperform the GR4H model: LSTMs have a higher NS (0.61 and 0.58 for LSTMs in comparison with 0.52 for GR4H) and a lower Bias (1.2 and 7.8 for LSTMs in comparison with 14 for GR4H). However, an additional use of the KGE shows a reverse situation where GR4H outperforms both LSTMs (KGE is 0.73 for GR4H in comparison with 0.65 and 0.67 for LSTMs). Further decomposition of KGE into separate components that describe the contribution of correlation, variance, and bias clearly shows that the reason for this behavior is the higher contribution of variance component for both LSTMs. Visual inspection of simulated hydrographs shows that LSTMs do not capture small runoff peaks.
Results show that a split-sample test is a helpful tool for model efficiency and robustness analysis, but in case of
implementing rainfall-runoff models for operational use in real setting it is better to use all available data for model
calibration/training disregarding results of split-sample test: all models show better results when calibrated/trained on
a full period. Thus, our findings also confirm the same conclusion which was provided in [
Using confined forcing (only precipitation time series) to drive LSTM models shows a good potential in the light of further integration of LSTM-based model into operational setting driven by the only precipitation data obtained from weather radars. Trained on a full period, the LSTM_P model shows comparable results with the LSTM_PPE model: the former has a lower NS and higher Bias, but also a lower variance. The lower variance of LSTM_P model shows its best performance for small runoff peaks (where runoff is under 1 mm/hour). This behavior proves higher and faster response of LSTM_P model to precipitation signal in comparison with the more inert LSTM_PPE model, which does additionally utilize potential evapotranspiration signal. 3.3
Results do not provide a clear answer to that question. We highlight six events where the peak runoff is above 2 mm/hour for testing the models’ efficiency for flash floods predictions on the validation period (Fig. 5). For three of them (Fig. 5, events B, C, D) LSTM_PPE shows better results than GR4H, for two events there is a tie (Fig. 5, events A, E), and only for one event (Fig. 5, event F) GR4H is better. Results of LSTM_P model is worse: it is better for runoff predictions than GR4H for one event only (Fig. 5, event B), there is almost no difference between models for two events (Fig. 5, events D, F), and GR4H outperforms LSTM_P for the remaining three events (Fig. 5, events A, C, E).
Flash floods predictions are not in the particular focus of this study, which strives to show the performance of
different types of models in general. However, the LSTM_PPE model shows a great potential for flash floods
predictions with results that are at least comparable with the process-based GR4H model. Unfortunately, for bigger
flash floods (where the peak runoff is above 2 mm/hour), LSTM_P shows limited reliability. For the events with
duration longer than one hour it is critical to include potential evapotranspiration time series in LSTM model forcing,
which indirectly represents inert basin state. This finding contrasts with Toth and Brath results [
One hydrological and two LSTM models were extensively evaluated for runoff predictions at an hourly temporal
resolution on the small basin in France with the Mediterranean climate using an updated split-sample test and multiple
efficiency criteria. The main findings of the present work can be summarized as the following:
1. Deep learning belongs to the groundbreaking technologies that are revolutionizing many fields of scientific
research by setting up the new level of predictability [
A review of previous research studies in the field of data-driven hydrology leaves us feeling that we as a community
are walking in a circle. Despite extensive guidelines how to organize research in a sustainable way and push it further
[
The present study was mainly funded by the Russian Foundation for Basic Research (RFBR), project no. 19-0500087 A. The hydrological model development and evaluation part (Sect. 2.2, 3.2) was supported by the Russian Science Foundation, project no. 16-17-10039. Georgy Ayzel was financially supported by Geo.X, the Research Network for Geosciences in Berlin and Potsdam.
Georgy Ayzel would like to thank Météo-France for making available the meteorological SAFRAN data and Dr. Nathalie Folton (Irstea Aix-en-Provence) and Dr. Guillaume Thirel (Irstea) for providing the meteorological and discharge gauge data. Georgy Ayzel thanks Dr. Maik Heistermann and Dr. Guillaume Thirel for their helpful recommendations and positive criticism of this study.
Computational experiments have been carried out using GPU computation resources of the Machine Learning Group of the University of Potsdam (Potsdam, Germany) and the Shared Facility Center “Data Center of FEB RAS” (Khabarovsk, Russia).