Weather Forecasts Through an LSTM-CBR Twin System

Abstract

In this paper, we explore two methods for explaining LSTM-based temperature forecasts using previous 14 day progressions of humidity and pressure. First, we propose and evaluate an LSTM-CBR twin system that generates nearest-neighbors that can be visualised as explanations. Second, we use feature attributions from Integrated Gradients to generate textual explanations that summarise the key progressions in the past 14 days that led to the predicted value.

-

LSTM-CBR

1. Introduction

Inaccurate weather forecasting can have a serious impact to life, property and businesses. For instance, farmers rely on weather forecasting to plan activities such as ploughing, harvesting and cultivation [ 1 ]. In 2018, the inability of weather forecasts to detect an early monsoon indirectly trapped 12 Thai children and their football coach in a cave for 18 days — and caused the death of two of their rescuers 1. Eficiently predicting weather can help to minimise these losses in the future [ 2 ]. Howbeit, weather is stochastic and its patterns are non-linear, making it a challenge to obtain precise predictions. The problem lends itself well to neural networks for their ability to model non-linearity. Particularly, weather forecasting suits sequential methods such as the Long Short-Term Memory (LSTM) [ 3 ], to allow for the utilisation of time-series data in the prediction process [ 2 ]. By making use of the recurrent nature of the LSTM, information can be fed back through diferent steps along a time-series, capturing trends, seasonality and more, which all help increase predictive power.

ICCBR XCBR’22: 4th Workshop on XCBR: Case-based Reasoning for the Explanation of Intelligent Systems at ICCBR-2022, CEUR Workshop Proceedings

Although weather forecasts are common in everyday life, we seldom seek for deeper insights to explain how a prediction was made. On the contrary — if we did ask for further explanation — we could pass better judgement on the validity of predictions. Explainable AI (XAI) has become increasingly prevalent in recent times. This is because despite deep learning methods now excelling in performance in a variety of domains, they often remain as black-boxes; we are unable to ‘look inside’ to understand why a prediction was made. Methods such as Integrated Gradients [ 4 ] have been useful for producing saliency maps in images to highlight the important pixels that weigh heavily on the outcome of a classification. They have also been applied in a time-series setting but are often unsophisticated and can be dificult to interpret — presenting a need for dedicated time-series solutions [ 5, 6 ]. Literature shows that some (albeit few) attempts have been made to explain weather forecasts in a time-series setting such as by score-maximisation and occlusion analysis visualisations [ 7 ], or applying LIME [ 8 ]. However, none of these methods propose the use of case-based reasoning (CBR) to provide explanations. Using an approach grounded in similarity, such as CBR, a posteriori knowledge can be drawn upon to help both prediction and explanation. Past experiences can be drawn upon to make arguments for why a decision was made. We propose an approach that takes the learned LSTM embeddings from the prediction stage and uses these to build a case-base to provide explanations that can be visualised. We will also use Integrated Gradients and NLG to provide further explanation by means of a report.

2. Methodology

2.1. Predicting Temperature with LSTM

We use the default prediction model provided for the challenge which consists of two bidirectional LSTM layers.

2.2. Building the Twin LSTM-CBR System

We use the Clood CBR 2 system as the twin. Clood CBR is a distributed CBR system which supports high scalability. The system has a micro-service architecture which splits an application into a set of smaller and interconnected services that scale to meet varying demands [ 9 ]. The output embeddings from the final LSTM layer are extracted for each input row and are fed into the Clood CBR system to form the case base. For a given row and LSTM prediction that we want to explain, we query the Clood system using the nearest-neighbors approach. This gives us the top three similar rows based on the embedding similarity. To establish the quality of the twin, we compare our approach to a baseline twin system that uses the raw features instead of the embeddings.

We use Mean Absolute Error (MAE) as the evaluation metric to compare the two twin systems as it is one of the common measures of forecasting error in time-series analysis. MAE measures the absolute diference | − | between a prediction and an actual observation where the individual diferences share the equal weight. The 3 nearest neighbours for a prediction are retrieved from the case-base and the mean MAE is calculated between their predicted values. Comparing the mean gives an indication of whether the cases are better aligned through the use of embeddings — or not.

=

∑=1 | − | (1)

2.3. Generating Explanations

A template-based Natural Language Generation approach to generate explanations was taken. Based on Integrated Gradients, we extracted feature importance attributions for each day over three diferent time frames. These were as follows: the entire two-week period; the final week; and the final three days. The prediction, feature attributions and general information from the dataset were then slotted into the template to produce a weather forecast report (as seen in Figure 1). Furthermore, to aid interpretability, we add visualisations (emojis, graphics and colour) to our textual explanation to indicate the weather prediction.

3. Results

3.1. LSTM-CBR Twin Evaluation

The results in Table 1 prove that the twin LSTM-CBR system based on embeddings performs better than a twin modelled around raw features. The embedded approach achieved an MAE of 0.066 degrees whereas the raw features approach achieved an MAE of 0.133 degrees. This means that using embeddings, the twin system is able to find better nearest-neighbours, therefore can produce better explanations.

3.2. Nearest Neighbors Explanations Using LSTM-CBR Twin

In Figures 2 and 3, we visualize the progression of the average humidity and pressure, respectively, over the previous 14 days for a sample query row, top 3 nearest neighbors and 3 randomly selected rows. As can be seen, the query and the nearest neighbors have a similar progression of the predictor variables as well as close predictions for the subsequent day temperature.

Embedded Features

0.066

Raw Features

0.133

4. Conclusion

In this paper we explore two methods for explaining LSTM predictions of weather forecast. We ifrst propose an LSTM-CBR twin system that explains a prediction using nearest-neighbors and then visualises the explanations through parallel coordinate plots. We show that using the LSTM embeddings can result in better twinning compared to raw features. As a second explanation method, we make use of feature attributions from Integrated Gradients in text templates to generate text explanations. We note that a current limitation of the twin system is that it considers all days as equally important when retrieving the nearest-neighbors. Therefore, future work could explore diferent weighting strategies (such as weights derived from feature attributions) for strengthening the twin system. A second limitation lies with the need to provide a template for the NLG-based explanation, which would difer between domains. Further work could explore the application of more advanced NLG techniques that allow the inference of templates in a new domain.

Acknowledgments

We would like to thank Professor Nirmalie Wiratunga for motivating us to take part in this challenge and for her advise throughout the competition.

References

[1] P. Hewage , A. Behera , M. Trovati , E. Pereira, M. Ghahremani , F. Palmieri , Y. Liu, Temporal convolutional neural (tcn) network for an efective weather forecasting using time-series data from the local weather station , Soft Computing 24 ( 2020 ) 16453 - 16482 .

[2] D. N. Fente , D. Kumar Singh , Weather forecasting using artificial neural network , in: 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT) , 2018 , pp. 1757 - 1761 . doi: 10 .1109/ICICCT. 2018 . 8473167 .

[3] S. Hochreiter , J. Schmidhuber , Long short-term memory , Neural computation 9 ( 1997 ) 1735 - 1780 .

[4] M. Sundararajan , A. Taly , Q. Yan , Axiomatic attribution for deep networks , CoRR abs/1703 .01365 ( 2017 ). URL: http://arxiv.org/abs/1703.01365. arXiv: 1703 . 01365 .

[5] T. Rojat , R. Puget , D. Filliat , J. Del Ser , R. Gelin , N. Díaz-Rodríguez , Explainable artificial intelligence (xai) on timeseries data: A survey , arXiv preprint arXiv:2104.00950 ( 2021 ).

[6] U. Schlegel , H. Arnout , M. El-Assady , D. Oelke , D. A. Keim , Towards a rigorous evaluation of xai methods on time series , in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), 2019 , pp. 4197 - 4201 . doi: 10 .1109/ICCVW. 2019 . 00516 .

[7] I. A. Abdellaoui , S. Mehrkanoon , Deep multi-stations weather forecasting: explainable recurrent convolutional neural networks , 2020 . URL: https://arxiv.org/abs/ 2009 .11239. doi: 10 .48550/ARXIV. 2009 . 11239 .

[8] M. Zdravković , I. Ćirić, M. Ignjatović , Explainable heat demand forecasting for the novel control strategies of district heating systems , Annual Reviews in Control ( 2022 ).

[9] I. Nkisi-Orji , N. Wiratunga , C. Palihawadana , J. A. Recio-García , D. Corsar , Clood cbr: Towards microservices oriented case-based reasoning , in: I. Watson, R. Weber (Eds.), CaseBased Reasoning Research and Development , Springer International Publishing, Cham, 2020 , pp. 129 - 143 .