Introduction

The current rise of the Internet of Things (IoT) technology has produced a wide range of sensing solutions that are progressively being integrated into our daily life devices such as mobile phones or wearables [1]. The combination of such sensing capabilities with Artificial Intelligence (AI) is producing Artificial Intelligence of Things (AIoT) applications that provide an enhanced user experience [2].

Within the climate prediction field, a variety of efforts has been made to develop complex models to perform seasonal climate prediction [3,4], however, this process is inevitably subject to many error sources [4]. Climate forecasting has been usually divided into two different approaches: deterministic and probabilistic forecasting. Deterministic forecasting aims to provide a quantitative estimate of the future value of climate variables. Probabilistic forecast, on the other hand, aims to provide the probability distribution for the future state of the variables. Since a forecast of the occurrence probability for a given event can bring greater economic value than a single deterministic forecast with uncertain accuracy [5], these methods tend to be preferred.

Time series forecasting is the prediction of future data values based on some collected data [6] and has been an area of great interest in science, engineering, and business. Traditional time series forecasting is usually approached by the analysis of its internal structure: autocorrelation, trend, seasonality, etc., to capture the pattern of long-time behavior of the system [6]. These models predict future values of a target 𝑦 𝑖 (𝑡) for a given observed value 𝑖 at a time 𝑡. In climatology, these observed values usually represent measurements from individual weather stations [7]. While this applies to univariate forecasting, the same components can be extended to multivariate models without loss of generality [8].

Traditional time series forecasting methods tend to focus on parametric models informed by the expertise in a given domain, including autoregressive, exponential smoothing, or structural models [7]. However, in recent years machine learning (ML) methods have provided a means to capture dynamics of time series directly from the data, instead of domain expertise information [9]. The increasing data availability and computing power have positioned ML-based time series forecasting models as the next generation of time series forecasting models [7]. One of the main techniques in ML is artificial neural networks (ANNs), which have proven to be a reliable tool for time series analysis [10], and numerous ANNs design choices have emerged given the diversity of time series problems across multiple domains [7]. Of these designs, the most commonly used one in time series forecasting is recurrent neural networks (RNNs) [11]. This is due to the natural interpretation of time series data as sequences of inputs and targets [7].

Although ML approaches have demonstrated very good prediction performance, they have a significant limitation regarding its explainability. Neural network models are considered as "black boxes", because their internal processes are difficult to interpret with respect to the predictions they produce [12]. Solving this problem is a requirement to audit the reasoning behind incorrect prediction taken by AI systems and foresee the data patterns that may led to a climate event. Assuming that ANNs achieve better performance than white-box models, in this paper we propose a post-hoc sliding-window method that enables the explanation of an ANN forecasting model for the weather prediction domain. This way, instead of using a transparent or white box model with lower performance than an ANN, we enable the use of these black-box prediction models but augmented with explanation capabilities. The post-hoc explanation system follows an explanation-by-example approach implemented through Casebased Reasoning (CBR). Here, time series are split into different time-window cases that serve as explanation cases for the outcome of the ANN prediction model.

In this paper we focus on the combination of an environmental sensor for mobile devices, such as the BOSCH Sensortec BME6801 capable of sensing several weather variables [13,14], with the potential of AI to compute climate predictions using forecasting techniques. Our ultimate goal is to provide an AIoT solution that brings the user the possibility to obtain personalized predictions based on the historical environmental data collected through the sensors installed ICCBR'22 Workshop Proceedings in a wearable or mobile device. Moreover, the aim to provide explainable predictions that improves the user's acceptance of the predictions. This way, instead of receiving a generic climate prediction based on the nearby weather stations (that can be relatively far), this AIoT solution provides personalized weather predictions based on the concrete locations of the user, incorporating an explanation mechanism that lets the user understand the previous climate values that led the AI to make a concrete forecasting prediction.

Paper runs as follows. Section 2 presents the background of this work. Then, Section 3 describes our case-based explanation method. Section 4 presents the evaluation results and section 5 concludes the paper and opens lines of future work.

Background

In a recent work [15], scientists have used eXplainable Artificial Intelligence (XAI) to estimate precipitation from satellite images. The aim of this research was to exploit the high potential of ANNs used with satellite images to capture spatial information and inform numerical modeling and forecasting, which could help to mitigate weather risks. Using XAI techniques, insights gained from long-range planning (LRP) have given scientists confidence that the ANN generated predictions based on a physically reasonable strategy, and thus helped build more confidence in their predictions. Furthermore, if scientists want to improve the model by testing different model architectures, knowing how physically consistent the different model decision strategies are, provides a criterion for distinguishing between models [16]. This consequence of using XAI techniques goes well beyond the prediction performance obtained by using models such as ANNs by themselves without any explicability.

Some works have focused on conterfactual explanations, but not many have aimed at time series. Despite this, conterfactual explanations has become very popular, with more than 100 methods proposed, one of these is the native guide method [17]; this explainability method defines some key properties of good counterfactual explanations such as proximity, sparsity, plausibility, and diversity.

Like weather forecasting, with respect to climate prediction at subseasonal, seasonal, and decadal timescales, empirical statistical models exhibit limited predictive skill due to the complex and non-stationary nature of the relationship between large scale modes and regional climate. To address this issue, data-driven ML methods that leverage information from the entire globe have shown improvements in predictive skill [18]. A number of studies have highlighted the potential of ANNs in predicting climate across a range of scales because of their ability to capture nonlinear relations [19], while more recent studies have used XAI methods to explain these ANNs and their strategies to increase trust and generate new knowledge [20]. Mayer and Barnes [21] used XAI to show that ANNs can identify when favorable conditions that lead to enhanced predictive skill of regional climate are present in the atmosphere or not. In a decadal climate prediction application, Toms et al. [22] used simulated data from climate models to explore sources of decadal predictability in the climate system. The results showed that there are several regions where surface temperature is practically unpredictable, whereas there are also regions of high predictability. Such an analysis could motivate further mechanistic investigation to physically establish new climate teleconnections, illustrating how XAI methods can help advance climate science [16]. Further XAI applications have focused on analyzing the impacts of climate change on the energy consumption in building, explaining the underlying reasons behind the predictions [23].

Method

The main contribution of this paper is the use of CBR for the generation of explanations associated to the prediction of the ANN. CBR systems are claimed to have a "natural" transparency as they are based on the reuse of previous experiences or examples. Therefore, we propose a particular solution for the explanation of the outcomes of the ANN to the experts, where this opaque, black-box ML system is explained by a more interpretable, white-box CBR system, following the so-called twin-systems approach [24]. This approach is illustrated in Figure 1, where the dataset is used as the input of the ANN and to create the explanatory cases provided by the CBR system. Explanatory cases are generated from the weather series using a sliding-window method over the whole time series: 𝐶 𝑡 = ⟨[𝑡 − 𝑤𝑠, 𝑡], 𝑅 𝑡+1 ⟩ where 𝑤𝑠 is the window size and 𝑅 𝑡+1 is the solution of the case, which corresponds to the meteorological reading for the next day. Analogously, given a query time stamp 𝑡 𝑞 , the ANN will predict the time series values for that date: 𝑃 𝑟𝑒𝑑(𝑡 𝑞 ). Then, the query for our CBR system will be the time

window 𝑄 = [𝑡 𝑞 − 1 − 𝑤𝑠, 𝑡 𝑞 − 1].

Next, the prediction given by the ANN for 𝑡 𝑞 is explained by means of the most similar explanatory cases to the current time window 𝑄, following the explanation-by-example paradigm illustrated in Figure 1.

Dataset and model

The historical dataset of the meteorological variables was recorded by the Mexican National Water Council (Comisión Nacional del Agua, CONAGUA) ground station located at the city of Mérida; among the data provided, the following variables were found, simulating the values that can be also obtained by the BOSCH BME680 sensor: temperature (T), vapor pressure (P), and relative humidity (H). The range dates of the records were from January 1, 2000 to September 30, 2018, where there is a daily record of temperatures with minimum, maximum, and average units, resulting in nine readings provided for each day. A RNN using LSTM cells was trained over the dataset of time windows and weather labels. 70% of the dataset was used during training, while the remaining 30% was used for testing purposes. The dataset is then divided into time windows with 𝑤𝑠 = 14, obtaining explanation cases with 14 days of weather evolution.

Time windows correlation

Under the assumption that time windows with similar climate will present similar outcomes for the next days, we could reuse previous time windows to explain new ones in the future, without the use of an ANN. To do so, we need to define several similarity metrics for the retrieval.

Let us define the Combined Correlation Index (CCI), which provides a way to measure how a given time window case C is related to a target query window Q:

CCI(C, Q) = 1 4 (𝜌(C, Q) − 2‖(C, Q)‖ + 3),(1)

where 𝜌 is the function that calculates the Pearson correlation coefficient, and the double bars represent the normalized Euclidean distance between those vectors. In our particular case, we define CCI f as the CCI of a given meteorological feature f (temperature, vapor pressure, or humidity), C f represents a fixed window for comparison of different (sliding) windows Q f , vectors of 14 values registered daily for the same aforementioned meteorological feature. The normalized Euclidean distance is simply the Euclidean distance between all possible vectors (windows) in the time series, but normalized so it is defined in the range [0, 1]. CCI is defined in the range [0, 1], where higher values indicate a greater similarity between time windows, and viceversa. The correlation component deals with the morphological similarity of the time windows, while the Euclidean distance component deals with the proximity between the time series in the given time windows.

An enhanced CCI can be obtained using the original CCI values, so that:

CCI f + (C f , Q f ) = (CCI f (C f , Q f ) + ℎ) 2 ,(2)

where ℎ corresponds to an "enhancement value" which increases higher CCI values (those over 0.5), while decreasing lower ones. In our paper, since CCI is in the range [0, 1], ℎ = 0.5, and CCI f + is in the range [0. 25, 2.25]. With this in mind, we define the full combined correlation index, FCCI, between two windows C f and Q f as the sum of the CCI f + computed for every meteorological feature (humidity, temperature and vapor pressure: H, T and P, respectively). This is mathematically defined as follows:

(C f , Q f ) = CCI H + (C H , Q H ) + CCI T + (C T , Q T ) + CCI P + (C P , Q P ). (3)

This value can also be normalized, dividing each FCCI result between the highest FCCI encountered, therefore FCCI is defined in the range [0, 1].

Retrieval process

The retrieval process begins with the thresholding of the FCCI values to filter the windows with the greatest similarity results. To threshold the correlations results we define a minimum similarity value 𝑠 𝑚𝑖𝑛 which was empirically defined a value of 0.7. The values of FCCI for all the time series are shown in Figure 2, with the 𝑠 𝑚𝑖𝑛 threshold overlaid as an horizontal yellow line. The resulting data after the threshold was applied is shown in Figure 3. However, the calculated FCCI values yield undesirable high frequency components as shown in Figure 3. To solve this problem, a low pass filter can be applied over the FCCI time series to smooth the readings. In our case, we performed a filtering phase which consisted of the multiple application of a simple moving average filter (MAF). After iteratively applying this filter until the resulting signal no longer changes, we obtain a smoothed time series for the FFCI. Smoothed time series can be seen in Figure 4.

The first numerical derivative of this time series is also obtained, helping in the identification of peaks and valleys in the FFCI, which will be used to segment it into groups of ascending and decreasing curves to obtain the maximum and minimum values.

The highest FFCI values are used to identify the 𝑘 most similar explanation cases to the time-window query 𝑄. These are the cases that are presented to the user in order to explain the prediction given by the ANN model. An example showing the windows of humidity, pressure, and temperature, for the highest FFCI value with respect to the time-window query, is presented in the left-hand side of Figure 5. For visual comparison purposes, we also identified the window corresponding to the lowest FFCI value, and is shown on the right-hand side of this same figure. Notice how we also included the prediction of the ANN for the 15th day, as well as the real value of the 15th day, the following value of these two windows for each climate feature, respectively.

Evaluation

Even though a visual inspection of the plots of the windows shows an evident difference between the highest and the lowest FCCIs, respectively, we proposed a quantitative approach to measure this difference. When a window is retrieved from any of the absolute highest peaks (maximum or minimum) in the FFCI signal, we calculated the mean absolute error (MAE) between the ANN prediction for the 𝑡 𝑞 date and the actual values in the solution of the case, 𝑅 𝑡 , containing the readings of the day represented by the explanation case, particularly:

where pred(𝑡 𝑞 ) is a vector containing the outputs of the ANN for each climate variable 𝑉 represented by the time series TS or the case, concretely: {H, P, T}, representing humidity, pressure, and temperature, respectively. 𝑅 𝑡 is another vector containing the three actual values of a given time window for each climate feature obtained from the explanation case. We calculated the MAEs of the 𝑘 = 20 highest and lowest FFCI values, respectively, which are shown in Table 1.

To determine if the MAEs of the prediction of the ANN against the actual value for each window corresponding to the highest FCCIs are indeed statistically lower than those corresponding to the lowest FCCIs, i.e., if there is a high contrast of the central tendencies of both samples implying that they come from truly distinct populations, we proposed to use the Mann-Whitney U-test. Also called the Mann-Whitney-Wilcoxon, Wilcoxon rank-sum test or simply U-test [25], the Mann-Whitney U-test is appropriate for samples which do not follow a normal distribution. The result of the U-test is the z statistic, which represents a measure of how large the contrast between the central tendencies of two samples is [26]. Usually 𝑧 would show a significant difference to the 0.05 level if it had a value of 1.96 and any greater value corresponds to an even greater difference. The 𝑧 statistic resulting from comparing both the 20 highest and 20 lowest values of FCCI was 5.28, showing a clear difference between these two sets.

Conclusions

This paper proposes a RNN for weather forecasting of three climatological variables: humidity, vapor pressure, and temperature. Using 14-day windows extracted from the time series of these variables, the RNN forecasts the corresponding values for the 15th day. More importantly, we also propose a XAI CBR-based methodology to illustrate how in similar cases the RNN yields similar predictions to the actual values of the climatological variables. The similarity between two different time windows, a query and a fixed case, was calculated with a proposed index, the FFCI, also proposed in this work, which takes into account both 1) the Euclidean distance between the time series and 2) their morphological aspect through the Pearson correlation coefficient.

The 20 cases corresponding to the highest and lowest values of the FCCI, and their respective 15th value, were statistically compared using the Mann-Whitney U-Test, yielding a 𝑧 statistic of 5.28, well over the 1.96 value commonly associated with a 95% confidence. This means that the 14-day time period prior to a given date has a clear impact on the forecast of the climatological variables of the following day. As future work, an optimization method could be applied to determine the number of days which most affect the forecasting of future variables of climatological variables.

The main conclusion of this work is that XAI CBR-based techniques can be used to determine if there is a statistically significant impact of the past values of climatological variables when trying to forecast future ones. Furthermore, these methodologies could be used to determine the optimal number of previous days which affect the behavior of climatological variables in the future. This can eventually help us to better understand complex processes related to the dynamics of climatology.