This paper presents CBR-foX (Case-Based Reasoning for forcasting eXplanations): a post-hoc slidingwindow method that enables the explanation of forecasting models. It applies the Case-Based Reasoning paradigm to provide explanations-by-example, where time series are split into diferent time-window cases that serve as explanation cases for the outcome of the prediction model. It has been designed for domain-expert users -without ML skills- that need to understand and how (future) predictions could be dependent of past time series windows. The main novelty of this approach is its reusability, as CBR-foX can be applied to any black-box forecasting model based on time-series. We propose a novel similarity function which deals with both the morphological similarity and the absolute proximity between the time series, together with several reuse strategies to generate the explanation cases. We propose an automatic evaluation approach based on computing the error (MAE) between the model prediction for and the actual values in the solution of the explanatory case. Then we apply this evaluation method to demonstrate the performance of the proposal on the given dataset. Finally, we provide a reusable implementation that can be directly applied to other time-series forecasting models and domains.
The goal of this development is to provide a reusable CBR method for the generation of post-hoc
explanations for a given black-box forecasting model. CBR systems are claimed to have a
“natural” transparency as they are based on the reuse of previous experiences or examples.
Therefore, we propose a particular solution for the explanation of the outcomes of the forecasting
model to the experts, where an opaque, black-box ML system is explained by a more interpretable,
white-box CBR system, following the so-called twin-systems approach [
The approach being presented is mainly addressed to the domain expert -without ML skillsthat needs to understand the outcomes a black-box forecasting model. This way, the CBR-foX method provides several explanation cases that illustrate how in similar past cases (time-series windows) the forecasting model yields similar predictions to the current values.
Under the CBR assumption that time windows with similar values will present similar outcomes for the following time stamps, we could reuse previous time windows as cases that explain new ones in the future, without the use of the forecasting model. To do so, we define a novel several similarity metric for the retrieval.
Let us define the Combined Correlation Index (CCI), which provides a way to measure how a given time window case C is related to a target query window Q:
CCI(C, Q) = 1 ( (C, Q) − 2‖(C, Q)‖ + 3), (1)
4 where is the function that calculates the Pearson correlation coeficient, and the double bars represent the normalized Euclidean distance between those vectors. The correlation component deals with the morphological similarity of the time windows, while the Euclidean distance component deals with the proximity between the time series in the given time windows. Remaining constants are used to normalize the range of the equation.
Next, we define the full combined correlation index, FCCI, between two windows Cf and Qf as the sum of the CCIf computed for every time-series feature. However, the calculated FCCI values yield undesirable high frequency similarity values as shown in Figure 2 (top). To solve this problem, a low pass filter can be applied over the FCCI time series to smooth the readings. In our case, we performed a filtering phase which consisted of the multiple application of a simple moving average filter (MAF). After iteratively applying this filter until the resulting signal no longer changes, we obtain a smoothed time series for the FFCI. Smoothed time series can be seen in Figure 2.
The highest FFCI values are used to retreive the most similar explanation cases to the time-window query . Next, we provide several reuse strategies to combine the time-series of these nearest-neighbours: average, max, min, etc. Then, our method can be configured present each original k-nn or/and the combined time-series obtained during the cases reuse. These are the explanation cases that are presented to the user in order to explain the prediction given by the forecasting model. An example is presented in the left-hand side of Figure 3 using the 1-NN. For visual evaluation purposes, we also show the less similar case on the right-hand side of this same figure.
Even though a visual inspection of the plots of the windows shows an evident diference between the highest and the lowest FCCIs, respectively, we proposed an automated quantitative approach to measure this diference. When a window is retrieved from any of the absolute highest peaks (maximum or minimum) in the FFCI signal, we calculated the mean absolute error (MAE) between the forecasting model prediction for the time-stamp and the actual values in the solution of the case, , containing the readings of the time-stamp represented by the explanation case, particularly:
|TS| ∈{TS} ∑︁ |pred()[ ] −
R[ ]|, (2) where pred() is a vector containing the outputs of the forecasting model for each variable represented by the time series TS. is another vector containing the actual values of a given time window for each time-series feature obtained from the explanation case. We calculated the MAEs of the top highest and lowest FFCI values, respectively, for the given dataset, which are shown in Table 1.
We propose a novel post-hoc explanation system that follows an explanation-by-example approach implemented through Case-based Reasoning (CBR). The CBR-foX explanation method splits time series into diferent time-window cases that serve as explanation cases for the outcome of the forecasting model. It has been designed for domain-expert users -without ML skills- that need to understand and how (future) predictions could be dependent of past time series windows.
This explanation method is completely reusable and proposes a novel similarity function 1 # Load required external libraries 2 loadImports() 3 # Load dataset 4 data = loadData() 5 # Load (or train) forecasting model 6 model = loadForecastingModel() 7 # Configuration parameters (with default values) 8 config = configExplanationParameters() 9 # Main explanation method 10 explain(data, model, config) named Combined Correlation Index (CCI), which deals with both the morphological similarity and the absolute proximity between the time series. We also provide several reuse approaches (max, min, average, etc.) that can be configured to generate an explanation cases from retrieved k-NNs.
The main novelty of this approach is its reusability, as CBR-foX can be applied to any black-box forecasting model based on time-series. The source code allows to configure all the required parameters such as input variables or time-window length. It has been designed to support its reusability and integration into explanation libraries or APIs. As illustrated in Listing 4, it isolates the domain dependent data and forecasting model. Then it provides a configuration method with default values. And, finally, the explanation process itself, encapsulated as an only executable method.
Github (full source code): Collab (online execution):
Jupyter Notebook (pdf):
Full execution video