Introduction

My PhD is focused on improving Predictive Process Monitoring (PPM) in several directions. The direction I am currently investigating is the improvement of PPM by exploiting explainable machine learning techniques.

Predictive Process Monitoring [1] is a research topic aiming at developing techniques that event logs extracted from information systems to predict how ongoing process executions will unfold up to their completion.

Explainable Machine Learning [2] is a research topic within the field of Explainable Artificial Intelligence (XAI), aiming at explaining why a given predictive model behaves in a certain way or in general how it will behave. Explainability techniques have been proven to be mature to be used, yet they have not been evaluated in-depth in the PPM scenario.

In this document, I focus on incorporating explainability techniques in PPM. In the PPM scenario we decided to identify three different actors to benefit from our work: (i) the business analyst; (ii) the machine; and (iii) the research scientist.

First, the document introduces the challenges of the work (Section 2), the approaches that have been/are going to be used for facing these challenges (Section 3), the results obtained so far (Section 4), and finally the position w.r.t. the State-of-the-art (Section 5).

Challenges

The three actors we identified have different understandings of the PPM scenario and different needs, this does not allow us to devise a solution that fits them all. To give the benefit of explainability techniques to the three aforementioned actors we focus on three challenges: (i) allowing business analysts to understand predictions and predictive models; (ii) allowing machines to enhance predictive models using the information gathered by the explainer as feedback; and (iii) allowing research scientists to leverage, adapt and build on top of explainability techniques for PPM.

The first challenge is using explainability techniques to allow business analysts to understand predictions and predictive models and to support them in other tasks, like decision making. The classical output of the explainability techniques have been tested in the general use-case of predictive model usage but not in the PPM scenario. What is missing is an investigation of whether the business analysts can be empowered with the same techniques. Empowering business analysts with the explanations for the predictive model could result in getting the trust of such users as well as in supporting them in making more informed decisions.

The second challenge is to allow machines to enhance the predictive model, by making use of the explanation as feedback. Since the explanation allows us to understand why the predictive model is behaving in a given way, we can use this explanation to characterise the wrong predictions. Understanding how the predictive model produces a wrong prediction would allow the machine to perform an automatic prevention, or at least correction, of the wrong predictions.

The third challenge is supporting research scientists with a representative set of explainability techniques for PPM that they can leverage and adapt to new problems. Up to now indeed, there is no code or API support for predictive model or prediction explanation out-of-the-box for PPM. Supporting the explanation of PPM models and data could be useful as a starting point for the research scientists who can build new solutions on top of them.

Addressing the Challenges

To address the aforementioned challenges we followed the methodology described in [3]. Each of the aforementioned challenges is addressed with the delivery of different artifacts and their respective validation. In particular, the used/planned approaches are: (i) the development/adoption of visual representation of the explainability techniques' output and a user evaluation to understand how to empower the business analysts with the explainability techniques' output, (ii) a technique to enhance predictive models through explanations, and (iii) the integration of a representative set of explainability techniques for PPM into a framework as easy-to-use APIs.

The first challenge is addressed with the development/adoption of several different visualisations of explanations and a user evaluation. In the user evaluation, we investigate whether explanations for PPM are understandable by business analysts. We then investigate whether business analysts are capable to exploit the information contained in the explanation to perform other tasks; e.g., decision making.

The second challenge is addressed developing and evaluating an algorithm that aims at understanding the reasons why a prediction is wrong and leveraging this information for improving the accuracy of the model. The algorithm mines explanation patterns from sets of traces, that aim at characterising the wrong predictions and use these patterns to lower down the relevance of the same patterns in the training set. To this aim, the returned predictions, each referring to an incomplete trace, are classified based on the value of the prediction and on the value for the ground truth. For instance, in the case of binary predictions, the prediction related to each trace is classified in terms of one of the confusion matrix quadrants, i.e., true positives, true negatives, false positives, and false negatives. Patterns of explanations can then be identified over each class of traces. The machine can then leverage patterns characterising the wrong predictions by acting upon the training set reducing their relevance. Finally, the predictive model can be re-trained with the new training set possibly presenting less misleading examples than those causing the initial predictive model to learn patterns determining wrong predictions.

The third challenge is addressed by providing a representative set of explainability techniques although most of the PPM solutions exploit common machine learning models so that explainability techniques can be easily applied to the PPM scenarios, the main point of attention required for adapting explainability techniques to the PPM scenario is related to how the PPM data are encoded/decoded, due to the particular type of data used in this domain.

Results

Tackling the three aforementioned challenges produced the following results.

Results for the first challenge. We do not have conclusive results for this challenge, we are carrying out the user-evaluation. To evaluate the quality of the explainability solutions for PPM we aim at evaluating: (i) if the user understands the explanations; and (ii) if the explanation can influence positively the work of the business analyst. To evaluate (i) we give the user several different explanations, and we gather its insights on what is understandable. To evaluate (ii) we ask the user to carry out several decision-making tasks with the explainability information we provide.

Results for the second challenge. This is the challenge for which we have the most tangible results. In the paper [4] we built a working solution that exploits feature-importance explanations to explain why a predictive model is wrong and eventually uses these feature-importance explanations to improve its accuracy. Given a trained predictive model, our solution computes the explanations for each of the traces of the validation set. Depending on the received predictions and on the actual labels, the traces are assigned to a quadrant of the confusion matrix. The traces contained in the quadrants of the correct and wrong predictions are then filtered keeping only the important features, using the feature-importance values received by the explainability technique. The filtered traces are then fed to a pattern miner, which retrieves the patterns characterising the correct and wrong predictions. We identify the occurrence of the patterns characterising the wrong predictions in the initial training set. The features identified by the patterns characterising the wrong predictions inside the identified set of traces are then shuffled to destroy the occurrence of these patterns, we then re-train the model and test it. This results in a significant improvement of the accuracy both on synthetic and real datasets. This solution currently works for the binary prediction problem and we are extending it to support the multiclass problem. The designed algorithm relies on Random Forest [5] and LIME [6], we identified the work in [7] as compatible with our approach and we are currently working with the authors on integrating our approaches to extend the significance of our respective contributions.

Results for the third challenge. To support the research scientists, we are continuing the development of Nirdizati [8]. We identified the works in (i) Partial Dependence Plot (PDP) [9], (ii) Individual Conditional Expectations (ICE) [10], (iii) Local Interpretable Model Explanations (LIME) [6], (iv) Shapley values for Explanation (SHAP) [11], (v) Skater [12], and (vi) Anchors [13] as suitable and representative of the state-of-the-art for explainability. We integrated them in Nirdizati, built the explanation module, and provided them as APIs.

Related Works

The body of previous work related to these challenges is the one that concerns: (i) PPM in general, (ii) the application of explainability techniques in PPM, and (iii) the development of PPM frameworks.

For what concerns the body of work related to PPM in general we can identify three streams of work: (i) outcome-oriented prediction [14,1,15,16], (ii) numeric/remaining time prediction [17,18,19], and (iii) next activity prediction [20,21,22,23,24]. These works differ from the work in [4] since they build the model once and for all and do not try to actively enhance the performance of the predictive model, or give any hint to the user on what moves the predictive model.

For what concerns the body of work related to the application of explainability techniques in PPM we identified [25,7,26,27,28]. We can divide them in those using a post-hoc explainability technique [25,7,26,27], and the one using an ante-hoc explainability technique [25]. All these works make next activity prediction, [25,7,26,27] with deep neural networks, whereas [28] with Bayesian Networks, and they all aim at providing the user with an understanding of what moved the predictive model towards a prediction. None of these works: (i) empirically evaluates the explanation delivered to the user, and (ii) actively enhance the accuracy of the predictive model.

For what concerns the body of work related to the development of PPM frameworks we identified, some plug-ins in ProM [29], for instance [30,18,1,31] and one plug-in in Apromore [32], which is [33]. They differ from our work on integrating explainability techniques in PPM frameworks because they do not support explainability techniques by design.