Explainable Prescriptive Process Analytics (Extended Abstract)

Abstract

-Within the realm of Process Mining, Process-Aware Recommender systems (PAR systems) are information systems that aim to monitor process executions, predict their future behaviour, and finding optimal corrective actions to reduce the risk of failure or to maximize a given reference Key Performance Indicator (KPI). While a PAR system is composed by monitoring, predictive analytics and prescriptive analytics, the focus has been heavily on the first two, and very little attention has been given to the last. Therefore, this PhD project firstly aims to develop a technique that is able to provide good evidence-based recommendations, rather then relying on subjective opinions. A second goal of the PhD project is to also incorporate techniques for Explainable AI inside PAR systems, in order to provide and understand the root causes that put forward certain recommendations; otherwise, the process' stakeholders and actors will unlikely trust and, hence, use them. Index Terms-Prescriptive Business Process Analytics, Processaware Recommender systems, Predictive models, Shapley Values, Explainable AI

I. RESEARCH PROBLEM AND MOTIVATION

Process-Aware Recommender systems are instances of a class of systems to monitor and predict how process instances are going to evolve, and to recommend the corrective actions to recover the instances with higher risk not to achieve the expected outcome. Conceptually, a PAR system is constituted by three main blocks: Monitoring, Predictive analytics and Prescriptive analytics. In the last years, a lot of research has been on the first two (commonly referred as Predictive Business Process Monitoring techniques) and several approaches have been proposed (see e.g. [ 10 ], [ 22 ]). Conversely, the last block is overlooked, assuming that the users, after being alerted of a potential failure, are able to find the proper corrective actions. However, it has been demonstrated by some on-the-field experts [ 5 ] to be not true. This is due to the fact that, without support, process actors make decisions on the basis of their subjective opinion, rather then relying on objective data, which comes from the event logs and record the past executions and the achieved outcome.

The first goal of this PhD project is therefore summarized by the following research question:

Research Question 01 How can we build a prescriptive business process analytics block that effectively maximizes a given reference KPI (Key Performance Indicator)?

In this PhD project, the outcome is measured through a customizable KPI function that, given an execution recorded in a log trace (which describes the life-cycle of a particular process instance, i.e. a case), looks at the activities executed and attributes values and returns a KPI value.

Explainable AI is another field that has been overlooked in the last years, assuming that a good level of accuracy is sufficient for the process’ stakeholders to trust the recommender system (as well as the prediction system). However, the process actors need to be convinced that the recommended actions are the most suitable ones to maximize the KPI of interest; otherwise they will not follow the suggestions given. This leads us to the second goal of this PhD project:

Research Question 02 How can users trust recommendations provided by a PAR system?

Finally, research results will be developed as software modules, integrated in the process-mining suite of myInvenio, and evaluated with users expert of the process’ domain, in order to assess the general validity of the framework and the usability of the tool from a user experience point of view.

II. LITERATURE ANALYSIS TABLE I: Analysis of related works wrt. relevant characteristics of PAR systems

Work RGeecnoemrimcKenPdIation RCeocnotmexmt-aenwdaarteion Generalizable iUnnddeeprelnydinegnttechnique Conforti et al. [ 4 ] + / - + + Maggi et al. [ 9 ] + / - + / - + + / Schobel et al. [ 16 ] + / - + / - - + Schonenberg et al. [ 17 ] + - - + Weinzierl et al. [ 23 ] - + / - + +

The analysis focuses on the questions mentioned above.

Table I shows the analysis of several works (one for each row) related to the first research question, wrt. relevant PAR systems’ characteristics (illustrated in the columns). The first is the possibility to recommend actions not only for improving a specific KPI (e.g. reducing the remaining time), but also for a generic, user-customizable KPI. The second is the ability to recommend actions using all the attributes of the events, while the third is the possibility to generalize recommendations also for unseen data. Finally, the last column shows if the developed recommender system is loosely coupled to the implementation of other PAR system’s components. In each row, a symbol + is shown if the developed recommender system tackles that particular problem, otherwise a symbol - is shown. In this PhD project the first goal is to develop a PAR system able to tackle all the problems described above.

The second goal deals instead with the problem of equipping a PAR system with explanations of the recommendations Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). given. Few approaches exist in the literature to explain machine learning models, arisen from the need to understand complex black-box algorithms like ensembles of Decision Trees and Deep Learning [ 2 ], [ 7 ], [ 8 ], [ 11 ], [ 14 ], [ 19 ]–[ 21 ].

Little research work has been conducted on explaining the outcome of process predictive monitoring. The most relevant work is by Rehse et al. [ 13 ], which also aims at providing a dashboard to process participants with predictions and their explanation. However, the paper does not provide sufficient details on the actual usage of the explainable-AI literature, and the very preliminary evaluation is based on one single artificial process that consists of a sequence of five activities.

Breuker et al. also try to tackle the problem [ 3 ], but their attempt is not independent of the actual technique employed for predictions. Furthermore, their explanations are only based on activity names, while explanations can generally involve resources, time, and more.

III. PROJECT ROADMAP

The construction of the Prescriptive Analytics module requires first the development of a predictive module. It will rely on simulating the possible customer-journey continuations and recommending those that will likely lead to higher satisfaction, according to a predictive model, which is learnt from the data recorded in the event log.

We did an assessment of the state of the art of predictive algorithms [ 10 ], [ 22 ], which showed that LSTM has proven to be among the most effective AI techniques for predictive monitoring. We built an implementation based on the work done by Navarin et al. [ 12 ] and extended it to predict a generic KPI of interest, instead of predicting only the remaining time. Afterwards, we built an explainable prediction framework based on SHAP, which allowed determining for each feature of the predictive vector how much it contributes to the prediction. This solution was chosen for several reasons. The SHAP implementation of the Shapley values for Deep Learning has the strong theoretical foundation of the original game theory approach, with the advantage of providing offline explanations that are consistent with the online explanations. Moreover, SHAP avoids the problems in consistency seen in other explanatory approaches (e.g. the lack of robustness seen in the online surrogate models, as analysed in [ 1 ]). Furthermore, it is independent of the machine- or deep-learning technique that is employed to make the predictions.1. A complete description about the implementation of the framework is described in [ 6 ]. 1An alternative could have been attention-based models, but a limitation is linked to the lack of consensus that attention weights are always correlated to feature importance [ 15 ], [ 18 ]

References

[1] Alvarez-Melis , D. , Jaakkola , T.S. : On the robustness of interpretability methods . arXiv preprint arXiv: 1806 . 08049 ( 2018 )

[2] Bahdanau , D. , Cho , K. , Bengio , Y.: Neural machine translation by jointly learning to align and translate . In: The 3rd International Conference on Learning Representations, ICLR 2015 , San Diego, CA, USA, May 7- 9 , 2015 , Conference Track Proceedings ( 2015 )

[3] Breuker , D. , Delfmann , P. , Matzner , M. , Becker , J.: Designing and evaluating an interpretable predictive modeling technique for business processes . In: Business Process Management Workshops . pp. 541 - 553 . Springer ( 2015 )

[4] Conforti , R., de Leoni, M. , Rosa , M.L., van der Aalst , W.M.P. , ter Hofstede , A.H.M.: A recommendation system for predicting risks across multiple business process instances . Decis. Support Syst . 69 , 1 - 19 ( 2015 )

[5] Dees , M. , de Leoni , M., van der Aalst , W.M.P. , Reijers , H.A. : What if process predictions are not followed by good recommendations? In: Proceedings of the Industry Forum at BPM 2019. CEUR Workshop Proceedings , vol. 2428 , pp. 61 - 72 . CEUR-WS.org ( 2019 )

[6] Galanti , R. , Coma-Puig , B. , de Leoni , M. , Carmona , J. , Navarin , N.: Explainable predictive process monitoring . arXiv: 2008 . 01807 ( 2020 )

[7] Lundberg , S.M. , Lee , S.I.: A unified approach to interpreting model predictions . In: Advances in neural information processing systems . pp. 4765 - 4774 ( 2017 )

[8] Lundberg , S.M. , Nair , B. , Vavilala , M.S. , Horibe , M. , Eisses , M.J. , Adams , T. , Liston , D.E. , Low , D.K.W. , Newman , S.F. , Kim , J. , et al.: Explainable machine-learning predictions for the prevention of hypoxaemia during surgery . Nature biomedical engineering 2 ( 10 ), 749 ( 2018 )

[9] Maggi , F.M. , Di Francescomarino , C. , Dumas , M. , Ghidini , C. : Predictive monitoring of business processes . In: Advanced Information Systems Engineering . pp. 457 - 472 . Springer ( 2014 )

[10] Ma´ rquez- Chamorro , A.E. , Resinas , M. , Ruiz-Corte´s, A.: Predictive monitoring of business processes: A survey . IEEE Transaction on Services Computing 11 ( 6 ), 962 - 977 ( 2018 )

[11] Meacham , S. , Isaac , G. , Nauck , D. , Virginas , B. : Towards Explainable AI : Design and Development for Explanation of Machine Learning Predictions for a Patient Readmittance Medical Application , pp. 939 - 955 (06 2019 )

[12] Navarin , N. , Vincenzi , B. , Polato , M. , Sperduti , A. : LSTM networks for data-aware remaining time prediction of business process instances . In: Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI 2017 ) ( 2017 )

[13] Rehse , J.R. , Mehdiyev , N. , Fettke , P. : Towards explainable process predictions for industry 4.0 in the dfki-smart-lego-factory . KI - Ku¨nstliche Intelligenz 33 ( 2 ), 181 - 187 ( Jun 2019 )

[14] Ribeiro , M.T. , Singh , S. , Guestrin , C. : “why should I trust you”: Explaining the predictions of any classifier . In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , San Francisco. pp. 1135 - 1144 ( 2016 )

[15] Sarthak , J. , Wallace , B.C. : Attention is not explanation . In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . pp. 3543 - 3556 . Association for Computational Linguistics ( 2019 )

[16] Schobel , J. , Reichert , M. : A Predictive Approach Enabling Process Execution Recommendations , pp. 155 - 170 . Springer International Publishing, Cham ( 2017 )

[17] Schonenberg , H. , Weber , B. , van Dongen , B. , van der Aalst , W.: Supporting flexible processes through recommendations based on history . In: Dumas, M. , Reichert , M. , Shan , M.C. (eds.) Business Process Management. pp. 51 - 66 . Springer Berlin Heidelberg ( 2008 )

[18] Serrano , S. , Smith , N.A. : Is attention interpretable? In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . pp. 2931 - 2951 . Association for Computational Linguistics, Florence, Italy ( 2019 )

[19] Shrikumar , A. , Greenside , P. , Kundaje , A. : Learning important features through propagating activation differences . In: Proceedings of the 34th International Conference on Machine Learning-Volume 70 . pp. 3145 - 3153 . JMLR. org ( 2017 )

[20] Shu , K. , Cui , L. , Wang , S. , Lee , D. , Liu, H.: Defend: Explainable fake news detection . In: International Conference on Knowledge Discovery & Data Mining, SIGKDD . pp. 395 - 405 . ACM ( 2019 )

[21] Sundararajan , M. , Taly , A. , Yan , Q. : Axiomatic attribution for deep networks . In: Proceedings of the 34th International Conference on Machine Learning-Volume 70 . pp. 3319 - 3328 . JMLR. org ( 2017 )

[22] Teinemaa , I. , Dumas , M. , Rosa , M.L. , Maggi , F.M. : Outcome-oriented predictive process monitoring: Review and benchmark . ACM Trans. Knowl. Discov. Data 13 ( 2 ), 17 : 1 - 17 : 57 ( 2019 )

[23] Weinzierl , S. , Zilker , S. , Stierle , M. , Matzner , M. , Park , G.: From predictive to prescriptive process monitoring: Recommending the next best actions instead of calculating the next most likely events . In: Proceedings der 15. Internationalen Tagung Wirtschaftsinformatik , 2020