Explainable Prescriptive Process Analytics (Extended Abstract) Riccardo Galanti∗† ∗ myInvenio, Reggio Emilia, Italy, † University of Padua, Padua, Italy, Email: riccardo.galanti@my-invenio.com, riccardo.galanti@studenti.unipd.it Abstract—Within the realm of Process Mining, Process-Aware process instance, i.e. a case), looks at the activities executed Recommender systems (PAR systems) are information systems and attributes values and returns a KPI value. that aim to monitor process executions, predict their future Explainable AI is another field that has been overlooked behaviour, and finding optimal corrective actions to reduce the risk of failure or to maximize a given reference Key Performance in the last years, assuming that a good level of accuracy is Indicator (KPI). While a PAR system is composed by monitoring, sufficient for the process’ stakeholders to trust the recom- predictive analytics and prescriptive analytics, the focus has mender system (as well as the prediction system). However, been heavily on the first two, and very little attention has been the process actors need to be convinced that the recommended given to the last. Therefore, this PhD project firstly aims to actions are the most suitable ones to maximize the KPI of develop a technique that is able to provide good evidence-based recommendations, rather then relying on subjective opinions. A interest; otherwise they will not follow the suggestions given. second goal of the PhD project is to also incorporate techniques This leads us to the second goal of this PhD project: for Explainable AI inside PAR systems, in order to provide Research Question 02 How can users trust recommenda- and understand the root causes that put forward certain rec- tions provided by a PAR system? ommendations; otherwise, the process’ stakeholders and actors Finally, research results will be developed as software will unlikely trust and, hence, use them. Index Terms—Prescriptive Business Process Analytics, Process- modules, integrated in the process-mining suite of myInvenio, aware Recommender systems, Predictive models, Shapley Values, and evaluated with users expert of the process’ domain, in Explainable AI order to assess the general validity of the framework and the usability of the tool from a user experience point of view. I. R ESEARCH PROBLEM AND MOTIVATION II. L ITERATURE A NALYSIS Process-Aware Recommender systems are instances of a TABLE I: Analysis of related works wrt. relevant characteris- class of systems to monitor and predict how process instances tics of PAR systems are going to evolve, and to recommend the corrective actions Generic KPI Context-aware Underlying technique to recover the instances with higher risk not to achieve the Work Recommendation Recommendation Generalizable independent expected outcome. Conceptually, a PAR system is constituted Conforti et al. [4] Maggi et al. [9] +/- +/- + +/- + + - +/- by three main blocks: Monitoring, Predictive analytics and Schobel et al. [16] +/- +/- - + Schonenberg et al. [17] + - - + Prescriptive analytics. In the last years, a lot of research has Weinzierl et al. [23] - +/- + + been on the first two (commonly referred as Predictive Busi- ness Process Monitoring techniques) and several approaches The analysis focuses on the questions mentioned above. have been proposed (see e.g. [10], [22]). Conversely, the Table I shows the analysis of several works (one for each last block is overlooked, assuming that the users, after being row) related to the first research question, wrt. relevant PAR alerted of a potential failure, are able to find the proper systems’ characteristics (illustrated in the columns). The first corrective actions. However, it has been demonstrated by some is the possibility to recommend actions not only for improving on-the-field experts [5] to be not true. This is due to the a specific KPI (e.g. reducing the remaining time), but also for fact that, without support, process actors make decisions on a generic, user-customizable KPI. The second is the ability to the basis of their subjective opinion, rather then relying on recommend actions using all the attributes of the events, while objective data, which comes from the event logs and record the third is the possibility to generalize recommendations also the past executions and the achieved outcome. for unseen data. Finally, the last column shows if the developed The first goal of this PhD project is therefore summarized recommender system is loosely coupled to the implementation by the following research question: of other PAR system’s components. In each row, a symbol + Research Question 01 How can we build a prescriptive is shown if the developed recommender system tackles that business process analytics block that effectively maximizes a particular problem, otherwise a symbol - is shown. In this given reference KPI (Key Performance Indicator)? PhD project the first goal is to develop a PAR system able to In this PhD project, the outcome is measured through a tackle all the problems described above. customizable KPI function that, given an execution recorded The second goal deals instead with the problem of equipping in a log trace (which describes the life-cycle of a particular a PAR system with explanations of the recommendations Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). given. Few approaches exist in the literature to explain ma- [3] Breuker, D., Delfmann, P., Matzner, M., Becker, J.: Designing and chine learning models, arisen from the need to understand evaluating an interpretable predictive modeling technique for business processes. In: Business Process Management Workshops. pp. 541–553. complex black-box algorithms like ensembles of Decision Springer (2015) Trees and Deep Learning [2], [7], [8], [11], [14], [19]–[21]. [4] Conforti, R., de Leoni, M., Rosa, M.L., van der Aalst, W.M.P., ter Little research work has been conducted on explaining the Hofstede, A.H.M.: A recommendation system for predicting risks across multiple business process instances. Decis. Support Syst. 69, 1–19 outcome of process predictive monitoring. The most relevant (2015) work is by Rehse et al. [13], which also aims at providing [5] Dees, M., de Leoni, M., van der Aalst, W.M.P., Reijers, H.A.: What a dashboard to process participants with predictions and their if process predictions are not followed by good recommendations? In: Proceedings of the Industry Forum at BPM 2019. CEUR Workshop explanation. However, the paper does not provide sufficient Proceedings, vol. 2428, pp. 61–72. CEUR-WS.org (2019) details on the actual usage of the explainable-AI literature, [6] Galanti, R., Coma-Puig, B., de Leoni, M., Carmona, J., Navarin, N.: and the very preliminary evaluation is based on one single Explainable predictive process monitoring. arXiv:2008.01807 (2020) [7] Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model artificial process that consists of a sequence of five activities. predictions. In: Advances in neural information processing systems. pp. Breuker et al. also try to tackle the problem [3], but their 4765–4774 (2017) attempt is not independent of the actual technique employed [8] Lundberg, S.M., Nair, B., Vavilala, M.S., Horibe, M., Eisses, M.J., Adams, T., Liston, D.E., Low, D.K.W., Newman, S.F., Kim, J., et al.: for predictions. Furthermore, their explanations are only based Explainable machine-learning predictions for the prevention of hypox- on activity names, while explanations can generally involve aemia during surgery. Nature biomedical engineering 2(10), 749 (2018) resources, time, and more. [9] Maggi, F.M., Di Francescomarino, C., Dumas, M., Ghidini, C.: Pre- dictive monitoring of business processes. In: Advanced Information Systems Engineering. pp. 457–472. Springer (2014) III. P ROJECT ROADMAP [10] Márquez-Chamorro, A.E., Resinas, M., Ruiz-Cortés, A.: Predictive mon- itoring of business processes: A survey. IEEE Transaction on Services The construction of the Prescriptive Analytics module re- Computing 11(6), 962–977 (2018) quires first the development of a predictive module. It will rely [11] Meacham, S., Isaac, G., Nauck, D., Virginas, B.: Towards Explainable AI: Design and Development for Explanation of Machine Learning on simulating the possible customer-journey continuations and Predictions for a Patient Readmittance Medical Application, pp. 939– recommending those that will likely lead to higher satisfaction, 955 (06 2019) according to a predictive model, which is learnt from the data [12] Navarin, N., Vincenzi, B., Polato, M., Sperduti, A.: LSTM networks for data-aware remaining time prediction of business process instances. recorded in the event log. In: Proceedings of the IEEE Symposium Series on Computational We did an assessment of the state of the art of predictive Intelligence (SSCI 2017) (2017) algorithms [10], [22], which showed that LSTM has proven [13] Rehse, J.R., Mehdiyev, N., Fettke, P.: Towards explainable process pre- dictions for industry 4.0 in the dfki-smart-lego-factory. KI - Künstliche to be among the most effective AI techniques for predictive Intelligenz 33(2), 181–187 (Jun 2019) monitoring. We built an implementation based on the work [14] Ribeiro, M.T., Singh, S., Guestrin, C.: “why should I trust you”: done by Navarin et al. [12] and extended it to predict a generic Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and KPI of interest, instead of predicting only the remaining time. Data Mining, San Francisco. pp. 1135–1144 (2016) Afterwards, we built an explainable prediction framework [15] Sarthak, J., Wallace, B.C.: Attention is not explanation. In: Proceedings based on SHAP, which allowed determining for each feature of of the 2019 Conference of the North American Chapter of the Associa- tion for Computational Linguistics: Human Language Technologies. pp. the predictive vector how much it contributes to the prediction. 3543–3556. Association for Computational Linguistics (2019) This solution was chosen for several reasons. The SHAP [16] Schobel, J., Reichert, M.: A Predictive Approach Enabling Process implementation of the Shapley values for Deep Learning has Execution Recommendations, pp. 155–170. Springer International Pub- lishing, Cham (2017) the strong theoretical foundation of the original game theory [17] Schonenberg, H., Weber, B., van Dongen, B., van der Aalst, W.: Sup- approach, with the advantage of providing offline explanations porting flexible processes through recommendations based on history. that are consistent with the online explanations. Moreover, In: Dumas, M., Reichert, M., Shan, M.C. (eds.) Business Process Management. pp. 51–66. Springer Berlin Heidelberg (2008) SHAP avoids the problems in consistency seen in other [18] Serrano, S., Smith, N.A.: Is attention interpretable? In: Proceedings explanatory approaches (e.g. the lack of robustness seen in the of the 57th Annual Meeting of the Association for Computational online surrogate models, as analysed in [1]). Furthermore, it is Linguistics. pp. 2931–2951. Association for Computational Linguistics, Florence, Italy (2019) independent of the machine- or deep-learning technique that [19] Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features is employed to make the predictions.1 . A complete description through propagating activation differences. In: Proceedings of the 34th about the implementation of the framework is described in [6]. International Conference on Machine Learning-Volume 70. pp. 3145– 3153. JMLR. org (2017) [20] Shu, K., Cui, L., Wang, S., Lee, D., Liu, H.: Defend: Explainable fake R EFERENCES news detection. In: International Conference on Knowledge Discovery & Data Mining, SIGKDD. pp. 395–405. ACM (2019) [1] Alvarez-Melis, D., Jaakkola, T.S.: On the robustness of interpretability [21] Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep methods. arXiv preprint arXiv:1806.08049 (2018) networks. In: Proceedings of the 34th International Conference on [2] Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly Machine Learning-Volume 70. pp. 3319–3328. JMLR. org (2017) learning to align and translate. In: The 3rd International Conference on [22] Teinemaa, I., Dumas, M., Rosa, M.L., Maggi, F.M.: Outcome-oriented Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, predictive process monitoring: Review and benchmark. ACM Trans. 2015, Conference Track Proceedings (2015) Knowl. Discov. Data 13(2), 17:1–17:57 (2019) [23] Weinzierl, S., Zilker, S., Stierle, M., Matzner, M., Park, G.: From 1 An alternative could have been attention-based models, but a limitation is predictive to prescriptive process monitoring: Recommending the next linked to the lack of consensus that attention weights are always correlated best actions instead of calculating the next most likely events. In: to feature importance [15], [18] Proceedings der 15. Internationalen Tagung Wirtschaftsinformatik, 2020