Explainable Process Predictions (xPP): A Holistic Framework and Applications (Extended Abstract) Nijat Mehdiyev German Research Center for Artificial Intelligence (DFKI) Saarland University Saarbrücken, Germany nijat.mehdiyev@dfki.de A. Black-Box Machine Learning for Process Prediction I. INTRODUCTION The predictive strength of the adopted machine learning Business process prediction also referred to as predictive models is one of the most important prerequisites for generating process monitoring or predictive business process management reliable robust, and consistent explanations. For this purpose, we is a branch of process mining that pursues the objective to have investigated various black box approaches for different predict the target of interest by using the activities from the process prediction problems. In one of our earlier studies we process traces. Recently, several studies have been conducted to applied pre-trained stacked autoencoder based deep neural explore the applicability of various machine learning approaches networks to address next event prediction problem after carrying for different problems in the process prediction context such as out an intensive data-preprocessing procedure on the event log next event prediction, process outcome prediction, prediction of data including n-gram encoding, extracting data flow and service level agreement violations, remaining time prediction, resource features and feature hashing [1]. Consequently, this risk prediction, cost prediction, prediction of activity delays etc. study was extended by applying hyperparameter optimization of The recent research also suggests that the black-box machine the adopted deep learning approach and by addressing learning approaches especially deep learning methods provide imbalanced classification problem [2]. In another study, a deep superior results for process prediction problems compared to LSTM approach was applied on sensor time series data from the conventional approaches. However, these opaque, non- process industry to detect the quality of the semi-finished transparent models lack the capabilities to provide explanations products and accordingly to predict the next production process about their reasoning trace or delivered outcomes. This in turn step [3]. The necessity of this approach and an overview of other introduces the barriers to operationalizing data-driven decision- relevant methods and systems for industrial predictive process making since the users tend not to use the outcomes by such analytics were presented in [4]. Furthermore, a discussion of artificial advice givers due to the lack of understanding or predictive process analytics based on the log data generated by justification. Manufacturing Execution Systems (MES) was introduced in [5] II. THESIS CONTRIBUTION which also presented a use-case from individual manufacturing. Explainable Artificial Intelligence (XAI) has recently B. A Framework for Explainable Process Predictions reemerged as an important research domain with the purpose to Making process predictions delivered by black-box machine establish the trust between human users and AI systems by learning models explainable is a multi-dimensional and multi- making the communication understandable and transparent. faceted issue that requires to consider the context of explanation Although, the predictive process analytics has been recently situation, the users‘ preferences and backgrounds, the nature of emerging as an important research area, the explainability issues underlying processes, the defined technical and economic in this domain have only been partially addressed. To fill this objectives, the organizational factors, etc. when developing research gap, this thesis makes three major contributions. First, explanation systems. Hence, it conceivable to suggest there is no this study explores the applicability of black-box machine “one-fits-all” explainable process prediction solution and a learning approaches, particularly deep neural networks, for systematic approach is required in generating adequate different process prediction problems in various domains. explanations by incorporating the implications from various Second, this study attempts to propose a theory-driven dimensions of the decision-making environment. To fill this conceptual framework which is presumed to guide developing research gap, this thesis aims to propose a conceptual explainable process prediction solutions by providing an framework which is supposed to guide the process mining overview of important aspects of decision-making situations. practitioners and researchers in designing and developing the Third, the applicability of explainable process predictions is explanation solutions for predictive process monitoring illustrated by adopting well-recognized or developing new problems. For this purpose, our study [6] proposed an initial explanation methods for different use-cases. holistic framework by analyzing, combining and adapting the Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). propositions from the explainable artificial intelligence research applied which facilitate the domain experts to examine the domain. The constructs of the proposed framework include explanations from different perspectives. In the third use-case, subjects, objectives, instruments/techniques, context, we explored the applicability of the explainable process generation time and other various elements. The subjects of predictions for an incident management with the purpose to explainable process prediction solutions are various make model decisions for domain experts justifiable [6]. stakeholders with different levels of knowledge background, Different from previous two studies, which adopted known XAI expertise levels and explanation preferences. Process owners, methods, in this study we have proposed a novel local post-hoc process/data analysts, process/data engineers, domain experts, explanation approach by defining the local regions from the regulatory authorities, supervisory bodies etc. are some of the validation data by using the intermediate learned representations key users for each of them customized explanation methods of the applied black box approach. Particularly, the learned have to be designed that conform to these users’ mental models. neural codes from the last hidden layer of the applied neural These users pursue various objectives such as justification, network were used as input to clustering algorithm to define the ratification, verification, duplication, debugging, learning, local regions. Finally, a local surrogate decision tree was fitted. satisfaction, effectiveness/efficiency etc. For instance, the By adopting this approach, we aimed to avoid the shortcomings knowledge engineers such as process engineers are more related to perturbation-based approaches such as LIME or interested in the reasoning mechanism of the black-box Shapley. Furthermore, our other studies on explainable process approaches, follow mainly the verification objective and prefer analytics address various use-cases and approaches [9], [10]. the explanation methods that provide algorithmic transparency. On the other side, the domain experts or process owners with III. OUTLOOK limited machine learning background opt to justify the goodness The future work comprises activities to enhance the of the models’ individual decisions. There have been numerous proposed framework for explainable process predictions studies to investigate various XAI techniques by proposing through the lens of information systems theory, particularly different taxonomies. E.g. the explanation techniques can be activity theory combined with the design science research. classified as methods related to transparency which aim to Furthermore, a thorough integration of explanation methods will examine how the model works or as post-hoc explanation be examined by ensuring the robust transitions among them with approaches that attempt to extract the explanation from learned the goal to facilitate the users in examining the explanations model without investigating the reasoning mechanism of the from different perspectives. An investigation of xPP for other model. Regarding the relationship with the underlying black- use-cases such as explaining the reasons for algorithmic fairness box model the explanations are categorized as model-specific or violations is another future research direction. model-agnostic approaches. A further categorization refers to the scope of the generated explanations. The global explanation REFERENCES approaches attempt to deliver explanations for the whole dataset [1] N. Mehdiyev, J. Evermann, and P. Fettke, “A multi-stage deep learning whereas the local explanation techniques make the individual approach for business process event prediction,” IEEE 19th Conference observations explainable. Generation time informs the users on Business Informatics, (CBI) 2017, vol. 1, pp. 119–128. when the explanations are generated. They can be obtained [2] N. Mehdiyev, J. Evermann, and P. Fettke, “A Novel Business Process before developing the black-box machine learning models (pre- Prediction Model Using a Deep Learning Method,” Bus. Inf. Syst. Eng., pp. 1–15, 2018. model), during the model implementation (in-model) and finally [3] N. Mehdiyev, J. Lahann, A. Emrich, P. Fettke, and P. Loos, “Time Series after the model is trained (post-model). Classification using Deep Learning for Process Planning: A Case from Process Industry,” in Procedia Computer Science, 2017, pp. 114, 242-249. C. Applying/Developing Explainable Process Prediction [4] N. Mehdiyev, A. Emrich, B. Stahmer, P. Fettke, and P. Loos, “iPRODICT Solutions - Intelligent process prediction based on big data analytics,” in CEUR In this thesis, the proposed conceptual framework is Workshop Proceedings (BPM Industry Track), 2017, vol. 1985. instantiated to develop various explainable process prediction [5] P. Fettke, L. Mayer, and N. Mehdiyev, “Big-Prozess-Analytik für solutions. For illustrative purposes, three use-cases are presented Fertigungsmanagementsysteme (MES),” in Big Data: Anwendung und which we have already examined in various studies [6]–[8]. In Nutzungspotenziale in der Produktion, 1st ed., M. Steven and T. Klünder, Eds. Stuttgart: Kohlhammer, 2020, pp. 215–239. the firs use-case, the main users of the explainable process [6] N. Mehdiyev and P. Fettke, “Explainable Artificial Intelligence for prediction solutions are the process owners of a Dutch Process Mining: A General Overview and Application of a Novel Local autonomous administrative authority [7]. They are interested to Explanation Approach for Predictive Process Monitoring,” understand the user behavior globally that leads to using more arXiv2009.02098, 2020. – accepted to the edited volume "Interpretable expensive channels such as sending messages. We applied first Artificial Intelligence: A perspective of Granular Computing" (published a deep neural network to predict when the users tend to send by Springer) messages. Since the objective in this use-case was defined as [7] N. Mehdiyev and P. Fettke, “Prescriptive Process Analytics with Deep Learning and Explainable Artificial Intelligence,” in 28th European understanding the global user behavior to make a strategic Conference on Information Systems (ECIS), 2020. process enhancement decision, we have adopted the Partial [8] N. Mehdiyev and P. Fettke, “Local Post-Hoc Explanations for Predictive Dependence Plots (PDP), a global, model-agnostic post-hoc Process Monitoring in Manufacturing,” arXiv2009.10513, 2020 explanation technique to generate the relevant causal [9] J.-R. Rehse, N. Mehdiyev, and P. Fettke, “Towards Explainable Process explanations. In the second use case, we explored the potential Predictions for Industry 4.0 in the DFKI-Smart-Lego-Factory,” KI- of explainable process predictions to enable data-driven process Künstliche Intelligenz, pp. 1–7, 2019. planning in manufacturing [8]. For this purpose, two [10] P. Lübbecke, N. Mehdiyev, and P. Fettke, “Substitution of hazardous complementary local post-hoc explanation approaches, Shapley chemical substances using Deep Learning and t-SNE,” in Wirtschaftinformatik (WI), 2019. Values, and Individual Conditional Expectation (ICE) plots are