The increasing reliance on Machine Learning (ML) models for critical decision-making necessitates the development of trustworthy systems that foster confidence among all relevant stakeholders. To address this need, Explainable Artificial Intelligence (XAI) aims to improve the transparency of ML models, enabling a better understanding of their decision-making processes. Concurrently, Uncertainty Quantification (UQ) has emerged as a crucial ML research area, emphasizing the estimation and communication of uncertainties inherent to model predictions. Despite the importance of both XAI and UQ in facilitating informed decision-making, there is a noticeable gap in integrating these techniques efectively. This paper highlights our recent research endeavors to explore the synergy between XAI and UQ for predictive process monitoring. Our ifrst contribution, submitted to the Decision Support Systems journal, involves leveraging UQ to communicate the uncertainty present in ML explanations, ultimately promoting trust in the generated course of actions. Our second contribution, submitted to the Annals of Operations Research, employs XAI techniques to elucidate the factors contributing to ML model uncertainty, providing valuable insights for refining and enhancing the models. These insights are intended to shape future ML research in predictive process monitoring, fostering the development of more transparent, robust, and reliable systems.
1. Introduction
tional processes [
The acceptance of AI as a credible source of guidance is still a hurdle to be overcome, signifying the need for transparency. In response, two promising areas of ML research have risen to prominence: Explainable Artificial
2nd International Workshop on Process Management in the AI era
(N. Mehdiyev);
bidirectional integration of these research fields (see
Figure 1). Our findings and progress in this endeavor are
presented in this short paper. Section 2 discusses our first
contribution, where we employ the chosen UQ approach
This section summarizes our recently proposed method- color-coding.
ology for assessing and conveying uncertainty in ML For uncertainty-aware PDPs, we consider a set of
seexplanations [
Our proposed approach quantifies the model uncer- of stochastic forward passes is performed to calculate
tainty using the Monte Carlo dropout technique, a well- credible intervals for model predictions. After
conforestablished method in the deep learning domain. Uti- malization, these credible intervals serve as the basis for
lizing this method allows us to not only generate point identifying uncertainty profiles in PDP. Subsequently,
estimates from the posterior distributions but also to we count the number of allocations to each uncertainty
calculate corresponding credible intervals for assessing profile and place the majority profile for the examined
predictive uncertainties. However, a notable drawback predictor value, which is used to define the color in the
of this UQ method is the absence of formal guarantees. PDP. This process is repeated for all unique values of
To address this limitation, we implement Conformal Pre- the predictors. The pairs of predictor values and average
diction, a post-processing technique designed to tackle model predictions are plotted to create the PDP, with each
this specific challenge [
The pivotal step in integrating UQ with XAI in this coded according to the predominant uncertainty profile study lies in constructing uncertainty profiles. These for each point. Further visualizations, such as unconformalized credible intervals or pie charts representing 1submitted to Decision Support Systems uncertainty profile memberships for each predictor value, are introduced for a more comprehensive understanding of the uncertainty distribution.
The study includes expert interviews to assess the
suitability of the proposed approach and designed interface interval-based representation provides valuable insights
for a predictive process monitoring problem in the man- into the uncertainty of model outcomes, facilitating a
ufacturing domain. deeper comprehension of the model’s predictive
capabilities and inherent limitations.
3. Explaining the Machine To ensure model explainability, the study utilizes
SHapley Additive Explanations (SHAP), which provide local
Learning Uncertainty and global post-hoc explanations of the model’s
uncertainties (see Figure 3). Our methodology represents a
Our second manuscript2 employs an XAI technique to departure from traditional practices. Instead of relying
elucidate the factors contributing to ML model uncer- on point predictions, we use prediction intervals as the
tainty [
The study uses supervised learning to probabilistically the importance of fully leveraging the extensive data estimate a production-related parameter, specifically the resources available for informed decision-making. processing time of production events. This aspect of the method involves collecting and preparing process event data from Manufacturing Execution Systems (MES), 4. Conclusion which coordinate and track operational processes. The problem is treated as a predictive process monitoring problem, necessitating specific preprocessing, encoding, and feature engineering techniques to align with business and operational process data requirements.
To tackle model uncertainty, the study proposes using Quantile Regression Forests (QRF), an extension of the traditional Random Forests technique. QRF is designed for estimating conditional quantiles for high-dimensional predictor variables, ofering a non-parametric and precise approach for estimating prediction intervals. An
butions concerning the bidirectional integration of UQ and XAI. To the best of our knowledge, our study is the ifrst to merge UQ and XAI within the context of predictive process monitoring problems. We are confident that this work lays the foundation for developing responsible AI solutions for this domain. Future research can further enhance these studies by incorporating insights from relevant related research areas, such as privacy-preserving AI, algorithmic fairness, reliability, and safety, as well as human-centered design. By amalgamating these diverse perspectives, we hope to contribute to the development 2submitted to Annals of Operations Research of more robust, ethically sound, and user-oriented AI systems. This integrated approach aims to address not just the technical aspects of AI development but also the ethical and societal implications, leading to more holistic and beneficial AI solutions.