Leveraging the Full Potential of Conformance Checking in Practice

Abstract

As one of the main applications of process mining, conformance checking algorithms have been developed and extended in past research. Conventional techniques for conformance checking require a to-be model and an event log as input and determine whether a completed or running process instance deviates or not. However, these techniques only provide a solution to compare intended and actual behavior, thus not focusing on whether input requirements can be fulfilled and outputs can be used in practice. In this PhD thesis, we aim to help practitioners to leverage the full potential of conformance checking. We intend to develop approaches that tackle problems with respect to input requirements in the practical application and equip managers with detailed information on the output of conformance checking, i.e., process deviations, including their causes, and efective prevention strategies.

Keywords

• eol>Process Mining
• Conformance Checking
• Practical Application

1. Introduction

The conformance of business processes is crucial for the success of organizations [ 1 ]. To ensure it, process mining research proposed techniques for conformance checking, which compare process behavior in an event log to the intended behavior in form of a to-be model [ 1 ]. For that, three main technical solutions have been developed: rule-checking, token-based replay, and alignments [ 1 ]. They can be used to check whether process instances deviated at run-time and quantify how good the recorded behavior fits the intended behavior. Alignments are considered the most sophisticated approach as they provide the most fine-granular information (i.e., where exactly process instances deviated) and symmetrically view to-be model and event log [ 1 ].

Although the aforementioned techniques provide solutions for the comparison of model and event log, they might not always be applicable in practice because their input requirements are not fulfilled [ 2 ]. For example, the to-be model cannot be provided [ 3 ] or existing techniques cannot check process conformance to constraints with multiple objects based on event logstandards like XES [ 4 ]. However, to leverage the full potential of conformance checking, process managers should actually apply conformance checking and use its results to improve the process, which is rarely happening in practice [ 5 ]. To address this problem, we want to answer the research question: RQ1: How can we enhance conformance checking such that its input requirements are easier to fulfill in practice?

Furthermore, the output of conformance checking, i.e., the mere information whether a process instance deviated in the past, might not help practitioners directly to improve their processes [ 1 ]. Rather, they need additional information about how and why the process deviated to take measures that increase process conformance [ 1 ]. Currently, managers have to manually conduct analyses about the deviations and how to prevent them in the future [ 6 ] although automated analyses would significantly speed up and secure the efectiveness of process improvement. To address this problem, we want to answer a second research question: RQ2: How can we provide practitioners with automated analyses of conformance checking output?

2. Research Agenda

In order to answer both research questions, we planned diferent topics of research targeted either at fulfilling input requirements or using outputs of conformance checking techniques. Some are ongoing and others are planned in the future. They are summarized in Fig. 1.

2.1. RQ1: How to fulfill input requirements?

Previous research has found that the input requirements of conformance checking (i.e., to-be model and event log) are dificult to fulfill in practice because either the required data is not available or it does not capture the process in the necessary level of detail [ 2, 6 ]. We aim to develop approaches that allow more process managers to perform conformance checking despite these problems. Concretely, we focus on the potential absence of to-be process models and on the presence of multiple conformance-relevant objects in one process. Absence of to-be process models. Organizations may not have defined to-be models for their processes because it is a time-consuming and error-prone task to define these models [ 3 ]. Thus, a mandatory input requirement for conformance checking might not be fulfilled. However, it is very likely that process managers can define few deviating process instances. Therefore, we want to investigate the possibility to train a machine learning model based on few-shot learning that is able to perform a conformance check without a to-be model as input. To do so, we want to apply a triplet loss function [ 7 ] or few shot learning techniques like weakly supervised learning or transfer learning [ 8 ] and validate whether we find the same process instances to be deviating as conventional techniques. This includes a verification that mining a process model with known discovery techniques followed by conformance checking is not equally efective. Multiple conformance-relevant objects in one process. Most existing conformance checking techniques assume that a process can be defined by a single case notion, i.e., all actions are related to exactly one object define a process instance. However, a process might only be conforming if multiple objects fulfill desired criteria [ 9 ]. For example, consider a delivery process where multiple customers are supplied in multiple shipments with multiple items and the process is only conforming if all customers receive the prescribed quantity of all items. To account for multiple objects in a process instead of using only one case identifier, object-centric conformance checking has been proposed [ 4 ]. However, it has not been applied in practice. We want to develop an object-centric conformance checking approach in an existing industry cooperation, which will provide practical requirements and evaluation data. This includes an assessment which event log format is applicable in practice as a object-centric standard has yet to be determined [ 9 ].

2.2. RQ2: How to use output in practice?

To quickly provide process managers with the right information about deviations so they can improve their process conformance, several automatic analyses can be of help. We decided to predict, explain, and discover patterns in deviations as well as assess their desirability. We aim to validate these approaches with real-life data that is publicly available or shared with us by a company partner and illustrate generated insights.

Predict deviations. Process managers can proactively manage process conformance if they know which deviations will happen in ongoing process instances. This prediction task faces several challenges as multiple deviations can occur and they often occur infrequently, leading to data imbalance. Existing approaches for deviation prediction either lack the ability to predict which activity will deviate and to do this suficiently early [ 10 ] or cannot deal with the imbalanced nature of this task [ 11 ]. Thus, we trained a neural network that can cope with the challenges of this task and predict deviations suficiently early, allowing process managers to prevent them [ 12 ].

Explain deviations. Conformance checking techniques only identify deviations and cannot provide any reasons for their occurrence [ 6 ]. To explain these occurrences, we aim to apply causal discovery techniques and discover true reasons for non-conforming behavior, thus extending our previous work [ 13 ]. We want to illustrate its practical value in a real-life application. This extends approaches that try to explain non-conformance through mere correlations [14] as well as approaches that apply causal discovery to other process elements [15]. Discover deviation patterns. The results of trace alignments are fine-granular (i.e., whether an activity is inserted or missing). However, two deviations might form a pattern like swapped activities or replaced activities. These patterns are more meaningful for practitioners as they aggregate problems on a managerial level. Some works have conceptualized which patterns can potentially occur [16] whereas other have included these patterns in the conformance check by formally modeling them in the to-be model [17]. Nevertheless, there is no approach that can discover patterns and thus synthesize deviations in trace alignments. We aim to develop a rule-based approach that derives all patterns in trace alignments.

Assess desirability of deviations. Some deviations from the to-be model might be desirable because they might be necessary in emergency situations [17]. Practitioners would profit from approaches that classify whether a deviation is desirable or not. To develop an approach that automatically classifies this desirability, we plan to extract their semantic meaning [18] and either use rule-based or machine learning classifications (possibly including Large Language Models).

Acknowledgments

This PhD thesis is supervised by Jana-Rebecca Rehse. [14] M. de Leoni, W. M. P. van der Aalst, M. Dees, A general process mining framework for correlating, predicting and clustering dynamic behavior based on event logs, Inf Syst 56 (2015). [15] M. Qafari, W. van der Aalst, Case level counterfactual reasoning in process mining, in:

Intel Inf Syst, Springer, 2021, pp. 55–63. [16] M. Hosseinpour, M. Jans, Process deviation categories in an auditing context, SSRN (2020). [17] A. Adriansyah, B. F. v. Dongen, N. Zannone, Controlling break-the-glass through alignment, in: SCSM, 2013, pp. 606–611. [18] A. Rebmann, H. van der Aa, Enabling semantics-aware process mining through the automatic annotation of event logs, Inf Syst 110 (2022) 102111.

References

[1] J. Carmona , B. van Dongen , A. Solti , M. Weidlich , Conformance Checking - Relating Processes and Models , Springer, 2018 .

[2] S. Dunzer , M. Stierle , M. Matzner , S. Baier , Conformance checking: A state-of-the-art literature review, in: S- BPM ONE , ACM, 2019 , pp. 1 -- 10 .

[3] F. Friedrich , J. Mendling , F. Puhlmann , Process Model Generation from Natural Language Text , in: CAiSE, Springer, 2011 , pp. 482 - 496 .

[4] L. Liss , J. N. Adams , W. van der Aalst, Object-centric alignments, 2023 . arXiv: 2305 . 05113 .

[5] L. Reinkemeyer , Process mining in action, Process Mining in Action Principles, Use Cases and Outloook ( 2020 ).

[6] D. Borrego , I. Barba , Conformance checking and diagnosis for declarative business process models in data-aware scenarios , Expert Syst Appl 41 ( 2014 ) 5340 - 5352 .

[7] A. Hermans , L. Beyer , B. Leibe , In defense of the triplet loss for person re-identification , arXiv preprint arXiv:1703.07737 ( 2017 ).

[8] Y. Wang , Q. Yao , J. T. Kwok , L. M. Ni , Generalizing from a few examples: A survey on few-shot learning , ACM Comput Surv 53 ( 2020 ).

[9] W. M. P. van der Aalst , Object-centric process mining: Unraveling the fabric of real processes , Mathematics 11 ( 2023 ).

[10] S. Weinzierl , S. Dunzer , J. Tenschert , S. Zilker , M. Matzner , Predictive business process deviation monitoring , in: ECIS , 2021 , pp. 1 -- 10 .

[11] L. Genga , C. Di Francescomarino , C. Ghidini , N. Zannone , Predicting critical behaviors in business process executions: When evidence counts , in: BPM Forum , Springer, 2019 , pp. 72 - 90 .

[12] M. Grohs , P. Pfeifer , J.-R. Rehse , Business process deviation prediction: Predicting nonconforming process behavior , in: ICPM , 2023 .

[13] M. Grohs , J.-R. Rehse , Attribute-based conformance diagnosis: Correlating trace attributes with process conformance , in: ICPM Workshops , Springer, 2023 , pp. 203 - 215 .