Conformance checking is a key subfield of process mining. Despite its importance, there is relatively little adoption of advanced (procedural) conformance checking methods in the industry, which can be attributed to two limitations of state-of-the-art techniques: first, the poor scalability resulting from their inherent worstcase exponential nature. Second, the high cognitive load needed to interpret their results, which renders them inaccessible to non-experts. This Ph.D. project aims to close these gaps by developing conformance checking techniques suitable for industrial settings. Our research is structured into two workstreams. First, we investigate eficient techniques when one only needs to compute a single number to quantify the degree of discrepancy between an event log and a process model. In parallel, we investigate how to generate conformance diagnostics that can be easily understood without technical enablement. In both workstreams, there has been progress in terms of accepted papers that validate the problem and our proposed solutions.
Conformance checking is concerned with identifying and quantifying discrepancies between event logs
and process models. It is a central task in process mining, being an enabler for tasks such as process
discovery (by measuring the degree of discrepancy between the log and the model) and enhancement
(by correlating patterns of deviation with KPI performance). Despite that, state-of-the-art replay-based
conformance checking techniques such as token-based replay [
This Ph.D. project is motivated by problems encountered in industry when trying to productize conformance checking techniques. In particular, we learn from two key limitations of existing methods (with a special focus on trace alignments) which, in our experience, explain their poor adoption: first, their poor scalability, and second, the hard-to-digest diagnostics produced by them. Our research is split into two workstreams, depending on the conformance task to be solved.
Task 1 (Computing a Metric) The simplest task of conformance checking is to quantify the degree of
discrepancy between an event log and a process model as a single number metric. On the one hand,
conformance metrics are expected to ofer a series of quality guarantees such as determinism, monotonicity,
and robustness to partial mismatches. On the other hand, applications such as optimization-based
process discovery [
As shown in [
RQ Task 1.1 Introduce method for process trees 1.2 Extend to free-choice Petri nets 1.3 Formalize the stochastic problem 1.4 Extend to stochastic process trees 2.1 Define the diagnostics framework 2.2 Discover data and time patterns 2.3 Discover arbitrary patterns 2.4 Minimize arbitrary patterns
Status / Open Challenges
Published [
Correctly interpreting the result of alignment techniques requires awareness of the above limitations,
raising the entry bar for non-experts. This problem has been acknowledge in the literature [
Our research agenda is summarized in Table 1.
To answer the first research question, we build on the technique introduced in [
nets, we can already provide a polynomial-time approximation method based on decomposing the net and we are currently working on improving the approximation. For stochastic process trees, we might need diferent approaches since compositional methods do not work well with their stochastic perspective.
Evaluation: This research focuses on a well-studied problem in process mining. We resort to standard evaluation setups consisting of real-world and synthetic datasets to measure the runtime of the approach and to compare the induced model ranks with existing state-of-the-art techniques. The scalability of the approach must be measured across multiple dimensions: the size of (the state space of) the model, the size of the log, the number of distinct event types, and the degree of discrepancy between the model and the log.
To answer the second research question, we first observe that diagnostics provided by declarative
methods are often more understandable to non-expert users, however, declarative models are unsuitable
to model certain types of processes [
Templates
A. Discover Constraints
Declarative
Model
B. Constraint Minimization
C. Verify
Log
Diagnostics Log
Using the discovered constraints to verify the event log ensures that the diagnostics are on a higher
level and, thus, more understandable. Furthermore, the approach enjoys good quality properties such
as determinism and monotonicity of the generated diagnostics. Our first work [
Discover Arbitrary Patterns (2.3): Currently, the framework is limited to a set of constraint template patterns (such as the set of DECLARE patterns) which might not best capture a deviation. We would like to discover arbitrary behavioral patterns, i.e. arbitrary logical formulae beyond a fixed set of templates. For that, we must solve two challenges: First, we must devise an algorithm to discover arbitrary logical formulae (step A). Second, we must devise a mechanism to verbalize the discovered formulae to the user (step C). We believe that the latter can be tackled to some extend using NLP techniques such as rule-based methods or large language models. Therefore, we focus our attention on the first problem, for which our current idea is to extend existing methods [18] to work in our setting. In parallel, we are also evaluating ad hoc approaches that work by analyzing the structure of the process models [19]. Evaluation: Our evaluation will be primarily qualitative. To validate that our diagnostics are understandable, we plan to conduct a user study. However, user studies sufer from reproducibility issues. Therefore, we also aim to develop a set of reference process models for public datasets and demonstrate our method on top of them. With that, we aim to develop a “benchmark” for conformance diagnostics.
This Ph.D. project is a cooperation between the RWTH Aachen University and Celonis Labs GmbH. It is supervised by Prof. Dr. Ir. Wil van der Aalst and fully funded by Celonis.