Business Process Simulation: From Valid Models to Valuable Insights

Abstract

Business process simulation (BPS) has traditionally relied on expert‐crafted models to anticipate process changes and support decision‐making. Recent data‑driven methods democratize BPS by automatically deriving simulation models from event logs, increasing accessibility to non-experts. However, this shift introduces two critical challenges: ensuring that automatically generated models are valid without modeling expertise, and providing analysis approaches that go beyond singular “what‑if” runs to surface deeper process insights. This doctoral project tackles these challenges through two complementary streams. First, we establish foundations for valid simulation by systematically reviewing existing validation practices and identifying threats to internal validity in common data‑driven research methods. Second, we leverage these validated models to develop approaches for quantifying process robustness and translating simulation outputs into actionable explanations. Together, these contributions aim to make data‑driven BPS more valid and insightful, advancing its practical utility for strategic decision support. This paper outlines the four interrelated research goals of this doctoral project covering the two streams, related research activities addressing these goals, and current progress.

Keywords

• Process mining
• Business process simulation
• Data-driven simulation

1. Introduction

Business process simulation (BPS) is a valuable tool for supporting decision-making in organizations [ 1 ]. BPS refers to the abstraction of business processes into executable simulation models that allow for the generation of numerous synthetic process instances. By simulating how a process behaves under diferent conditions, BPS enables the analysis of potential changes without implementing them in reality [ 1 ]. This allows organizations to anticipate outcomes, reduce risks and costs associated with process changes, and support continuous improvement and innovation [ 2 ]. While BPS traditionally required substantial manual modeling efort [ 3 ], recent years have seen the emergence of data-driven approaches that derive simulation models from historical event data [ 4, 5, 6, 7, 8, 9 ]. These advances have improved simulation accuracy and reduced the entry barrier for applying simulation, thereby enabling broader adoption beyond expert users.

Despite this democratization, BPS remains primarily limited to low‑stakes experimentation rather than driving strategic, high‑impact decision-making [ 10 ]. Two key requirements to make data-driven BPS more usable are only partially met by current research. First, the validity of the simulation models needs to be ensured. A previously widely used technique to validate simulation models relied on expert judgment [ 11 ]. However, with the growing use of data-driven techniques and the involvement of users with less modeling and domain expertise, alternative and more automatable validation approaches are required [ 12 ]. Second, users need better support for using simulation to gain deeper insights. While traditional BPS often focused on answering singular “what-if” questions, the increased accessibility of simulation enables users to explore deeper insights and make sense of complex simulation behavior. Prior eforts have explored ways to improve and ensure BPS model validity by proposing new evaluation methods [ 12, 13 ]. Furthermore, usability through more advanced analysis methods based on

CEUR

ceur-ws.org simulation [ 14, 15 ]. However, there is still a limited understanding of how BPS tools can be ensured to be valid and insightful to drive adoption further.

With this doctoral project1, we aim to strengthen this understanding. To do this, we structure the project into two complementary focus areas, each targeting one of the above requirements and together ensuring that data‑driven BPS becomes both valid and insightful: Validation focus: The first focus area concentrates on establishing the foundations for valid simulation models. We critically review existing validation practices of agent-based simulation models due to the agent focus of the larger doctoral project. Furthermore, we evaluate whether newly proposed data-driven practices actually provide the level of validity they are intended to ensure.

RG1 Validation Practices: Systematically review and classify validation methods for agent‑based and data‑driven BPS models, establishing a comprehensive understanding of existing methods. RG2 Validity Threats: Identify and categorize internal validity threats in the commonly applied research method to build accurate data-driven BPS models.

Insight focus: The second focus area builds on these validated models to develop novel analysis approaches that make simulation more insightful in practice. By proposing new approaches that make use of validated simulation outputs, we aim to support deeper, data-driven insights into the behavior and robustness of business processes. We aim to achieve this by using simulation to assess the processes’ robustness and explain such findings.

RG3 Process Robustness: Develop a simulation‑based approach to measure how key performance indicators respond to systematic parameter changes, supporting process robustness analysis. RG4 Explainability: Develop an approach that translates complex simulation outputs into clear, actionable explanations, helping users understand the drivers behind observed results. By first ensuring that models are valid (RG1–RG2) and then providing tools for deeper insights (RG3–RG4), this project ensures that data‑driven BPS becomes both valid and insightful for strategic decision support.

The remainder of this description of the intended doctoral project is structured as follows. Section 2 provides related work on the analysis of simulation results. Section 3 gives an overview of the current state of the project and the research plan. Section 4 concludes the paper with an outlook.

2. Related Work

In this section, we provide a concise overview of the literature on both focus areas—validating datadriven BPS models and gaining insights from them—and describe how the doctoral project relates to this research.

Validity of BPS models. Research related to RG1 and RG2 primarily addresses validating BPS approaches through evaluation. This process involves estimating how accurately a simulation approach reflects certain process dimensions, such as control-flow, which is crucial for a more informed use of BPS. This has been done for individual approaches [ 4, 16 ] or comparing diferent ones [ 17, 6 ]. Recently, two evaluation frameworks were proposed to assess the quality of simulation models. To achieve that, one framework compares simulated processes with historical data for various process dimensions [ 12 ], to provide deeper insight into the simulation model’s quality. Another framework focuses on evaluating the quality of simulated processes by assessing their efectiveness in predictive process monitoring tasks, comparing the performance of models trained on simulated data with those trained on real data [ 13 ]. In addition, an initial validation of a “what-if” module of a simulation approach has been proposed [ 5 ], giving a first indication of how well the simulation under varying simulation parameters works. These eforts partially address both RG1 and RG2. However, as we show in our work described in Section 3.2, current methods have deficiencies that do not always allow users to rely on them. Under which conditions this is the case remains to be identified. 1Robert Blümel is part of a joint project of SAP Signavio, Prof. Van der Aa (University of Vienna), Prof. Stuckenschmidt (University of Mannheim), and a doctoral student, supervised by Prof. Stuckenschmidt. The project’s main focus lies on agent-based business process simulation.

Insights from data-driven BPS. While validation research has made significant strides, work on drawing deeper insights from data‑driven BPS is still emerging. Some studies embed simulation within broader process analytics to support end‑to‑end analyses [ 10, 18, 19, 20 ], but these require extensive system integration. Others improve usability by ofering interactive scenario building [ 14 ] or by applying optimization strategies for parameter tuning and resource allocation [ 15, 21, 22 ]. These eforts demonstrate the value of simulation beyond singular what‑if runs and demonstrate simulation’s potential for insight in democratized settings, as planned in our RG3. Yet, they typically focus on individual parameter adjustments, rather than on systematically assessing how multiple factors interact or on quantifying model sensitivity.

Research related to RG4 remains restricted to general explainability methods. In the field of artificial intelligence applications, various explainability methods exist that are becoming more prominent and sophisticated (see, e.g., [ 23 ]). In addition to those general methods, approaches such as the discovery of causal dependencies from event logs [24] could serve as a basis for explainability in the field of BPS. However, despite these advances, no research has been proposed to leverage these or other approaches to further explain BPS results.

3. Current State and Research Plans

Building on the two complementary focus areas—establishing model validity and enabling deeper insights—this doctoral project is currently organized into three active research streams, each corresponding to one of the first three research goals. The first stream (RG1) conducts a systematic literature review to map out existing validation practices for agent‑based and data‑driven BPS models; the second stream (RG2) investigates and mitigates threats to internal validity in data‑driven workflows; and the third stream (RG3) develops a simulation‑based approach for assessing process robustness under systematic variation. For each of these streams, we outline the specific problem being targeted, our contributions toward overcoming it, and its current state.

3.1. Validation of Agent-based Simulation Models

As a first work stream, this doctoral project contributes to a broader systematic literature review (SLR) on agent-based modeling and simulation (ABMS) in the context of collaborative business processes. ABMS is increasingly recognized as a suitable paradigm for simulating business processes, particularly due to its ability to model complex interactions among autonomous agents and capture emergent behavior in decentralized systems [ 9 ]. These characteristics make it well-suited for representing organizational dynamics, where local agent behaviors lead to global outcomes. The review investigates four core areas: (i) conceptual modeling, (ii) the discovery of agents, (iii) architectural designs for agent-based simulation models, and (iv) the validation of such models. While all four areas are explored as part of the joint review, for this doctoral project, we specifically focus on the fourth area: the validation of agent-based simulation models in order to gain an understanding of previous methods (RG1). Problem. Despite the recognized potential of ABMS in simulating business processes, a significant gap remains in the literature concerning an overview of the validation of agent-based simulation models. Validation is crucial to ensure these models accurately represent real-world systems. Understanding validation methods is essential to addressing RQ1, as it reveals what insights can be drawn from simulated logs regarding a model’s accuracy in real-world processes. While two recent SLRs have begun to explore ABMS in relation to process mining [25, 26], their focus is narrowly confined to process mining contexts, lacking a broader perspective on ABMS applications in business process management. As a result, related literature that does not explicitly employ process mining techniques needs to be reviewed, providing a more comprehensive understanding of ABMS validation for business processes. Contribution. To address these gaps, we undertake the following: 1. Conceptual goals clarification. We examine the goal of validation within ABMS. 2. Technique classification. We collect and classify the validation techniques used, referencing

Sargent’s technique overview [ 11 ]. 3. Quantification analysis. We analyze how validation outcomes are quantified, focusing on commonly used metrics and their input and scope.

These objectives define the specific contribution of this doctoral work within the larger review efort. Current state. The SLR is currently in progress, following Kitchenham’s [27] methodology for conducting literature reviews. The study planning has been completed, and the study execution phase is nearing completion. After applying inclusion and exclusion criteria to a pool of nearly 1,700 deduplicated papers, 89 were identified as relevant to our research questions.

3.2. Threats to the Validity of Data-driven Business Process Simulation

The second work stream of this doctoral project focuses on investigating threats to validity in BPS to address RG2. As previously discussed, BPS facilitates what-if analyses for organizations, allowing them to foresee the impacts of potential changes on their processes [ 1 ]. Hence, organizations rely on a certain quality in their simulation model and the assessment as to whether the simulation model actually has this quality. To arrive at this assessment, typically, data-driven approaches streamline model creation and evaluation through a structured six-step method using historical event data [ 4 ], However, this method can introduce systematic errors that bias evaluation results and distort confidence in simulation model quality.

Problem. This work addresses threats to internal validity in this six-step method. A method is internally valid when changes in the observed outcomes are a direct result of the manipulations within the method itself, rather than external factors [28]. Hence, internal validity is compromised when factors beyond the intended scope of a method afect the observed outcomes, introducing systematic errors in evaluation results [29]. In BPS, maintaining internal validity means that evaluation results must solely reflect the algorithm’s true quality, uninfluenced by factors such as concept drift or data leakage. Such influences can lead to incorrect assessments of a model’s capacity to accurately simulate real-world processes. Contribution. To address this challenge, we make the following threefold contribution: 1. Method explication. We explicate the six-step research method as an object of study, clearly defining its entities, relationships, and procedural decisions. This follows Frank’s definition, which states that a research method includes both terminology and a corresponding process [30]. 2. Threat identification. We identify five concrete threats to internal validity, illustrating each with empirical examples drawn from public event logs [ 12 ]. 3. Best practices. We propose five best practices to mitigate these threats, thereby enhancing the validity of simulation assessments and supporting informed business process and IS redesign. These contributions aim to improve the validity of BPS, thereby addressing RG2 by ensuring that the insights gained from simulation models are accurate and not distorted by threats to internal validity. Current State. Currently, we await peer reviews for this research stream following a journal submission. Prior to this, the project underwent two conference submissions and was rejected, primarily due to methodological critiques. In response, we undertook revisions to strengthen both the methodological foundation and empirical validation.

3.3. Process Robustness: A Simulation-based Robustness Assessment Approach

When analyzing their processes to identify areas for further improvement, organizations can draw on a wide range of process analysis techniques. Conformance checking compares the observed as‑is behavior against a reference model to pinpoint deviations [31]. These deviations then guide structural or behavioral adaptations—such as increasing flexibility to better handle variability [ 32]. Simulation further extends these analyses by allowing practitioners to evaluate process performance under diferent configurations [ 2 ]. Complementing this, resilience analysis examines how well a process tolerates short‑term shocks in parameters like arrival rates or resource availability [33].

In practice, process changes are guided by specific performance targets—most notably key performance indicators (KPIs) such as cycle time thresholds. Although simulation can be used to compare varying simulation parameters with respect to these KPIs, users lack insight into how far each parameter can be varied before KPI targets are violated. Prior work has enhanced simulation with interactive scenario building [ 14 ] and optimization‑based parameter tuning [ 15, 21, 22 ], yet no existing method systematically uncovers the limits within which parameter changes remain viable. Consequently, businesses remain uncertain both about where there is room for improvement and about where further adjustments will breach KPI constraints.

To close this gap, we develop a simulation‑based approach to assess process robustness—the ability of a process to maintain KPI compliance under systematic parameter variation. Drawing on robustness concepts from manufacturing [34] and bioprocess engineering [35], we define robustness as the capacity of a process to tolerate changes in key parameters without breaching specified KPI limits. This directly addresses RG3 by equipping users with structured insights into their process’s tolerance to change. Specifically, our contribution aims to: 1. Formalize process robustness. Identify the intervals for each parameter (e.g., arrival rate, task duration, resource count) within which all KPI targets remain satisfied. 2. Implement adaptive exploration. Use a gradient‑based search to eficiently navigate the high‑dimensional parameter space. 3. Generate semifactual reports. Translate findings into human‑readable semifactual statements such as, “Even if arrival rate increases by up to 15%, average cycle time remains below 48 hours.” Together, these contributions ensure that simulation results do not merely predict outcomes under ifxed settings but also reveal the boundaries of safe change, thereby making simulation outputs more actionable for risk‑aware decision‑making.

Current state. This work is currently under development. Following the algorithm engineering methodology by Mendling et al. [28], we have identified the real-world problem as well as the algorithmic task. Based on that, we are currently in the algorithm design phase.

4. Outlook

This doctoral project aims to advance business process simulation (BPS) by addressing key challenges that have emerged from its growing democratization through data-driven approaches. In particular, it focuses on two focus areas: ensuring that simulation models are valid and supporting their efective use for gaining actionable process insights.

Initial work has begun to address this on three research streams. First, a systematic literature review is underway to examine validation techniques for agent-based simulation models, with data extraction currently in progress (RG1). Second, we have identified five threats to the internal validity of data-driven BPS evaluation methods and proposed corresponding mitigation practices; this work is currently being revised for journal submission (RG2). Third, we are designing an approach to use simulation capabilities to assess process robustness to derive more insights from various simulation scenarios (RG4).

Future work will extend the project by adding a fourth research stream with a stronger emphasis on explainability (RG4). This stream aims to develop an approach to make simulation outputs more transparent and interpretable to support the adoption of insights in practice. This includes investigating how contributing factors to simulation outcomes can be made more transparent to support a valid and more informed use of the gained insights.

Declaration of generative AI

During the preparation of this work, the authors used ChatGPT in order to improve the readability and language of the manuscript. After using this service, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article. [24] Z. D. Bozorgi, I. Teinemaa, M. Dumas, M. La Rosa, A. Polyvyanyy, Process mining meets causal machine learning: Discovering causal rules from event logs, in: ICPM, IEEE, 2020, pp. 129–136. [25] F. Bukhsh, R. Govers, R. Bemthuis, M. Iacob, Exploring the integration of agent-based modelling, process mining, and business process management through a text analytics–based literature review, in: The Oxford Handbook of Agent-based Computational Management Science, Oxford University Press, 2024. [26] R. H. Bemthuis, S. Lazarova-Molnar, Towards integrating process mining with agent-based modeling and simulation: State of the art and outlook, Expert Systems with Applications 281 (2025) 127571. [27] B. Kitchenham, S. Charters, et al., Guidelines for performing systematic literature reviews in software engineering version 2.3, Engineering 45 (2007) 1051. [28] J. Mendling, H. Leopold, H. Meyerhenke, B. Depaire, Methodology of algorithm engineering, arXiv (2023). [29] P. Swanborn, A common base for quality control criteria in quantitative and qualitative research,

Quality and quantity 30 (1996) 19–35. [30] U. Frank, Towards a pluralistic conception of research methods in information systems research,

ICB-Research Report 7, Universität Duisburg-Essen, 2006. [31] W. van der Aalst, Process Mining: Data Science in Action, Springer, Berlin Heidelberg, 2016. [32] H. Schonenberg, R. Mans, N. Russell, N. Mulyar, W. van der Aalst, Process flexibility: A survey of contemporary approaches, in: International Workshop on Cooperation and Interoperability, Architecture and Ontology, Springer, 2008, pp. 16–30. [33] A. Kraus, J.-R. Rehse, H. van der Aa, Data-driven assessment of business process resilience, Process

Science 1 (2024) 4. [34] N. Stricker, G. Lanza, The concept of robustness in production systems and its correlation to disturbances, Procedia CIRP 19 (2014) 87–92. [35] L. Becker, J. Sturm, F. Eiden, D. Holtmann, Analyzing and understanding the robustness of bioprocesses, Trends in Biotechnology 41 (2023) 1013–1026.

References

[1] M. Dumas , M. Rosa , J. Mendling , H. Reijers , Fundamentals of business process management, Springer, 2018 .

[2] W. Van der Aalst , J. Nakatumba , A. Rozinat , N. Russell , Business process simulation, Handbook on Business Process Management 1 : Introduction , Methods, and Information Systems ( 2010 ) 313 - 338 .

[3] J. Banks , Handbook of simulation: principles, methodology, advances, applications, and practice, John Wiley & Sons, 1998 .

[4] M. Camargo , M. Dumas , O. González-Rojas , Automated discovery of business process simulation models from event logs , Decision Support Systems 134 ( 2020 ) 113284 .

[5] M. Camargo , D. Báron , M. Dumas , O. González-Rojas , Learning business process simulation models: a hybrid process mining and deep learning approach , Information Systems 117 ( 2023 ) 102248 .

[6] F. Meneghello , C. Di Francescomarino , C. Ghidini , Runtime integration of machine learning and simulation for business processes , in: ICPM, IEEE, 2023 , pp. 9 - 16 .

[7] O. López-Pintado , M. Dumas , J. Berx , Discovery, simulation, and optimization of business processes with diferentiated resources , Information Systems 120 ( 2024 ) 102289 .

[8] O. López-Pintado , M. Dumas , Business process simulation: Probabilistic modeling of intermittent resource availability and multitasking behavior , Information Systems 127 ( 2025 ) 102471 .

[9] L. Kirchdorfer , R. Blümel , T. Kampik , H. van der Aa, H. Stuckenschmidt, Discovering multi-agent systems for resource-centric business process simulation , Process Science 2 ( 2025 ) 4 .

[10] T. Kampik , C. Warmuth , A. Rebmann , R. Agam , L. N. Egger , A. Gerber , J. Hofart , J. Kolk , P. Herzig , G. Decker , et al., Large process models: a vision for business process management in the age of generative ai , KI-Künstliche Intelligenz ( 2024 ) 1 - 15 .

[11] R. G. Sargent , Verification and validation of simulation models , Journal of simulation 7 ( 2013 ) 12 - 24 .

[12] D. Chapela-Campa , I. Benchekroun , O. Baron , M. Dumas , D. Krass , A. Senderovich , A framework for measuring the quality of business process simulation models , Information Systems 127 ( 2025 ) 102447 .

[13] K. Özdemir , L. Kirchdorfer , K. Amiri Elyasi , H. van der Aa, H. Stuckenschmidt, Rethinking business process simulation: A utility-based evaluation framework , in: BPM Forum , Springer, 2025 .

[14] D. Barón-Espitia , M. Dumas , O. G. Rojas , Coral: Conversational what-if process analysis , in: ICPM Doctoral Consortium/Demo , 2022 , pp. 118 - 122 .

[15] O. López-Pintado , J. Rosenbaum , J. Berx , M. Dumas , Optimos: A tool for simulation-driven business process optimization ( 2024 ).

[16] M. Camargo , M. Dumas , O. González-Rojas , Learning accurate LSTM models of business processes , in: BPM , Springer, 2019 , pp. 286 - 302 .

[17] M. Camargo , M. Dumas , O. González-Rojas , Discovering generative models from event logs: data-driven simulation vs deep learning , PeerJ Computer Science 7 ( 2021 ) e577 .

[18] M. Dumas , F. Fournier , L. Limonad , A. Marrella , M. Montali , J.-R. Rehse , R. Accorsi , D. Calvanese , G. De Giacomo, D. Fahland , et al., Ai-augmented business process management systems: a research manifesto , ACM Transactions on Management Information Systems 14 ( 2023 ) 1 - 19 .

[19] D. Chapela-Campa , M. Dumas , From process mining to augmented process execution , Software and Systems Modeling 22 ( 2023 ) 1977 - 1986 .

[20] A. Tour , A. Polyvyanyy , A. Kalenkova , Agent system mining: Vision, benefits, and challenges , IEEE Access 9 ( 2021 ) 99480 - 99494 .

[21] J. Bejarano , D. Barón , O. González-Rojas , M. Camargo , Discovering optimal resource allocations for what-if scenarios using data-driven simulation , Frontiers in Computer Science 5 ( 2023 ) 1279800 .

[22] O. López-Pintado , M. Dumas , M. Yerokhin , F. M. Maggi , Silhouetting the cost-time front: Multiobjective resource optimization in business processes , in: BPM Forum , Springer, 2021 , pp. 92 - 108 .

[23] W. Ding , M. Abdel-Basset , H. Hawash , A. M. Ali , Explainability of artificial intelligence methods, applications and challenges: A comprehensive survey , Information Sciences 615 ( 2022 ) 238 - 292 .