Process prediction enables the forecast of a process instance based on event log data from processoriented information systems. Recently, deep learning methods have improved the state-of-the-art in various process prediction tasks. Currently, most research addresses the control flow or time perspective. However, critical information is often hidden in context variables and represented in various data types such as tabular data, image data, and sensor data. This work explores the influence of context variables on process prediction. The main objective is to develop new deep learning-based methods to increase the eficiency and applicability of multimodal process prediction.
Process prediction, also referred to as predictive business process monitoring or predictive
process analytics is a subset of process mining that tries to gain predictive insights into the
future of ongoing process instances. Organizations can leverage these predictions to adjust
ongoing process execution in real time to prevent undesirable outcomes [
In recent years, deep learning-based approaches to process prediction have attained popularity
and led to breakthrough results in various tasks such as next step prediction, remaining time,
and outcome prediction [
Existing work mainly focuses on common event log attributes, especially activities, resources, or timestamps. In practice, however, a few event log attributes rarely determine the future of a process instance alone. Other context variables represented in various data types might also be relevant. These may be tabular data directly associated with the event log, including event attributes such as resource or timestamp or case attributes such as customer or product. In addition, context variables can contain unstructured data such as text, images, and sensor data. These contextual variables are often critical, as they can directly influence the continuation of the process and thus significantly improve the prediction quality.
While deep learning techniques have been successfully applied to structured and unstructured data, research on process prediction has mainly involved structured information such as activities, resources, or timestamps. To address this research gap, this thesis investigates how context variables of various data types can be processed and combined for process prediction. Designing and implementing deep learning-based methods for multimodal process prediction is the primary goal of this thesis. In particular, the following three research questions shall be addressed.
RQ1. What are the strengths and weaknesses of existing deep learning-based process prediction architectures regarding data processing, modeling, and evaluation? RQ2. How can diferent contextual variables be efectively incorporated into multimodal process prediction? RQ3. How can process prediction enable other predictive process analytics such as trace clustering or anomaly detection?
RQ1 is an empirical knowledge question that involves identifying existing approaches for process prediction and determining their capabilities, limitations, and tradeofs. In particular, it examines whether there is a superior method or whether specific methods are preferable depending on situational circumstances.
In contrast, RQ2 aims to provide a deeper understanding of the diferent use cases in which process prediction is used in practice. For this purpose, several use cases from diferent application domains with diferent context variables should be examined. Furthermore, the context variables that have the most significant efect on the continuation of the process should be identified. Based on these findings, efective multimodal prediction methods should be developed. These methods should support relevant contextual variables and build on sound design decisions that have been benchmarked against alternative approaches in RQ1.
Finally, RQ3 investigates the potential of neural networks for other predictive process monitoring tasks. In natural language processing, language models have been exploited for solving complex downstream tasks like machine translation, summarization, or question answering. Similarly, with this research question, we want to explore whether neural networks originally trained for process prediction can be advantageously leveraged for anomaly detection, trace clustering, etc.
The research project follows the design science research (DSR) paradigm for Information Systems (IS) research [6]. Hevner et al. propose seven guidelines (G1-G7) to address a research problem in the IS domain.
G1. Design as an Artifact. The result of the research project should be one or more novel artifacts.
G2. Problem Relevance. The addressed problem should be relevant to the IS research area. G3. Design Evaluation. The artifacts should be evaluated through an appropriate method to prove their eficiency and utility. G4. Research Contribution. The artifacts should either solve an unsolved problem or improve on the existing solutions for the problem.
G5. Research Rigor. The artifacts should be defined through formal definitions, pseudo code, or source code in order to ensure consistency and reproducibility.
G6. Design as a Search Process. The artifacts should be designed through an iterative search process. The process consists of defining problem space and solution criteria, designing the solution, and verifying the solution based on the criteria.
G7. Communication of Research. The results and outcomes of the research project should be communicated to academics and practitioners.
We plan to implement the above guidelines as follows. To focus on relevant research problems, we performed a structured literature review (SLR) in order to screen existing deep learningbased process prediction methods and reveal trade-ofs and limitations ( G2). Through the SLR, we identified two major research problems: (i) a lack of a documented and reproducible benchmark for process prediction methods; (ii) a focus on a selection of event log variables leading to a neglect of relevant context information. Accordingly, we set to develop the following novel artifacts (G1, G4): (i) a modular framework for benchmarking novel prediction methods against existing methods. (ii) three new deep learning-based process prediction methods that can efectively incorporate tabular data, image data, and sensor data for multimodal process prediction. We show how these developed predictive methods can be utilized to solve a variety of predictive process monitoring tasks, including next-step prediction, outcome prediction, remaining-time prediction, trace clustering, and anomaly detection. Each artifact is designed formally by justifying all design decisions and reporting pseudo-code or source code (G5). Additionally, we describe the experimental setup of each artifact in detail and release the conducted experiments whenever possible. All artifacts are assessed by empirical evaluations on real-live event logs or synthetic datasets (G3). Finally, all artifacts that we introduce in this thesis are developed through an iterative design flow ( G6), while major milestones are presented at conferences and published in journals (G7).
To address RQ1, we performed a structured literature review concerning the state-of-the-art of deep learning-based process prediction [7]. The review compares 32 process prediction approaches that are classified along diferent carefully selected dimensions such as neural network architecture, prediction target, input features, and data processing methods. In particular, the review showed the benefits and drawbacks of the existing approaches. It revealed research gaps that played a significant role in determining the research direction of this dissertation project.
Most of the other author’s contributions have addressed RQ2 (e.g. [8, 9, 10]). In [11], we covered a use case from the steel industry where sensor data are processed through an LSTM autoencoder to detect the quality of semi-finished products. Furthermore, we used the learned representations of the autoencoder to predict the next step in the production process. In contrast to most existing prediction approaches, we consider sensor data as an additional input source next to typical event log data.
In [12], we proposed a novel prediction method that can process a flexible number of inputs while supporting categorical and continuous variables by using gramian angular fields and convolutional neural networks. The approach can be adapted through configuration to handle diferent prediction tasks, including next step prediction, next resource prediction, remaining time prediction, and outcome prediction. Its efectiveness is measured by comparing the prediction quality with existing approaches on publicly available datasets. The results show that the proposed method is an efective alternative to the more frequently used LSTM-based approaches.
In [13], we investigated how deep learning can be applied to computer-aided designs (CAD) to support the planning process in a ”one-of-a-kind production” scenario of a manufacturing ifrm. To this end, we utilized convolutional neural networks to cluster process instances based on their CAD images and connected them with event log data to gather predictive insights.
Only initial research has been conducted concerning RQ3. In [12], we showed that the hidden representations of process prediction models could also be utilized for process trace clustering. Additionally, in [9], we examined the potential of anomaly detection through process prediction for a purchase order handling process. To this end, we investigated the efectiveness of an unsupervised language model pretraining with a target task classifier finetuning applied to process prediction and anomaly detection.
Our other studies examined the strengths and weaknesses of machine learning methods in various use cases [14, 15].
Future work comprises activities to analyze the integration of other context attributes of additional data types, such as text data. In addition, thorough benchmarking of design decisions in process prediction research will be conducted, with a focus on comparability and transparency. Finally, further research is needed to deepen the understanding of how an underlying predictive model afects the quality of downstream tasks. To this end, we plan to further explore the extent of a predictive model’s impact on anomaly detection. Intell. Lect. Notes Bioinformatics), volume 10253 LNCS, Springer Verlag, 2017, pp. 477–492. doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 3 1 9 - 5 9 5 3 6 - 8 _ 3 0 . [5] I. Verenich, M. Dumas, M. L. Rosa, F. M. Maggi, I. Teinemaa, Survey and cross-benchmark comparison of remaining time prediction methods in business process monitoring, ACM Transactions on Intelligent Systems and Technology (TIST) 10 (2019) 1–34. [6] A. R. Hevner, S. T. March, J. Park, S. Ram, Design science in information systems research,
MIS quarterly (2004) 75–105. [7] D. A. Neu, J. Lahann, P. Fettke, A systematic literature review on state-of-the-art deep learning methods for process prediction, Artificial Intelligence Review (2021) 1–27. [8] S. Dadashnia, P. Fettke, P. Hake, J. Lahann, P. Loos, S. Klein, N. Mehdiyev, T. Niesen, J.-R.
Rehse, M. Zapp, Exploring the potentials of artificial intelligence techniques for business process analysis (2017). [9] O. Gutermuth, J. Lahann, J.-R. Rehse, M. Scheid, S. Schuhmann, S. Stephan, P. Fettke, Eficient and compliant purchase order handling - a contribution to bpi challenge 2019 (2019). [10] S. Klein, J. Lahann, L. Mayer, D. Neu, P. Pfeifer, A. Rebmann, M. Scheid, B. Willems, P. Fettke, Business process intelligence challenge 2020: Analysis and evaluation of a travel process, in: 10th Business Process Intelligence Challenge at the Int. Conf. on Process Mining (ICPM), 2020. [11] N. Mehdiyev, J. Lahann, A. Emrich, D. Enke, P. Fettke, P. Loos, Time series classification using deep learning for process planning: A case from the process industry, Procedia Computer Science 114 (2017) 242–249. [12] P. Pfeifer, J. Lahann, P. Fettke, Multivariate business process representation learning utilizing gramian angular fields and convolutional neural networks, in: International Conference on Business Process Management, Springer, Cham, 2021, pp. 327–344. [13] N. Mehdiyev, L. Mayer, J. Lahann, P. Fettke, Deep learning-based clustering of processes and their visual exploration: An industry 4.0 use case for small, medium-sized enterprises, Expert Systems n/a (????) e13139. URL: https:// onlinelibrary.wiley.com/doi/abs/10.1111/exsy.13139. doi:h t t p s : / / d o i . o r g / 1 0 . 1 1 1 1 / e x s y . 1 3 1 3 9 . a r X i v : h t t p s : / / o n l i n e l i b r a r y . w i l e y . c o m / d o i / p d f / 1 0 . 1 1 1 1 / e x s y . 1 3 1 3 9 . [14] J. Lahann, M. Scheid, P. Fettke, Utilizing machine learning techniques to reveal vat compliance violations in accounting data, in: 2019 IEEE 21st conference on business informatics (CBI), volume 1, IEEE, 2019, pp. 1–10. [15] J. Lahann, M. Scheid, P. Fettke, Towards optimal free trade agreement utilization through deep learning techniques (2020).