Neural Networks for Explainable Next-Activity Prediction

Abstract

The prediction of the next-activity in the ongoing execution (event trace) of a business process provides useful information to take smart decisions for mitigating ineficiencies, preventing undesired activities and boosting business values of enterprises. In this paper, we describe PROPHET: a deep learning approach based on heterogeneous graph neural networks, to strike a balance between accurate predictions and interpretability in the next-activity prediction task. Specifically, it represents event traces as heterogeneous graphs by using diferent node types to express diferent characteristics of several events, and diferent types of graph links to express relationships between event characteristics, as well as relationships between events. It uses heterogeneous Graph Attention Networks (GATs), to achieve good accuracy in next-activity predictions. Finally, it integrates a modified version of the GNN Explainer algorithm to disclose explainable information related to characteristics, events and relationships between events that mainly influenced the prediction. Experiments with various benchmark event logs show the accuracy of PROPHET compared to several current state-of-the-art methods and draw insights from explanations recovered through the GNN Explainer algorithm.

1. Introduction

Predictive Process Monitoring (PPM) is a family of predictive approaches to predict the unfolding of ongoing executions (event traces) of a business process based on knowledge learned from historical event logs. In particular, predicting the next-activity in an ongoing event trace is a crucial task in PPM as such predictive information may allow analysts to intervene proactively and prevent undesired behaviors. Notably, with the recent boom of deep learning, the PPM literature has achieved amazing results using diferent deep learning architectures, e.g., Long Short-Term Neural Network (LSTMs) [ 1, 2, 3 ], Convolutional Neural Networks (CNNs) [ 4, 5 ], Vision Transformers (ViTs) [ 6 ] and Large Language Models (LLMs) [ 7 ], to yield accurate predictions of the next-activity in ongoing event traces. On the other hand, multi-view learning has recently emerged as a fundamental deep learning direction for PPM approaches. In particular, in [ 1, 6 ], the multi-view learning strategy has allowed the development of lfexible deep neural models that gain accuracy in next-activity prediction problems thanks to their ability to leverage the diversity of the information in the multiple characteristics of an event. Nevertheless, deep neural models generally learn opaque models, while easier-to-explain predictive models are becoming increasingly desirable in PPM applications [ 8, 2, 6 ]. Finally, in the last decade, graph representation learning has recently gained prominence across various domains, attracting intense research attention in deep learning [ 9 ]. Accordingly, a few recent PPM studies [ 10, 11, 12, 13 ] have explored the performance of various Graph Neural Network (GNN) architectures for graphs trained for PPM problems. However,

CEUR Workshop

ISSN1613-0073 previous PPM studies consider homogeneous graphs to mainly represent the activity control-flow and eventually use external data synopsis to encode the multi-view information associated with event nodes. In addition, the priority of previous graph-based PPM studies was mainly to provide accurate predictions of future states of ongoing event traces without paying extensive attention to explain predictions achieved with graph representations of event traces.

Based on the premises reported above, we describe a PPM graph-based approach, named PROPHET (explainable Predictive heteROgeneous graPH nEural neTworks), that has been recently presented in [ 14 ]. It trains a heterogeneous Graph Neural Network (GNN) model to predict the executing activity of the next event in an ongoing event trace and explains which characteristics of the ongoing trace mainly contribute to the decision. Specifically, we introduce the design of an event data engineering scheme to map ongoing event traces recorded in an historical event log into heterogeneous graphs by embracing the multi-view representation of event data. We illustrate a heterogeneous graph learning approach that integrates the attention mechanism implemented in the GAT layer [ 15 ] into a heterogeneous GNN. We describe an extension of the heterogeneous GNN Explainer algorithm [ 16 ], that is used to disclose how specific event and trace characteristics, as well as event relationships, influence the local decisions of the proposed predictive model. Finally, we illustrate the main results of an extensive experimentation, which show the capability of the presented approach to achieve accuracy comparable to related deep multi-view learning and graph learning approaches presented in the recent PPM literature, as well as disclose useful explanations of the model behavior in terms of the most informative inputs.

The paper is organized as follows. Section 2 reports the basics of the proposed approach. Section 3 describes the training stage of PROPHET, while Section 4 illustrate the experimental study to evaluate the accuracy and explainability performance of PROPHET. Finally, Section 5 draws conclusions.

2. Basics

An event trace is a record of a business process that shows the stages of its execution through a sequence of events. An event is portrayed from multiple views, where a view describes each event based on a specific characteristic. Specifically, each event is represented in two mandatory views associated with the activity and the timestamp (indicating when the activity occurred), respectively, and in supplementary views, with = 1, … , associated with optional event characteristics (e.g., resource responsible for the activity or the cost involved in completing it). In addition, an event trace may be associated with supplementary global characteristics, ℎ with ℎ = 1, … , , which describe general trace information (e.g., the age of a patient at the time of her hospitalization). Let be the set of activity names, be the set of trace identifiers, be the set of timestamps, with 1 ≤ ≤ be the set of values in the -th characteristic of an event, while ℎ 1 ≤ ℎ ≤ be the set of values in the ℎ-th global characteristic of an event trace. For simplicity, let us denote C = [ 1, … , ] – the vector of global trace characteristics , = 1×, … , × – the domain of C, V = [, , 1, … , ] – the vector of local characteristics and = × × 1 × … × – the domain of V.

Definition 1 (Event). Given the event universe ℰ = × , an event ∈ ℰ is a tuple = (, , , 1, … , ) that represents the occurrence of activity in trace at timestamp with characteristics 1, … , . Let us introduce the functions ∶ ℰ ↦ such that () = , ∶ ℰ ↦ such that () = , ∶ ℰ ↦ such that () = , ∶ ℰ ↦ such that () = and = 1, … , . Definition 2 (Trace). Let ℰ ∗ denote the set of all possible sequences on ℰ. A trace is a pair ( , c) ∈ ℰ ∗ × where: 1. = ⟨ 1, … , ⟩ ∈ ℰ ∗ is an event sequence so that: (a) ∀ = 1, … , , ∃ ∈ ℰ such that ( , ) = and ( ) = , and (b) ∀ = 1, … , − 1, ( ) ≤ ( +1 ). 2. c = ( 1, … , ) ∈ is the vector of the values of the optional global trace characteristics.

Let us introduce the function ∶ ℰ ∗ × ℕ ↦ ℰ such that ( , ) = .

Definition 3 (Event log).

Given a business process , an event log ℒ ∈ ℬ(ℰ historical event traces recoded for a specific business process. ∗ × ) is a collection of

Definition 4 (Prefix event sequence).

Given a sequence ∈ ℰ ∗, the prefix event sequence = ⟨ 1, … , ⟩ is the event sub-sequence of starting from the beginning of , with 1 ≤ = | | < | | . in , i.e., ( ) = ( +1 ) with +1 = ( , + 1) .

Notice that the activity ( +1 ) = +1 corresponds to the next-activity of the prefix event sequence

Definition 5 (Prefix trace).

Given the trace ( , c) ∈ ℰ ∗ × , its prefix trace with length is the pair ( , c) where is the the prefix event sub-sequence of with length equal to , while c is the vector of global characteristic values recorded in the trace.

A graph is a smart data structure that allows the representation of entity characteristics and entity relationships. In this study, we introduce a graph encoding strategy to transform a prefix trace into a heterogeneous graph.

Definition 6 (Graph).

Let denote the set of all graphs. A graph ∈ is an ordered pair ( , ℒ ), where is the set of nodes, and ℒ is the set of weighted links between nodes, i.e., ℒ ⊆ × × ℝ . Each node ∈

is associated with a node type and records a value. Each link ( 1, 2, ) ∈ ℒ is a link between nodes 1 ∈ and 2 ∈ associated with a numerical value ∈ ℝ , called weight.

Let be the set of distinct node types appearing in a graph . In the heterogeneous graph considered in this study | | > 1. In this study, a prefix trace ( , c) is represented as an heterogeneous graph that reserves a distinct type of nodes for each distinct type of characteristics. Hence, values of local event characteristics and global trace characteristics are recorded into 3 + distinct types of nodes (i.e., | | = 3 + ). In fact, a type of nodes is associated with the vector of global trace characteristics, a type of nodes is associated with the activity characteristic, a type of nodes is associated with the timestamp characteristic, and a type of nodes is associated with each one of the optional event characteristics recorded in the event sequence of the prefix trace. Regarding links, includes three types of links: (1) links pertaining intra-view relationships on the same local event characteristic observed in diferent events (i.e., links connecting two nodes belonging to the same node type); (2) links pertaining inter-view relationships involving two diferent local event characteristics measured in the same event (i.e., links connecting two nodes belonging to diferent node types); (3) links that establish a relationship between the vector of the global trace characteristics and each specific local event characteristic and (i.e., links connecting the node representing the global trace characteristics to any other node). Figure 1 shows an example of the transformation of a prefix trace into a heterogeneous graph.

Let ℒ ∈ ℬ(ℰ ∗ × ) be an event log, ⊆ ℬ(ℰ ∗ × × ) be the multiset of all heterogeneous graphs that represent prefix traces extracted from traces recorded in ℒ and labeled with the nextactivity values recorded in the corresponding traces. Let us consider a function ∶ ↦ , such that ()

predicts the expected next-activity +1 of the graph . Based on these premises, we frame the next-activity prediction task as a multi-class, heterogeneous graph classification problem. According to this formulation can be trained by resorting to a graph-based learning technique from the multiset of the labeled heterogeneous graphs representing the prefix traces extracted from the event traces recorded in ℒ

and labeled with the next-activity in the corresponding traces.

3. The PROPHET approach

The PROPHET approach takes an event log LOG as input to learn the next-activity predictive function . The learning stage is three-stepped.

(a) Prefix trace (b) Heterogenous graph

In Step 1 PROPHET takes ℒ as input and generates the multi-set of labeled prefix traces as output.

To this purpose, it first transforms timestamp information recorded in an event into the time in seconds passed from the beginning of the trace. Then, it converts all numerical characteristics into categorical by applying the equal-frequency discretization algorithm as in [ 6 ]. The number of discretization bins of a characteristic is set equal to the average number of distinct categories in the original categorical characteristics recorded in ℒ . After this step, ℒ contains all characteristics in the categorical format. Subsequently, it uses the Continuous-Bag-of-Words (CBOW) architecture of the Word2Vec scheme [ 17 ] to transform each categorical characteristic of a prefix trace extracted from ℒ into a numeric representation. CBOW is an embedding approach that was originally formulated in the field of natural language processing and as recently used in several PPM approaches (e.g., [ 3, 6 ]). It uses a feed-forward neural network to predict a target value from the neighbored context. In this work, a distinct CBOW architecture is trained for each characteristic recorded in the prefix traces.

In Step 2 PROPHET converts each labeled prefix trace ( , c, +1 ) into a labeled heterogeneous graph (, +1 ). For each event ∈ , every event characteristic recorded in is mapped into a graph node that contains the embedding representation of the characteristic value The embedding is the one computed with the corresponding CBOW model. The type of the node is that one-to-one associated with the mapped characteristic. At the same time, the embeddings of the values the global trace characteristics collected in c are recorded into the trace-based node of the graph. Once the nodes of have been generated, the links between the created nodes are produced. Regarding inter-view links (which pertain nodes of diferent types), in this work, we generate the inter-view links considering the graph nodes whose types are associated with each pair of consecutive views in the vector V that represent the event characteristics recorded in ℒ . Let be the the multi-set of labeled heterogeneous graphs extracted from ℒ .

In Step 3 PROPHET learns a GNN model equipped with of GAT layers [ 15 ] to obtain a next-activity prediction function from . The GAT learns iteratively new node representations, diferentiating the learning process concerning the link types that are represented in the graph, and aggregating information coming from each neighborhood.

Finally, the GAT model is equipped with a variant of GNN Explainer [ 16 ] that allows us to explain the decisions of the GAT model by extracting insights on how each node and link contribute to a decision. The basic implementation of the GNN Explainer algorithm works as follows. Given a heterogeneous graph = ( , ℒ ) , the prediction =̃ () can be equally written as =̃ ( X, A), where X and A are the list of data feature matrices and the list of adjacency matrices that are, respectively, associated with . The basic GNN Explainer algorithm explains this decision by jointly computing the following outputs: a node feature mask X̂ = [X̂tn ∈ [ 0, 1 ]1× , ∀ ∈ ] and a link mask  = [Âtl ∈ [ 0, 1 ] ( , ), ∀ ∈ ℒ]. The algorithm attempts to perturb the input graph , composing a new “masked” graph =̂ ( , ̂ ℒ ̂) so that X = [Xtn = Xtn ⊙ X̂tn, ∀ ∈ ] is the list of data feature matrices associated with ̂. The higher the value of X̂tn[] , the higher the importance of the corresponding -th piece of information on the prediction. Notably, X̂tn[] is a unique value estimated within for the nodes of with type . On the other hand, the higher the value of ̂ [, ] , the more important the corresponding link. So, the main issue of this basic algorithm is that it cannot identify the most important node subset | |×| | , = in the graph. For this reason, we adapted GNNExplainer to estimate the most important subgraph for a given prediction. Specifically, we modified the shape of parameter the rest of the algorithm unchanged comprising the element-wise multiplications to obtain the masked graph ̂. In this way, the higher the value estimated for X̂tn[] the more important the efect of the X̂tn ∈ [ 0, 1 ]| |×1, ∀ ∈ keeping node on the decision.

4. Experimental study

and explainability performance of PROPHET.

4.1. Event logs and experimental set-up

In this section, we describe some results of the experimentation conducted for evaluating the accuracy Event log description: domain (F– Finance, M – Maintenance, A – Administration, C – Customer Service, H – Healthcare) of the business process, number of traces (♯Trace), number of events (♯Event), number of activities (♯Activities), mean length of traces (Avg length), number of global trace characteristics (# Trace characteristics) and number of local event characteristics (# Event characteristics)

Domain #Trace #Events #Activities Avg length #Trace characteristics #Event characteristics F F M F A C F H C 13087 9658 819 42995 6886 3804 5123 1897 23906 60849 72413 2351 193849 36796 13710 62740 6973 178078 10 6 5 8 19 9 25 10 13 5 7 3 5 5 4 4 7 12 1 1 3 5 3 0 5 4 3 3 3 6 4 4 2 3 3 2

We used nine real-life event logs that are all available on the 4TU Centre for Research 1, except for SP2020 that is available on Zenodo2. A summary of the characteristics of the event logs considered in the evaluation is reported in Table 1. For each event log, a temporal split was done to divide the log into training and testing traces. Specifically, we sorted the event traces of each event log by their starting timestamps. The first two-thirds of the sorted traces were chosen for training the predictive model, while the remaining one-third was reserved for evaluating the model’s performance on unseen data. The implementation of PROPHET is available online. 3

4.2. Accuracy performance analysis

We evaluated the accuracy performance of both PROPHET and several state-of-the art deep neural methods formulated in the PPM field for the next-activity prediction task. The related methods considered in the evaluation study train: a LSTM model in [ 1 ], a BiLSTM model in [ 18 ], a Transformer model in [ 19 ], a ViT model [ 6 ], a CNN with Inception model in [ 4 ], a GCN model in [ 12 ] and GRNN model concatenated with a LSTM model in [ 13 ]. The methods described in [ 1 ], [ 18 ] and [ 19 ] adopt a sequence representation of prefix traces. The methods described in [ 6 ] and [ 4 ] adopt a color image representation of prefix traces. The method described in [ 12 ] adopts homogeneous instance graphs to represent prefix traces, while the method described in [ 13 ] handles homogeneous graphs extracted by replaying prefix traces on a Petri Net discovered from the event log and event-related temporal data extracted from the remaining views recorded in the event log. All related methods, except for [ 18 ] and [ 19 ], were originally formulated within the multi-view learning schema. However, to provide a fair comparison, we ran all related methods by accounting for all views recorded in the considered event logs. For each event log, for each PPM method, we measured the accuracy of each next-activity predictive function through the macro FScore. computed as = , where is the number of distinct activities recorded in the event log. This multi-class accuracy metric was selected as it is commonly measured in imbalanced domains. Figure 2 shows the macro FScore measured for both PROPHET and the related methods. These results show that PROPHET commonly outperforms the related methods having [ 13 ] as runner-up. In particular, the method described in [ 13 ] performs better than PROPHET in BPIC20R only, by training a GRNN model followed by an LSTM model. In any case, PROPHET is the runner-up of the comparative analysis in this event log also. =1 1 ∑

4.3. Explanation performance analysis

In this section, we illustrate some examples of the decision explanations produced by PROPHET. Figures 3a and 3b show the heterogeneous graphs of two prefix traces recorded in the testing set of the event log (a) ”ME” (b) ”CS”

Invoice and enriched with the explanations obtained with the extension of GNN Explainer algorithm. PROPHET correctly predicts ”ME” – ”Manual enter the order number-Entered” and ”CS” – ”Compare of Sums missing” as the next activity of the two ongoing traces, respectively. Both prefix traces share the same ”SM - PSM - SC - CO” activity prefix, where: ”SM” – ”Start Missing”, ”PSM” – ”Process Start Missing”, ”SC” – ”Status Change to being approved missing” and ”CO” – ”Check Order number missing”. However, the two prefix traces represent ongoing executions of the Invoice business process, which will proceed with executing two diferent activities in the future. Therefore, this example shows a case where the control flow of activities does not provide enough information to correctly predict the two diferent next activities for the two prefix traces. Instead, PROPHET is able to yield the correct next-activity decision for both prefix traces by leveraging the multi-view information recorded in the event log. In addition, the explanation of the decisions highlights which characteristics already recorded for the ongoing trace helps PROPHET model to disentangle the next-activity ”ME” from the next-activity ”CS” in the considered prefix traces. In particular, the produced explanations show that, although the activities executed in the two prefix traces are all performed by the resource ”S” (Server), some diferences occur in their time cycle by highlighting which information of each time cycle is actually relevant for predicting the correct next activity in the two cases. Notably, activities ”SM” and ”PSM” have been executed before 0.5 seconds passed from the beginning of the trace. However, in Figure 3a, activities ”SC” and ”CO” have been executed in the time interval that goes from 0.5 to 1.5 seconds passed from the beginning of the trace with the timestamp of ”CO” that has high relevance (greater than 0.5) on the correct decision ”ME” as the next-activity of the prefix trace. In Figure 3b, activities ”SC” and ”CO” have been executed in the time intervals that go from 1.5 to 2.5 seconds and 2.5 to 4.5 seconds passed, respectively, from the beginning of the trace. Notably, the timestamps of ”PSM”, ”SC” and ”CO” have all high relevance (greater than 0.5) on the correct decision ”CS” as the next activity of the prefix trace. Hence, the decision is explained as follows: the longer the time passed from the beginning of the ongoing trace before the execution of activities ”SC” and ”CO”, the more important the symptoms indicating ”CS” as the upcoming activity of the ongoing trace. This information can be used at the process management level to plan the work requested to better manage the execution, as well as the consequences of the execution of activity ”CS” on the current process execution.

5. Conclusion

In this work, we present a PPM approach that represents ongoing traces of a business process as heterogeneous graph and learns GAT model for the next-activity prediction task. In addition, it resorts to an extended version of the GNN Explainer algorithm to equip GAT’s decisions with explanations regarding trace characteristics mainly afecting the decision An extensive experimentation shows the accuracy of the proposed approach also compared to several related PPM methods. In addition, it provides some examples of the produced explanations. As future work, we plan to explore how concept drift detection may be integrated in the proposed graph-based approach, in order to keep the model accurate over time. In addition, we intend to start the investigation of how next-activity decision explanations disclosed with the proposed approach may be leveraged in a prescriptive PPM perspective.

Declaration on Generative AI

The authors have not employed any Generative AI tools.

Acknowledgments

Vincenzo Pasquadibisceglie and Giovanna Castellano are partially supported by the project FAIR Future AI Research (PE00000013), Spoke 6 - Symbiotic AI, under the NRRP MUR program funded by the NextGenerationEU. Rafaele Scaringi is funded by a Ph.D. fellowship within the framework of the Italian “D.M. n. 352, April 9, 2022” - under the NRRP, co-supported by “Exprivia S.p.A.” (CUP H91I22000410007). The work of Donato Malerba is in partial fulfillment of the project MAD - The Additive Metamorphosis of Design (ARS01_00717) funded by MUR.

References

[1] V. Pasquadibisceglie , A. Appice , G. Castellano , D. Malerba , A multi-view deep learning approach for predictive business process monitoring , IEEE Trans. Serv. Comput . 15 ( 2022 ) 2382 - 2395 . doi: 10 .1109/TSC. 2021 . 3051771 .

[2] L. Aversano , M. L. Bernardi , M. Cimitile , M. Iammarino , C. Verdone , A data-aware explainable deep learning approach for next activity prediction , Eng. App. of Artif. Intell . 126 ( 2023 ) 106758 . doi:https://doi.org/10.1016/j.engappai. 2023 . 106758 .

[3] V. Pasquadibisceglie , A. Appice , G. Castellano , D. Malerba , DARWIN : An online deep learning approach to handle concept drifts in predictive process monitoring , Eng. Appl. Artif. Intell . 123 ( 2023 ) 106461 . doi: 10 .1016/J.ENGAPPAI. 2023 . 106461 .

[4] V. Pasquadibisceglie , A. Appice , G. Castellano , D. Malerba , Predictive process mining meets computer vision , in: BPM Forum 2020 , volume 392 of LNBIP , Springer, 2020 , pp. 176 - 192 .

[5] H. Weytjens , J. De Weerdt, Process outcome prediction: Cnn vs. lstm (with attention) , in: BPM 2020 International Workshops , Springer, 2020 , pp. 321 - 333 .

[6] V. Pasquadibisceglie , A. Appice , G. Castellano , D. Malerba , JARVIS: Joining adversarial training with vision transformers in next-activity prediction , IEEE Trans. Serv. Comput . ( 2023 ) 1 - 14 . doi: 10 .1109/TSC. 2023 . 3331020 .

[7] V. Pasquadibisceglie , A. Appice , D. Malerba , LUPIN: A LLM approach for activity sufix prediction in business process event logs , in: 6th International Conference on Process Mining, ICPM 2024 , IEEE, 2024 , pp. 1 - 8 . doi: 10 .1109/ICPM63005. 2024 . 10680620 .

[8] R. Galanti , M. de Leoni, M. Monaro , N. Navarin , A. Marazzi , B. Di Stasi , S. Maldera , An explainable decision support system for predictive process analytics , Eng. App. of Artif. Intell . 120 ( 2023 ) 105904 . doi:https://doi.org/10.1016/j.engappai. 2023 . 105904 .

[9] M. Li , A. Micheli , Y. G. Wang , S. Pan , P. Lió , G. S. Gnecco , M. Sanguineti , Guest editorial: Deep neural networks for graphs: Theory, models, algorithms, and applications , IEEE Trans. Neural Networks Learn. Syst . 35 ( 2024 ) 4367 - 4372 . doi: 10 .1109/TNNLS. 2024 . 3371592 .

[10] S. Weinzierl , Exploring gated graph sequence neural networks for predicting next process activities , in: BPM 2022 International Workshops, Springer, 2022 , pp. 30 - 42 .

[11] I. Venugopal , J. Töllich , M. Fairbank , A. Scherp , A comparison of deep-learning methods for analysing and predicting business processes , in: IJCNN 2021 , IEEE, 2021 , pp. 1 - 8 . doi: 10 .1109/ IJCNN52387. 2021 . 9533742 .

[12] A. Chiorrini , C. Diamantini , L. Genga , D. Potena, Multi-perspective enriched instance graphs for next activity prediction through graph neural network , J. Intell. Inf. Syst . ( 2023 ) 1 - 21 .

[13] E. Rama-Maneiro , J. C. Vidal , M. Lama , Embedding graph convolutional networks in recurrent neural networks for predictive monitoring , IEEE Trans. Knowl. Data Eng . 36 ( 2024 ) 137 - 151 . doi: 10 .1109/TKDE. 2023 . 3286017 .

[14] V. Pasquadibisceglie , R. Scaringi , A. Appice , G. Castellano , D. Malerba , PROPHET: Explainable predictive process monitoring with heterogeneous graph neural networks , IEEE Trans. Serv. Comput . 17 ( 2024 ) 4111 - 4124 . doi: 10 .1109/TSC. 2024 . 3463487 .

[15] S. Brody , U. Alon , E. Yahav, How attentive are graph attention networks? , in: ICLR 2022 , 2022 , pp. 1 - 26 .

[16] Z. Ying , D. Bourgeois , J. You , M. Zitnik , J. Leskovec, GNNExplainer: Generating explanations for graph neural networks , in: NeurIPS 2019 , 2019 , pp. 9240 - 9251 .

[17] T. Mikolov , K. Chen , G. Corrado, J. Dean , Eficient estimation of word representations in vector space , in: ICLR 2013 , 2013 .

[18] B. Wickramanayake , Z. He , C. Ouyang , C. Moreira , Y. Xu , R. Sindhgatta , Building interpretable models for business process prediction using shared and specialised attention mechanisms , KnowledgeBased Systems 248 ( 2022 ) 1 - 22 . doi:https://doi.org/10.1016/j.knosys. 2022 . 108773 .

[19] Z. A. Bukhsh , A. Saeed , R. M. Dijkman , Processtransformer: Predictive business process monitoring with transformer network , CoRR abs/2104 .00721 ( 2021 ).