Predicting the next activity of a running process is an important aspect of process management. Recently, arti cial neural networks, so called deep-learning approaches, have been proposed to address this challenge. This demo paper describes a software application that applies the Tensor ow deep-learning framework to process prediction. The software application reads industry-standard XES les for training and presents the user with an easy-to-use graphical user interface for both training and prediction. The system provides several improvements over earlier work. This demo paper focuses on the software implementation and describes the architecture and user interface.
The prediction of the future development of a running case, given information
about past cases and the current state of the case, i.e. predicting trace su xes
from trace pre xes, is a core problem in business process intelligence (BPI).
Recently, deep learning [
Because of the sequential nature of process traces, recurrent neural networks
are a natural t to the problem of process prediction. An initial application of
RNN to BPI [
The software application described here extends the previous approaches
in a number of ways. Primarily, it is more general and exible with respect
to the case and event attributes that can be used as predictor or predicted
variables. Whereas [
Additionally, our approach can predict multiple variables, for example the activity as well as the resource information of the next event. We use either shared or separate RNN layers for each predicted attribute.
Finally, the software tool presented in this demo paper includes a graphical user interface that guides novice users and avoids the need for specialized coding, it provides integration with a graphical dashboard, and a stand-alone prediction application with an easy-to-use prediction API.
As a demo paper, this paper focuses on the software implementation,
architecture and user interface with only a short exposition of the neural network
background. In particular, we describe the input and output handling of the
neural network (Sections 4.2, 4.4), as this is where our approach di ers from [
A recurrent neural network (RNN) is one in which network cells can maintain
state information by feeding the state output back to themselves, often using
a form of long short term memory (LSTM) cells [
Initial State 1 Initial State 2 b x k
input LSTM LSTM state state b x k
input LSTM LSTM state state b x k
input LSTM LSTM state state b x k
input LSTM LSTM state state b x k
input LSTM LSTM output output output output output b x k b x k b x k b x k b x k
Final State 1 Final State 2 Our software implementation is based on the open-source Tensor ow deep learning framework5. Tensors are generalizations of matrices to more than two dimensions, i.e. n-dimensional arrays. A Tensor ow application builds a computational graph using tensor operations. A loss function is a graph node that is to be minimized. It compares computed outputs to target outputs in some way, for example as cross-entropy for categorical variables, or root mean squared error for numeric variables. Tensor ow computes gradients of all tensors in the graph with respect to the loss function and provides various gradient descent optimizers. Training proceeds by iteratively feeding the Tensor or computational graph with inputs and targets and optimizing the loss function using back-propagated errors. 4
Our software system consists of two separate applications, one for training a deep-learning model, and another one for predicting from a trained model. This section describes the training application.
4.1
Training data is read from XES log les [
RNN training proceeds in \epochs". In each epoch, the entire training dataset is used. Before each epoch, the state of the RNN is reset to zero while trained network parameter values are retained. Within each epoch, training proceeds in batches of size b with averaged gradients to avoid overly large uctuations of gradients. The batch size b can be freely chosen by the user.
For each unrolled step s, the RNN accepts a oating point input tensor Is 2 Rb p, where b is the batch size and p can be freely chosen. Numerical and datetime predictors are encoded directly, each yielding a oating point tensor Is;i 2 Rb 1 for predictor attribute i. Categorical attributes, encoded as integer category numbers, are transformed using an embedding matrix Ei 2 Rli ki . This is a lookup matrix of size li ki, where li is the number of categories for predictor attribute i and ki can be freely chosen. Embedding lookup transforms an integer category number js;i to a oating point vector of size ki. The larger the value of ki, the better the attribute values are separated in the ki-dimensional space. At the same time, larger ki lead to increased computational demands. The output of the embedding lookup E(:) is a tensor Is;i 2 Rb ki = E(js;i). The tensors for all predictor attributes are concatenated, yielding a tensor Cs 2 Rb m where m is the sum of the second dimensions of the concatenated tensors. An input projection P I
s 2 Rm p can be applied so that the input to each unrolled step of the RNN is Is = Cs PsI . 4.3
In our approach, the user can train a model on multiple predicted event attributes (\target variables") concurrently, for example predicting the activity as well as the resource of the next event. For this, the user can choose to share the RNN layers across the di erent target variables, or to construct a separate RNN for each target variable. The input embeddings are shared in all cases. 4.4
Li = (Os;i The output of the RNN for each unrolled step is a tensor Os 2 Rb p. For a categorical predicted variable i, this is multiplied by an output projection PsO;i 2 Rp li to yield Os;i 2 Rb li = Os PsO;i. A softmax function is applied to generate probabilities over the li di erent categories Ss;i 2 Rb li = softmaxli (Os;i). A cross-entropy loss function Li = HSs;i (Ts;i) is then applied, comparing the output probabilities against \one-hot" encoded targets T . One-hot encoding is a vector with the element indicating the target category number equal to one and the remainder set to zero. For numerical attributes, the output Os is multiplied by a PsO;i 2 Rp 1 output projection, yielding Os;i 2 Rb 1 = Os PsO;i. This is compared to target values Ts;i using the mean square error (MSE) q Ts;i)2, the root mean square error (RMSE) Li =
Ts;i)2, (Os;i or the mean absolute error (MAE) loss function Li = jOs;i Ts;ij2. 4.5
Our software logs summary information about the proportion of correct predictions and the loss function value for each training step. The embedding matrices for all categorical predictor variables are saved at the end of training, together with the computational graph. This information can be read by the Tensorboard tool (Fig. 2) to visualize the training performance, the graph structure, and to analyze the embedding matrices using t-SNE or principal components projection into two or three dimensions. Finally, the entire trained network is saved in a \frozen", compacted form to be loaded into the prediction application. 5
The prediction application loads a trained model, saved by the training application, as well as the corresponding training con guration, and predicts trace su xes from trace pre xes. Trace pre xes are read from XES les. Because the trained model expects batches of size b, at most b trace pre xes are loaded at a time. The network state is initialized to zero and the trace pre xes are input to the trained network, encoded as described in Sec. 4.2. The network outputs for the last element of a trace pre x are the predictions for the attributes of the next event. For categorical attributes, the integer output indicating the category number is translated back to the character string value. For datetime-typed attributes, the attribute value is computed by adding the predicted value to the attribute value of the prior event. The predicted event is then added to the trace pre x and the prediction process can be repeated. In this way, su xes of arbitrary length can be predicted. The user can stop prediction when an end-ofcase marker has been predicted, or after a speci ed number of events have been predicted. The predicted su xes are written back to an XES le. 6
Our software is implemented in Python 2.7 and uses Tensor ow 0.12 and Tkinter for the user interface. Figure 3 shows the main screen that guides the user through the selection of an XES le, the con guration of the RNN and training parameters, to the training of the model.
Figure 4 shows the main con guration screen with sections for multi-attribute classi ers, global event and case attributes, RNN and training parameters and a choice of optimizer. Any classi er, global event and case attribute may be included as predictor, and classi ers and global event attributes may be chosen as predicted attributes (targets). The user can specify embedding dimensions for categorical attributes; the default values are the square root of the number of categories. The RNN con guration allows the user to specify batch size, number of unrolled steps, number of RNN layers, number of training epochs, etc. Users can specify an optional input mapping and the desired size of the RNN input vector. Finally, users have a choice of di erent gradient descent optimizers offered by Tensor ow and can adjust their hyperparameters. Con gurations are automatically saved and the user can load saved con gurations.
Our focus is on providing a research tool for experimentation, rather than a production tool. Therefore, we have not made use of distributed Tensor ow and Tensor ow Serving. Tensor ow automatically allocates the compute operations to available CPUs and GPUs on a single machine. This provides adequate performance for the small size of typical event logs (megabyte rather than terabyte). We use our own prediction application with a simple API. 7
We presented a exible deep-learning software application to predict business processes from industry-standard XES event logs. The software provides an easyto-use graphical user interface for con guring predictors, targets, and parameters of the deep-learning prediction method.
We have performed initial validation of the software to verify the correct
operation of the software tool. Using the BPIC 2012 and 2013 datasets with
the model and training parameters reported in [