Creator Suite for Simultaneous Predictive Process Monitoring

Abstract

In the scope of predictive process monitoring (PPM), there exist many tools and techniques for monitoring individual running instances. However, in real-life settings, companies might have to monitor multiple, maybe thousands of instances simultaneously. Here, companies need to be presented with an overview of monitoring results in an aggregated form (i.e., over ALL instances). In this paper, we present a web-based dashboard tool that supports companies with exactly this form of aggregated insights - also referred to as simultaneous PPM. Our tool allows users to easily create metrics, visualizations and dashboards for aggregated predictive insights, and is integrated into Camunda. The tool was conceptually developed in interviews with industrial partners and has been evaluated in an initial user-study.

1. Introduction

As the field of PPM is quickly growing and gaining interest, more and more solutions and PPM techniques have been developed [ 1 ]. However, existing approaches mainly focus on predictions on the level of individual instances, “one-at-a-time”. In real-life settings, where companies may encounter thousands of instances running at the same time, it is however essential that companies are able to quickly gain key insights about ALL running instances in an aggregated form. For example, given all instances, what is the average remaining time?

With current systems that focus only on individual instances, gaining such predictive insights requires experts to check through the prediction values of individual instances manually. Given real-life settings, with hundreds or thousands of instances, this is simply not feasible. Here, methods are needed that present aggregated information and support companies in simultaneous PPM. In this work, we therefore present a novel dashboard system that allows companies to create dashboards for key predictive insights over all running instances (in Camunda).

Our tool is built on our previously introduced PPM engine for Camunda [ 2 ], which enables to make predictions in Camunda on an individual instance level. On the contrary, the work at hand extends our previous plugin with an entirely new layer for creating customized dashboards for simultaneous PPM. In particular, our novel tool introduces a web-based dashboard creator suite that allows users to easily create metrics, visualizations and customized dashboards for CEUR monitoring aggregated predictive insights, as shown in Figure 1. As our tool is tightly integrated with the Camunda system, it allows to directly monitor processes running in the Camunda Workflow System, without the need to transfer or export data to a separate PPM system. (a) Multiple, individual instance predictions (previous work) (b) Aggregation-based dashboard (this work)

Note that while there exist some (yet, few) solutions that ofer aggregated PPM insights – e.g., Nirdizati [ 3 ] ofers some insights such as average remaining time – the area of simultaneous PPM seems heavily underresearched in general. The presented tool therefore ofers new methods and techniques for predictive monitoring in real-life settings. Also, other than existing works, the focus of this tool is an actual dashboard creator component, which allows users to create customized dashboards according to company needs.

Our tool was designed in interviews together with experts in the fields, where we could gain qualitative insights into key requirements for (simultaneous) PPM tools in practice. Furthermore, our tool has been evaluated in an initial user-study, which confirmed the application’s userfriendliness. The tool is available as open-source1 and can be directly used with any Camunda system. In the following, we will present our tool as well as the conducted user-study.

2. Technical Foundation and Industrial Requirements

The presented dashboard tool builds on our previous PPM plugin for Camunda [ 2 ].2 As mentioned, the previous plugin treats and visualizes predictions only on a “per instance” level (cf. Figure 1). Therefore, the dashboard tool presented in this work is built on top of the existing plugin as an aggregation layer to facilitate simultaneous PPM.

To better understand requirements for such a dashboard tool from a practitioners perspective, the requirements engineering phase for this work included interviews with industrial partners. 1https://gitlab.uni-koblenz.de/fg-bks/camunda-ppm-dashboard 2The plugin in [ 2 ] allows to train predictive models directly in Camunda. For model training, the plugin can access and encode process data from the internal Camunda event log history. This includes activities as well as corresponding activity data. Then, for running instances, the plugin provides predictions for next activity, remaining time and risk (Risk is implemented as risk of process failure, but can be implemented with arbitrary cost functions).

EnMvoirdoenlmlinegnt DMecoidsieoln 
 FeeEdrrboar c
k Tanhdesdeisicnutesrsveidewthsewpreerveicoounsdpuluctgeidn ifnroam2-[p2h]aasteadCeasmigunn,sdhaoUwsnerinGFroiguupr.!eA2v(aid).eoFiorsftt,hwisedpirsecsuesnsitoend Supported
 is available online3. As a central outcome of this discussion, a consensus was made by thCeapabilities Execution
 expertInssttahllaatitonofering PPMMoodenllinag “per insta n(if acppelicalbele)vel” was noVatlidsaeteionn as plausibAlsesesimnenptractice, andTerminology that an aggregated view is necessary (Req1). We then conducted three in-depth interviews with experts in the field (cf. Figure 2 (b)). From these interviews, a clear requirement could be obtained, that it must be possible to Ecvraeluaatteioncustomized dashboards (as opposDeodcutmoenotafetrioinng a predefined dashboard) ( Req2). This is why a focus of this project was set on the actual WORKSHOPdashboard creator co mINpToEnRenVtI.EAWlsSo, P1 and P2 clearly stated that such a dashboard should not be integrated as a plugin within the Camunda engine itself, as typical users of such a dashboard General discussion at a
 In-depth innotetrvhieawves wacitche
 ss rights to the underlying Camunda engine (Req3). Camunda Userwgreoruep considered to gesneelercatlelyd experts

Any dashboard should therefore be presented in a separate application, and not be integrated in the Camunda Cockpit.4 Accordingly, we implemented our tool as a separate application.

ID P1 P2 P3

Interviewee

Head of Developer Relations, Camunda Head of Practice Area Business Process Management, Novatec Head of Division – Process Automation, Data Analytics, Data Strategy, Debeka Insurance

Duration

46min 47min 52min Analytical evaluation/
 Proofs for properties Usergroup Workshop

In-depth Interviews (a) 2-Phase interview design (b) In-depth interview overview

Further requirements could be obtained, both from the interviews and through a literature study. A full discussion of all requirements is however beyond the scope of this report and will be presented in a future work. We continue to describe our tool based on the above requirements.

3. Tool Description

The presented tool is a web-based application that allows to create, manage and use dashboards for simultaneous PPM (cf. Req1, Req2). Importantly, the tool does not focus on showing insights for individual process instances, but provides access to aggregated PPM insights, e.g., average remaining time, or all expected upcoming activities. The tool is tightly integrated with Camunda, so that live data from a running Camunda WFMS can directly be accessed without the need for manually transferring the data to the dashboard system. Users can create customized metrics and dashboards. For this, customized visualizations can also be created from a rich set of built-in visualisation-templates such as scatter plots or heatmaps.

A usage example for this is shown in Figure 3. To create a new dashboard, corresponding charts can be created using the integrated chart editor (1). Here, a data set and the corresponding 3https://www.youtube.com/watch?v=sZGIB3Qq8NI 4According to P1, this was also the reason why Camunda Optimize was implemented as a separate tool. visualization type can be defined and fine-tuned. When editing a dashboard, all previously defined visualizations can be added. For example, in Figure 3, the created dashboard features various, aggregated predictive insights, e.g., the average remaining time, average predicted risk, or a heatmap showing all tasks expected in the next week (2). Visualizations can be freely re-arranged or resized via drag&drop. Should a user want to change a concrete visualization in hindsight, this can be easily modified in the chart editor. All changes will then automatically be visible in all dashboards where the visualization was added. In line with Req3, the dashboard system queries the actual prediction data from an external Camunda engine –the previous plugin [ 2 ] has to be installed– and stores this data in a separate database. Per default, the dashboard tool fetches the process data (e.g., running instances) and the corresponding predictions (e.g., remaining times of all instances) once per day. The update interval can be modified, e.g., hourly. 1

All Predicted Activities 2 Average Remaining Time

Average Predicted Risk

Risk Distribution Highest Predicted Risk

Highest Remaining Time

In summary, the main features of the presented tool are as follows: • Create Customized Dashboards and Visualizations. Customized dashboards for simultaneous PPM can be created and maintained. An integrated editor allows users to create customizable metrics and visualizations showing aggregated PPM insights. When editing a dashboard, visualizations can be added, placed and resized freely along a grid. • Aggregate PPM Data Using Several Aggregation Algorithms. For each visualization, the data to be displayed can be pre-processed using one of many built in aggregation algorithms (e.g., aggregate data from instances with a remaining time of more than 2 minutes). By means of these built-in filters, a wide range of PPM use-cases can be covered. • Easy to use User Interface. All features are accessible without programming skills. The tool usability could also be confirmed in an initial user study (cf. Section 4). See also the supplementary document for a more detailed usage example.

4. Evaluation

Regarding the performance evaluation of our tool, the tool builds on our previous prediction plugin in [ 2 ] and does not add any further level of complexity here in regard to the predictions. We refer the reader to [ 2 ] for a previous evaluation with real-life datasets.

To evaluate usability, we conducted an initial user study with 12 participants. The participants conducted five typical tasks in the tool (e.g., create a dashboard). Then, the participants were shown five statements (e.g., “I found the tool easy to use”), to which they should indicate whether they agreed (on a 5-point Likert scale from strongly agree to strongly disagree). The statements were based on the technology acceptance model. Figure 4 shows our survey results. unnecessarily cumbersome complex to use

As can be seen, our tool was seen as easy to use and as quickly learnable. Furthermore, the majority of participants did not see the tool as cumbersome or unnecessarily complex. The 1 participant who found it complex indicated later that he was sure this would improve over time. We also asked open questions to gain feedback. A first iteration of improvements based on these suggestions was already conducted (e.g., improvements to the menu structure). We aim to present our tool to industrial partners for further feedback.

5. Conclusion and Outlook

Based on the conducted workshop and in-depth interviews, the need for tools that facilitate the predictive monitoring of multiple instances in an aggregated form seems clear. Importantly, the experts also emphasized the need for solutions that allow to create customized dashboards. To this end, our tool presented in this work ofers new capabilities for simultaneous PPM and the creation of highly customized dashboards. In general, we feel the area of simultaneous PPM is still rather neglected and there is still much need for future research. We aim to present our tool to industrial partners and will continue to extend our tool.

References

[1] C. Di Francescomarino , C. Ghidini , F. M. Maggi , F. Milani , Ppm methods: Which one suits me best? , in: BPM'18 , Springer, 2018 , pp. 462 - 479 .

[2] N. Bartmann , S. Hill , C. Corea , C. Drodt , P. Delfmann, Applied ppm and hyper parameter optimization in camunda , in: CAISE'21 Demo Track , Springer, 2021 , pp. 129 - 136 .

[3] W. Rizzi , L. Simonetto , C. Di Francescomarino , C. Ghidini , T. Kasekamp , F. M. Maggi , Nirdizati 2 .0, in: BPM'19 Demo Track, CEUR-WS, 2019 , pp. 154 - 158 .