BPMN models are often used by researchers to illustrate and validate new approaches that operate over such models. However, not always models are distributed in their source format but they are distributed as images within a document (e.g., a scientific contribution). To actually reuse those models, one has to manually redraw them. Manually redrawing a BPMN model is a time-consuming and error-prone activity. In this work, we present BPMN-Redrawer a tool that makes use of machine learning techniques for supporting the redrawing of BPMN models from images to actual models in .bpmn format. The tool is made available open source and it is open to contribution from the community.
.png
Upload Model Image
Detect BPMN Node Elements
Detect Connecting Objects
Detect Labels
Connect BPMN Nodes
Assign Labels to BPMN Nodes and Connecting Objects
Generate the BPMN Model
Adjust the
BPMN Model .bpmn
Download the BPMN Model .bpmn
In this paper we present BPMN-Redrawer, an approach, and the corresponding open source tool, to support the activity of “redrawing” BPMN models from images. BPMN-Redrawer comes in the form of a web application that provides a user interface to request the automatic redrawing of a BPMN model. The conversion of images in actual BPMN models can be seen as an object detection problem in which the to-be-detected objects are BPMN elements. Our approach makes usage of supervised machine learning algorithms and tools for the training of models capable of detecting BPMN elements. This approach could foster the reuse of BPMN models reported within documents, converting them from images to .bpmn files that can be used for conducting further activities.
Recently, a tool that shares our objective of speeding up the redrawing of BPMN models
has been developed [
The functionalities of BPMN-Redrawer are made available through a web application that allows users to upload .png images and request their conversion into actual BPMN models stored in .bpmn format. We refer to such a process with the term “model redrawing”. A schematization of such a process together with the technology involved, is reported in Fig. 1. The process of redrawing a BPMN model could be divided into three main phases: BPMN element detection, BPMN element linking, and BPMN model generation. In the following we report a detailed description of each phase and step.
Detection Phase. It is the main phase of the approach, that starts after a user uploads an
image. It is composed of three steps: BPMN nodes detection, BPMN connecting object detection,
and BPMN label detection. For detecting BPMN nodes and connecting objects we use the
well-established framework Detectron23 which provides state-of-the-art detection algorithms
with already trained baselines.BPMN nodes can be detected by means of their bounding boxes.
To do so, we fine-tuned a pre-trained Faster Region Based Convolutional Neural Networks
(R-CNN) using Stochastic Gradient Descent with a batch size of 4 and a decreasing learning
rate starting from 0.0025. As a Convolutional Neural Network (CNN) backbone, we chose the
ResNet-50 with Feature Pyramid Network (FPN). The detection of BPMN connecting objects
implies not only the discovery of its bounding box but also the prediction of two keypoints:
one for the head and one for the tail. For this reason, we fine tuned a pre-trained Keypoint
R-CNN network using Stochastic Gradient Descent with batch size at 4 and a learning rate
ifxed at 0.00025. In this case, as a backbone, we use a ResNet-101 with FPN. For detecting BPMN
labels, we used the Optical Character Recognition (OCR) engine Tesseract [
Linking Phase. It is the second phase of the redrawing process and it comprehends two steps: Connect BPMN Nodes and Assign Labels to BPMN elements. For such a connection we relied on the euclidean distance between elements: the target and the source of a connecting object are associated with the elements that are respectively closer to its head and tail, while each label is assigned to the closest element.
BPMN Model Generation Phase. It is the third and last phase of the approach. It is composed of two steps: BPMN model generation and BPMN model adjustment. The first step consists in parsing the results of the previous phases into a BPMN model (in .bpmn format). The resulting ifle is obtained by populating a BPMN template, that we created with the Jinja templating engine.4 Once the BPMN model is obtained, the user can either download it or adjust it using the integrated bpmn-js editor. This operation is facilitated by the display of the original image beside the editor as reported in Fig. 2.
BPMN-Redrawer is able to recognise up to thirty-two BPMN nodes and three BPMN connecting
objects (e.g., Sequence Flow, Message Flow, Data Association). All the elements are reported in
Fig. 3 together with their name and the average precision (AP) with which BPMN-Redrawer
recognises them. For conducting the training activities we started from a dataset of 663 BPMN
models images derived from models stored on the RePROSitory platform.5 However, considering
that the usage of the BPMN elements varies between models [
Some BPMN elements may present graphical markers (e.g., send/receive tasks, manual tasks,
script tasks etc.). In the actual version of the tool, elements are redrawn without markers leaving
to the user the possibility to adjust the model by means of the available editor and original
image (see Fig. 2). Some BPMN models may also report customised elements due to BPMN
extensions [
Diferent BPMN editors can have diferent ways of graphically representing the same BPMN element (e.g., adding a coloured background, having a diferent size of the element, etc.). In this ifrst version of BPMN-Redrawer we started from images of BPMN models designed with the bpmn-js editor therefore more accurate results are get when we ask BPMN-Redrawer to redraw a model originally designed with bpmn-js. We plan to improve our tool by extending the dataset we used for the training activities by also including images of BPMN models designed with 5Models Collection used: https://pros.unicam.it:4200/guest/collection/bpmn_redrawer 6BPMN-Redrawer Dataset: https://huggingface.co/PROSLab/BPMN-Redrawer-Dataset other editors such as: Signavio, Eclipse BPMN Modeler, etc.
Since BPMN-Redrawer operates starting from images of BPMN models, the quality of the image afects the capability of properly recognising and redrawing the elements. We are working to define possible indicators of the image quality that can impact the redrawing of the model. As future work we also plan to add functionalities to the platform in such a way to provide, together with the redrawn model also information about the quality of the obtained result, reporting whether the model is a valid BPMN model, and the amount of elements that have been redrawn. In addition, we plan to improve the results of the connecting objects and labels recognition steps by investigating additional techniques to be used.
BPMN-Redrawer is open source, this enables anyone to access the code, apply changes, train new machine learning models starting from diferent datasets of BPMN images, and easily deploy them. The same approach can be used for automatising the redrawing of other types of models such as: Petri Nets, Event Process Chain, UML Diagrams, etc.
BPMN-Redrawer tool is accessible at http://pros.unicam.it/bpmn-redrawer-tool. The screencast available at https://youtu.be/0e2qnbSp9XY shows a typical user experience. The source code is available at https://github.com/PROSLab/BPMN-Redrawer. A Docker image to easily deploy the tool is also made available at https://hub.docker.com/repository/docker/proslab/bpmn-redrawer.