Complex BPMN models can be decomposed vertically by using collapsed sub-processes and call activities. However, tool support to ease modelers and model readers with the task of following the links between such models is implemented diferently in modeling tools and it is unclear which variant is the best. Thus, the primary objective of the planned study is to understand strengths and weaknesses of diferent modeling support in tools and its implications on model comprehensibility. We analyzed modeling tools for diferent navigation options and found three diferent ways of support for modeling users. Based on those findings we designed an experiment for an eye tracking study, which analyzes the usability of the diferent implementation variants.
BPMN is the lingua franca for business process modeling. For serving as a
communication medium best, models must be as comprehensible as possible for
their stakeholders. Especially the larger models get, the less understandable they
become. There has been much research into BPMN understandability lately. This
includes two experiments in the area of BPMN layout that result in conflicting
advice for designers of large BPMN models: A study that compares the use of
diagrams with subprocesses vs. flat diagrams by Turetken et al. [
While Turetken et al. found that subprocesses make it more dificult for model readers to work with diagrams and lower understandability of the models significantly, Lübke et al. found that diagrams that are too large to fit on a single page reduce understandability. To overcome this, diagrams can be laid out to use more screen estate (e.g., multiline and snake layouts). However, this strategy takes only so far – larger diagrams must be partitioned horizontally by using link elements or vertically by using collapsed subprocesses.
Thus, the question arises how to structure large diagrams and which existing modeling and layout options come with the least penalties. Before this question can be answered, it needs to be established how to navigate subprocesses in BPMN most eficiently. While Turetken et al. used both paper and on-screen diagrams, we focus on screen-reading because – especially with executable BPMN models – modeling is done in a modeling tool and not on paper.
While there have been studies into BPMN tooling, e.g., regarding their
standards compliance [
The goal of our proposed experiment, according to the GQM
(Goal-QuestionMetric) approach as adopted by Nick & Tautz for research [
Prior to designing the expeirment we analyzed existing modeling tools for their implementation of navigating collapsed subprocesses. We could identify three navigation strategies, which we will use as diferent treatments in our experiment as shown in Fig. 1: No Support (A): Users need to open a new model in a new tab, which requires them to know which model to open, where it is saved etc. We found this, for example, in Camunda Modeler and Enterprise Architect.
Subprocess Symbol Link (B): Users can link and jump to a subprocess model by clicking on the (+) icon in the collapsed subprocess. The new process is then opened in a new tab. We found this in Signavio Academic Edition.
Breadcrumb Navigation (C): Users can also click on the (+) icon but the opened model is shown in the same editor window. On top of the window a path is shown where the user is currently located. We found this in ActiveVOS Designer and in BPMN.io’s development preview.
We decompose the overall research goal based on the identified navigation methods in modeling tools into the following research questions: – RQ1: How do diferent tool support implementations influence speed of the users to navigate subprocess hierarchies? – RQ2: How do diferent tool support implementations influence eficiency of the users to navigate subprocess hierarchies? – RQ2: How do diferent tool support implementations influence cognitive load of the users to navigate subprocess hierarchies? 3
Our experiment will compare the three identified tool implementations of subprocess navigation. This is the only independent variable. To eliminate any other influencing factor, we will not use diferent existing tools in our experiment but will implement BPMN viewers each supporting one of the three navigation methods only.
As a process model we use a large process model of the industrial project
Terravis, which is a Swiss large-scale process integration platform for end-to-end
integration of land register processes [
We want to recruit both students and professional software developers & process designers. Students are recruited in our lectures. For recruiting professionals we want to ofer a one-day training in specialized BPMN topics, e.g., testing, understandability of models etc., in exchange for the participation in the experiment.
Each participant will be randomly assigned to one of three groups: one group for each tool implementation option. As part of the experiment each participant is asked to answer four questions regarding the business process model: Two questions can be answered by looking at a single BPMN diagram within the hierarchy while two other questions can only be answered with information contained in diferent parts of the process hierarchy. By recording the participant behavior with eye tracking, we can also determine diferences in terms of visual efort and "gaze on target"ratios in addition to comprehensibility.
We want to measure and evaluate the following metrics for each experiment group, which serve as the dependent variables: – Speed: Time to Answer Questions, Time for Navigating the Process Hierarchy,
Number of Clicks used to navigate the process hierarchy – Eficiency: Questions Correctly Answered, Questions Incorrectly Answered, Task Eficiency (Correct Answers in Time), Number of Correct Navigations in Process Hierarchy, Number of Incorrect Navigations in Process Hierarchy – Cognitive Load: Average Fixation Duration, Number of Fixations, Dwell Time on BPMN Elements relevant to Question, Dwell Time on BPMN Elements irrelevant to Question, Pupil Diameter Size
These metrics will be tested for significant diferences in means between the diferent implementation options. Because there are three levels for the independent variable, ANOVA will be used for testing diferences in means of dependent variables.
For achieving the envisioned power of 0.8 for hypothesis testing, we require 3 groups of 37 participants each, to detect a diference in means with an efect size of 0.3 and, a significance level of 5%. 4
Within this paper we have outlined an experiment for analyzing diferent tool implementation options in BPMN tools to support users navigating process hierarchies. The next steps will be to set up the experiment, which includes develop modeling tools to be used in the experiment, recruit participants, and develop the questions to be asked in the experiment). When the COVID pandemic permits we will start recruiting both students in on-site lectures and reach out to software development companies for recruitment of professionals. If you are interested in participating in this experiment please contact us.
Bibliography