=Paper=
{{Paper
|id=Vol-3113/paper1
|storemode=property
|title=Towards an Experiment for Analyzing Subprocess Navigation in BPMN Tooling (short paper)
|pdfUrl=https://ceur-ws.org/Vol-3113/paper1.pdf
|volume=Vol-3113
|authors=Daniel Lübke,Maike Ahrens
|dblpUrl=https://dblp.org/rec/conf/zeus/LubkeA22
}}
==Towards an Experiment for Analyzing Subprocess Navigation in BPMN Tooling (short paper)==
https://ceur-ws.org/Vol-3113/paper1.pdf
Towards an Experiment for Analyzing Subprocess
Navigation in BPMN Tooling
Daniel Lübke1,2[https://orcid.org/0000−0002−1557−8804] and
Maike Ahrens2[https://orcid.org/0000−0002−9577−0598]
1
Digital Solution Architecture, Hanover, Germany
2
Leibniz Universität Hannover, Germany
daniel.luebke@digital-solution-architecture.com
{daniel.luebke,maike.ahrens}@inf.uni-hannover.de
https://www.digital-solution-architecture.com
Abstract. Complex BPMN models can be decomposed vertically by
using collapsed sub-processes and call activities. However, tool support
to ease modelers and model readers with the task of following the links
between such models is implemented differently in modeling tools and
it is unclear which variant is the best. Thus, the primary objective of
the planned study is to understand strengths and weaknesses of different
modeling support in tools and its implications on model comprehensibility.
We analyzed modeling tools for different navigation options and found
three different ways of support for modeling users. Based on those findings
we designed an experiment for an eye tracking study, which analyzes the
usability of the different implementation variants.
Keywords: BPMN · BPMN Tooling · Understandability · Subprocess
Navigation · Experiment
1 Motivation
BPMN is the lingua franca for business process modeling. For serving as a
communication medium best, models must be as comprehensible as possible for
their stakeholders. Especially the larger models get, the less understandable they
become. There has been much research into BPMN understandability lately. This
includes two experiments in the area of BPMN layout that result in conflicting
advice for designers of large BPMN models: A study that compares the use of
diagrams with subprocesses vs. flat diagrams by Turetken et al. [6, 7] and an
eye-tracking study comparing different layouts by Lübke et al. [4]:
While Turetken et al. found that subprocesses make it more difficult for
model readers to work with diagrams and lower understandability of the models
significantly, Lübke et al. found that diagrams that are too large to fit on a single
page reduce understandability. To overcome this, diagrams can be laid out to
use more screen estate (e.g., multiline and snake layouts). However, this strategy
takes only so far – larger diagrams must be partitioned horizontally by using link
elements or vertically by using collapsed subprocesses.
J. Manner, D. Lübke, S. Haarmann, S. Kolb, N. Herzberg, O. Kopp (Eds.): 14th ZEUS
Workshop, ZEUS 2022, Bamberg, held virtually due to Covid-19 pandemic, Germany, 24–25
February 2022, published at http://ceur-ws.org
Copyright © 2022 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
2 Daniel Lübke and Maike Ahrens
Thus, the question arises how to structure large diagrams and which existing
modeling and layout options come with the least penalties. Before this question
can be answered, it needs to be established how to navigate subprocesses in
BPMN most efficiently. While Turetken et al. used both paper and on-screen
diagrams, we focus on screen-reading because – especially with executable BPMN
models – modeling is done in a modeling tool and not on paper.
While there have been studies into BPMN tooling, e.g., regarding their
standards compliance [3], and into the comprehensibility of BPMN models in
general (e.g., with modular process models [1, 8]), to the best of our knowledge
the question of usability of subprocess navigation has not been researched yet.
As such, we have analyzed different tools regarding navigation options offered to
modelers for navigating process hierarchies with collapsed subprocesses. Based on
the identified options we propose an eye-tracking experiment to better understand
the impact of the different navigation options offered by modeling tools on BPMN
understandability.
2 Experimental Design
The goal of our proposed experiment, according to the GQM (Goal-Question-
Metric) approach as adopted by Nick & Tautz for research [5], will be:
For the purpose of understanding the effect of different modeling
tooling
with regard to the quality aspect of understandability
of the object of a large BPMN diagram decomposed with collapsed
subprocesses
from the viewpoint of a reader of that model.
Prior to designing the expeirment we analyzed existing modeling tools for
their implementation of navigating collapsed subprocesses. We could identify
three navigation strategies, which we will use as different treatments in our
experiment as shown in Fig. 1:
Fig. 1. Different Tool Implementation Choices: a) No linking of Subprocess, b) Link for
Opening the Subprocess in a new Tab, and c) Bread Crumb Navigation
Analyzing Subprocess Navigation in BPMN Tooling 3
No Support (A): Users need to open a new model in a new tab, which requires
them to know which model to open, where it is saved etc. We found this, for
example, in Camunda Modeler and Enterprise Architect.
Subprocess Symbol Link (B): Users can link and jump to a subprocess
model by clicking on the (+) icon in the collapsed subprocess. The new
process is then opened in a new tab. We found this in Signavio Academic
Edition.
Breadcrumb Navigation (C): Users can also click on the (+) icon but the
opened model is shown in the same editor window. On top of the window a
path is shown where the user is currently located. We found this in ActiveVOS
Designer and in BPMN.io’s development preview.
We decompose the overall research goal based on the identified navigation
methods in modeling tools into the following research questions:
– RQ1: How do different tool support implementations influence speed of the
users to navigate subprocess hierarchies?
– RQ2: How do different tool support implementations influence efficiency of
the users to navigate subprocess hierarchies?
– RQ2: How do different tool support implementations influence cognitive
load of the users to navigate subprocess hierarchies?
3 Planned Execution & Analysis
Our experiment will compare the three identified tool implementations of sub-
process navigation. This is the only independent variable. To eliminate any other
influencing factor, we will not use different existing tools in our experiment
but will implement BPMN viewers each supporting one of the three navigation
methods only.
As a process model we use a large process model of the industrial project
Terravis, which is a Swiss large-scale process integration platform for end-to-end
integration of land register processes [2]. The main process model is refined via
several layers comprising 33 processes in the hierarchy.
We want to recruit both students and professional software developers &
process designers. Students are recruited in our lectures. For recruiting profes-
sionals we want to offer a one-day training in specialized BPMN topics, e.g.,
testing, understandability of models etc., in exchange for the participation in the
experiment.
Each participant will be randomly assigned to one of three groups: one group
for each tool implementation option. As part of the experiment each participant
is asked to answer four questions regarding the business process model: Two
questions can be answered by looking at a single BPMN diagram within the
hierarchy while two other questions can only be answered with information
contained in different parts of the process hierarchy. By recording the participant
behavior with eye tracking, we can also determine differences in terms of visual
effort and "gaze on target"ratios in addition to comprehensibility.
4 Daniel Lübke and Maike Ahrens
We want to measure and evaluate the following metrics for each experiment
group, which serve as the dependent variables:
– Speed: Time to Answer Questions, Time for Navigating the Process Hierarchy,
Number of Clicks used to navigate the process hierarchy
– Efficiency: Questions Correctly Answered, Questions Incorrectly Answered,
Task Efficiency (Correct Answers in Time), Number of Correct Navigations
in Process Hierarchy, Number of Incorrect Navigations in Process Hierarchy
– Cognitive Load: Average Fixation Duration, Number of Fixations, Dwell Time
on BPMN Elements relevant to Question, Dwell Time on BPMN Elements
irrelevant to Question, Pupil Diameter Size
These metrics will be tested for significant differences in means between
the different implementation options. Because there are three levels for the
independent variable, ANOVA will be used for testing differences in means of
dependent variables.
For achieving the envisioned power of 0.8 for hypothesis testing, we require 3
groups of 37 participants each, to detect a difference in means with an effect size
of 0.3 and, a significance level of 5%.
4 Conclusions & Outlook
Within this paper we have outlined an experiment for analyzing different tool
implementation options in BPMN tools to support users navigating process
hierarchies. The next steps will be to set up the experiment, which includes
develop modeling tools to be used in the experiment, recruit participants, and
develop the questions to be asked in the experiment). When the COVID pandemic
permits we will start recruiting both students in on-site lectures and reach out
to software development companies for recruitment of professionals. If you are
interested in participating in this experiment please contact us.
Analyzing Subprocess Navigation in BPMN Tooling 5
Bibliography
1. Amine Abbad Andaloussi, Pnina Soffer, Tijs Slaats, Andrea Burattin, and
Barbara Weber. The impact of modularization on the understandability of
declarative process models: a research model. In NeuroIS Retreat, pages
133–144. Springer, 2020.
2. Walter Berli, Daniel Lübke, and Werner Möckli. Terravis – large scale business
process integration between public and private partners. In Erhard Plödereder,
Lars Grunske, Eric Schneider, and Dominik Ull, editors, Lecture Notes in
Informatics (LNI), Proceedings INFORMATIK 2014, volume P-232, pages
1075–1090. Gesellschaft für Informatik e.V., Gesellschaft für Informatik e.V.,
2014.
3. Udo Kannengiesser. Evaluation of bpmn tools. National Information Com-
munications Technology Australia, 2007.
4. Daniel Lübke, Maike Ahrens, and Kurt Schneider. Influence of diagram
layout and scrolling on understandability of BPMN processes: an eye tracking
experiment with BPMN diagrams. Information Technology and Management,
2021.
5. Markus Nick and Carsten Tautz. Practical evaluation of an organizational
memory using the goal-question-metric technique. In German Conference on
Knowledge-Based Systems, pages 138–147. Springer, 1999.
6. Oktay Turetken, Ahmet Dikici, Irene Vanderfeesten, Tessa Rompen, and Onur
Demirors. The influence of using collapsed sub-processes and groups on the
understandability of business process models. Business & Information Systems
Engineering, pages 1–21, 2019.
7. Oktay Turetken, Tessa Rompen, Irene Vanderfeesten, Ahmet Dikici, and Jan
van Moll. The effect of modularity representation and presentation medium
on the understandability of business process models in bpmn. In International
Conference on Business Process Management, pages 289–307. Springer, 2016.
8. Michael Winter, Rüdiger Pryss, Thomas Probst, Julia Baß, and Manfred
Reichert. Measuring the cognitive complexity in the comprehension of modular
process models. IEEE Transactions on Cognitive and Developmental Systems,
2021.