Towards an Experiment for Analyzing Subprocess Navigation in BPMN Tooling Daniel Lübke1,2[https://orcid.org/0000−0002−1557−8804] and Maike Ahrens2[https://orcid.org/0000−0002−9577−0598] 1 Digital Solution Architecture, Hanover, Germany 2 Leibniz Universität Hannover, Germany daniel.luebke@digital-solution-architecture.com {daniel.luebke,maike.ahrens}@inf.uni-hannover.de https://www.digital-solution-architecture.com Abstract. Complex BPMN models can be decomposed vertically by using collapsed sub-processes and call activities. However, tool support to ease modelers and model readers with the task of following the links between such models is implemented differently in modeling tools and it is unclear which variant is the best. Thus, the primary objective of the planned study is to understand strengths and weaknesses of different modeling support in tools and its implications on model comprehensibility. We analyzed modeling tools for different navigation options and found three different ways of support for modeling users. Based on those findings we designed an experiment for an eye tracking study, which analyzes the usability of the different implementation variants. Keywords: BPMN · BPMN Tooling · Understandability · Subprocess Navigation · Experiment 1 Motivation BPMN is the lingua franca for business process modeling. For serving as a communication medium best, models must be as comprehensible as possible for their stakeholders. Especially the larger models get, the less understandable they become. There has been much research into BPMN understandability lately. This includes two experiments in the area of BPMN layout that result in conflicting advice for designers of large BPMN models: A study that compares the use of diagrams with subprocesses vs. flat diagrams by Turetken et al. [6, 7] and an eye-tracking study comparing different layouts by Lübke et al. [4]: While Turetken et al. found that subprocesses make it more difficult for model readers to work with diagrams and lower understandability of the models significantly, Lübke et al. found that diagrams that are too large to fit on a single page reduce understandability. To overcome this, diagrams can be laid out to use more screen estate (e.g., multiline and snake layouts). However, this strategy takes only so far – larger diagrams must be partitioned horizontally by using link elements or vertically by using collapsed subprocesses. J. Manner, D. Lübke, S. Haarmann, S. Kolb, N. Herzberg, O. Kopp (Eds.): 14th ZEUS Workshop, ZEUS 2022, Bamberg, held virtually due to Covid-19 pandemic, Germany, 24–25 February 2022, published at http://ceur-ws.org Copyright © 2022 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 Daniel Lübke and Maike Ahrens Thus, the question arises how to structure large diagrams and which existing modeling and layout options come with the least penalties. Before this question can be answered, it needs to be established how to navigate subprocesses in BPMN most efficiently. While Turetken et al. used both paper and on-screen diagrams, we focus on screen-reading because – especially with executable BPMN models – modeling is done in a modeling tool and not on paper. While there have been studies into BPMN tooling, e.g., regarding their standards compliance [3], and into the comprehensibility of BPMN models in general (e.g., with modular process models [1, 8]), to the best of our knowledge the question of usability of subprocess navigation has not been researched yet. As such, we have analyzed different tools regarding navigation options offered to modelers for navigating process hierarchies with collapsed subprocesses. Based on the identified options we propose an eye-tracking experiment to better understand the impact of the different navigation options offered by modeling tools on BPMN understandability. 2 Experimental Design The goal of our proposed experiment, according to the GQM (Goal-Question- Metric) approach as adopted by Nick & Tautz for research [5], will be: For the purpose of understanding the effect of different modeling tooling with regard to the quality aspect of understandability of the object of a large BPMN diagram decomposed with collapsed subprocesses from the viewpoint of a reader of that model. Prior to designing the expeirment we analyzed existing modeling tools for their implementation of navigating collapsed subprocesses. We could identify three navigation strategies, which we will use as different treatments in our experiment as shown in Fig. 1: Fig. 1. Different Tool Implementation Choices: a) No linking of Subprocess, b) Link for Opening the Subprocess in a new Tab, and c) Bread Crumb Navigation Analyzing Subprocess Navigation in BPMN Tooling 3 No Support (A): Users need to open a new model in a new tab, which requires them to know which model to open, where it is saved etc. We found this, for example, in Camunda Modeler and Enterprise Architect. Subprocess Symbol Link (B): Users can link and jump to a subprocess model by clicking on the (+) icon in the collapsed subprocess. The new process is then opened in a new tab. We found this in Signavio Academic Edition. Breadcrumb Navigation (C): Users can also click on the (+) icon but the opened model is shown in the same editor window. On top of the window a path is shown where the user is currently located. We found this in ActiveVOS Designer and in BPMN.io’s development preview. We decompose the overall research goal based on the identified navigation methods in modeling tools into the following research questions: – RQ1: How do different tool support implementations influence speed of the users to navigate subprocess hierarchies? – RQ2: How do different tool support implementations influence efficiency of the users to navigate subprocess hierarchies? – RQ2: How do different tool support implementations influence cognitive load of the users to navigate subprocess hierarchies? 3 Planned Execution & Analysis Our experiment will compare the three identified tool implementations of sub- process navigation. This is the only independent variable. To eliminate any other influencing factor, we will not use different existing tools in our experiment but will implement BPMN viewers each supporting one of the three navigation methods only. As a process model we use a large process model of the industrial project Terravis, which is a Swiss large-scale process integration platform for end-to-end integration of land register processes [2]. The main process model is refined via several layers comprising 33 processes in the hierarchy. We want to recruit both students and professional software developers & process designers. Students are recruited in our lectures. For recruiting profes- sionals we want to offer a one-day training in specialized BPMN topics, e.g., testing, understandability of models etc., in exchange for the participation in the experiment. Each participant will be randomly assigned to one of three groups: one group for each tool implementation option. As part of the experiment each participant is asked to answer four questions regarding the business process model: Two questions can be answered by looking at a single BPMN diagram within the hierarchy while two other questions can only be answered with information contained in different parts of the process hierarchy. By recording the participant behavior with eye tracking, we can also determine differences in terms of visual effort and "gaze on target"ratios in addition to comprehensibility. 4 Daniel Lübke and Maike Ahrens We want to measure and evaluate the following metrics for each experiment group, which serve as the dependent variables: – Speed: Time to Answer Questions, Time for Navigating the Process Hierarchy, Number of Clicks used to navigate the process hierarchy – Efficiency: Questions Correctly Answered, Questions Incorrectly Answered, Task Efficiency (Correct Answers in Time), Number of Correct Navigations in Process Hierarchy, Number of Incorrect Navigations in Process Hierarchy – Cognitive Load: Average Fixation Duration, Number of Fixations, Dwell Time on BPMN Elements relevant to Question, Dwell Time on BPMN Elements irrelevant to Question, Pupil Diameter Size These metrics will be tested for significant differences in means between the different implementation options. Because there are three levels for the independent variable, ANOVA will be used for testing differences in means of dependent variables. For achieving the envisioned power of 0.8 for hypothesis testing, we require 3 groups of 37 participants each, to detect a difference in means with an effect size of 0.3 and, a significance level of 5%. 4 Conclusions & Outlook Within this paper we have outlined an experiment for analyzing different tool implementation options in BPMN tools to support users navigating process hierarchies. The next steps will be to set up the experiment, which includes develop modeling tools to be used in the experiment, recruit participants, and develop the questions to be asked in the experiment). When the COVID pandemic permits we will start recruiting both students in on-site lectures and reach out to software development companies for recruitment of professionals. If you are interested in participating in this experiment please contact us. Analyzing Subprocess Navigation in BPMN Tooling 5 Bibliography 1. Amine Abbad Andaloussi, Pnina Soffer, Tijs Slaats, Andrea Burattin, and Barbara Weber. The impact of modularization on the understandability of declarative process models: a research model. In NeuroIS Retreat, pages 133–144. Springer, 2020. 2. Walter Berli, Daniel Lübke, and Werner Möckli. Terravis – large scale business process integration between public and private partners. In Erhard Plödereder, Lars Grunske, Eric Schneider, and Dominik Ull, editors, Lecture Notes in Informatics (LNI), Proceedings INFORMATIK 2014, volume P-232, pages 1075–1090. Gesellschaft für Informatik e.V., Gesellschaft für Informatik e.V., 2014. 3. Udo Kannengiesser. Evaluation of bpmn tools. National Information Com- munications Technology Australia, 2007. 4. Daniel Lübke, Maike Ahrens, and Kurt Schneider. Influence of diagram layout and scrolling on understandability of BPMN processes: an eye tracking experiment with BPMN diagrams. Information Technology and Management, 2021. 5. Markus Nick and Carsten Tautz. Practical evaluation of an organizational memory using the goal-question-metric technique. In German Conference on Knowledge-Based Systems, pages 138–147. Springer, 1999. 6. Oktay Turetken, Ahmet Dikici, Irene Vanderfeesten, Tessa Rompen, and Onur Demirors. The influence of using collapsed sub-processes and groups on the understandability of business process models. Business & Information Systems Engineering, pages 1–21, 2019. 7. Oktay Turetken, Tessa Rompen, Irene Vanderfeesten, Ahmet Dikici, and Jan van Moll. The effect of modularity representation and presentation medium on the understandability of business process models in bpmn. In International Conference on Business Process Management, pages 289–307. Springer, 2016. 8. Michael Winter, Rüdiger Pryss, Thomas Probst, Julia Baß, and Manfred Reichert. Measuring the cognitive complexity in the comprehension of modular process models. IEEE Transactions on Cognitive and Developmental Systems, 2021.