BPMN, which is the underlying modeling notation of many BPM endeavours and business information system development projects, is a rich modeling language, which also offers redundant constructs, i.e., different syntax can express the same semantics. We want to investigate which syntactical constructs are preferred by model readers if different ways to model message exchanges are offered by BPMN. In an empirical study we asked 77 participants which BPMN model they prefer for expressing eight situations. We found that send tasks and intermediate message catch events are significantly preferred. Also, event-based gateways are preferred over boundary events for many variants of the Deferred Choice pattern.
work. In Section 3 the design of our empirical study is presented. The results are presented in Section 4 and an interpretation of those are given in Section 5. Finally, we conclude and give an outlook.
Quality of business process models is multi-faceted. Lindland et al. [
By following the Goal-Question-Metric (GQM) approach [
Understand the Subjective Preference with regard to Semantically Equivalent Elements in BPMN 2.0
from the viewpoint of a Model Reader.
This goal is refined into (research) questions. While BPMN has many redundancies, we concentrate on the ones below. We want to answer, which construct for each of the following pairs of semantically equivalent BPMN constructs are preferred:
sending messages: The send task and the intermediate message throw event both send a message.
sage, BPMN also offers a receive task and an intermediate message catch event for receiving
the end of a process execution, a send task and a none end event can be used. Alternatively, an message throw end event can be used.
RQ4 Deferred Choice between two messages (diff. prob.): A Deferred Choice [
RQ4. However, the incoming messages have the same probability, i.e., both events following the event-based gateway and both messages occur equally often.
to RQ4 but this time the Deferred Choice is not between two messages but instead resembles a deadline situation with a message event and a timer event. It is more probable to receive the message than to time-out. This pattern is presented as an event-based gateway with two following events or with a receive task with an interrupting timer boundary event.
lar to RQ6. However, the incoming message and the time-out have the same probability.
cerned with a Deferred Choice between two messages and a timer, i.e., a scenario in which one of two messages must be received within a certain time. This can – again – be modeled as an event-based gateway followed by two message events and one timer event, or by a receive task with two boundary events.
We measure the subjective preferences of study participants as the only metric for all research questions. For all research questions the null hypothesis H0 is that there is no preference for one of the two alternatives. Accordingly, H1 is that one of the two alternatives is preferred.
The study setup is similar to a previous study by Scholz & Lübke [
Participants were a) recruited from lectures of the authors and b) professionals were asked to participate. We tracked the group to which a participant belongs to by using different invitation links. Participation was voluntary and no incentives were given. The number of participants per group and a more detailed description is shown in Table 1. All in all, we had 87 participants in total. After removing those, who did not complete the quiz or changed their answers in between, 77 participants remained.
As a first step we performed a power test: For a two-sided hypothesis test with α = p = 0.05 and confidence β = 0.95 for a medium effect of h = 0.5 yields that we required at least 52 participants. As described above we recruited more participants than required. For eliminating extraneous variables we took following measures: We randomized the order in which questions (i.e., diagram pairs) were shown. Thereby, we try to eliminate learning and fatigue effects. We also randomized the order in which diagrams are shown.
The statistical evaluation of the gathered data is shown in Table 2. The statistical significance indicated by the p-values is marked by asterisks (*: p ≤ 0.05, **: p ≤ 0.01, ***: p ≤ 0.001). Similarily, the effect is denoted by pluses (+: h ≥ 0.2, ++: h ≥ 0.5, +++: h ≥ 0.8). 5. Interpretation 5.1. Evaluation of Results & Implications #A
#B The send task is significantly preferred over a message throw event (RQ1). It seems that participants see the sending of a message more as a task, i.e., an active action, and therefore prefer the task instead of an event.
In contrast to RQ1, participants significantly prefer a message catch event for waiting on a message receive (RQ2). Interestingly, it is inconsistent to use different syntax for sending and receiving messages. This can mean that perhaps participants differentiate between active and passive/waiting elements.
There is no significant difference for sending a message at the process end (RQ3). In contrast to a significant preference for a send task during the process, there is no clear preference for a send task with an end event or a message end event. It seems that the additional penalty of a second symbol and its associated space requirements is not worth to keep up the semantic difference experienced in RQ1.
When modeling a Deferred Choice between two messages which arrive with different probabilities, participants prefer the use of an event-based gateway (RQ4). It may be that the visuals of two white envelopes – one in the receive task and one in the boundary event – is not attractive. Participants have an even stronger preference for the gateway if the probability of the messages are the same (RQ5).
When modeling a time-out, i.e., a Deferred Choice between a message and a timer, neither the gateway nor the boundary event is preferred – regardless of whether the timer is as likely to occur (RQ6) or is only triggered as an exception (RQ7). This contrasts with the results from RQ4/5, which are structurally the same but use a different second event. While more participants liked the gateway for same probabilities of events and more participants liked the boundary event for exceptional cases, these differences were not significant. More research has to further clarify whether there is a difference with a small effect or not.
If the Deferred Choice is between two messages and a timer event (RQ8) there is a strong, significant preference to the event-based gateway. However, we cannot attribute to why this is: While in our study planning we wanted to examine the effect of a larger number of boundary events, another possible explanation is that a send task with a message boundary event is disliked as RQ4 and RQ5 have shown.
Because we only measured subjective preferences no quantative data on model comprehension could be measured. This study still gives insights into model perception, especially with different variants of the Deferred Choice pattern. Like all studies which include students, the question of generalizability arises. However, we have seen that no differences between our groups exist – this also means that the group of professionals does not behave significantly different from the students. While we had a considerable amount of participants, some research questions gave non-significant results with a small effect size in the range of 0.1 ≤ h ≤ 0.2. To have adequate power in the statistical tests, more participants (approx. 350) are required.
Following from these results practitioners should amend existing modeling guidelines by the following rules: 1) Use Send Tasks for sending messages during process execution, 2) use Message Catch Events for receiving messages during process execution, and 3) use Event-based Gateways when implementing the Deferred Choice pattern when receiving multiple messages. Modelers should keep in mind that this is the first study to examine these constructs. Hopefully, future studies will strengthen or refute these results and thus these proposed modeling guidelines.
Within this paper we presented our empirical study with students from two universities and professionals on the subjective preference of syntactically redundant, message-related constructs in BPMN. We found a strong subjective preference for send tasks over message throw events within the process-flow and for message catch events over receive tasks. We also found that Deferred Choices in event-based gateways are preferred over boundary events in the case of two message events or three events. We could find no significant preference for Deferred Choices with a message and a timer (“time-outs”) or for the sending of a message on process completion. While the results are interesting in themselves, this study lays the foundation for further empirical inqueries: Follow up studies, especially experiments, can investigate and compare understandability of redundant BPMN message-related constructs. This way, especially eye-tracking experiments, can be used to gather quantative data to evaluate whether the subjective preferences match the differences in objective understandability in the future, and further developing modeling guidelines for BPMN.
We’d like to thank all participants who took part in our study. Additionally, we like to thank Kurt Schneider, Dieter Kähny, and Barbara Ulrich for distributing the quiz within their classes and organizations.