=Paper=
{{Paper
|id=Vol-1859/bpmds-05-paper
|storemode=property
|title=Designing Visual Decision Making Support with the Help of Eye-tracking
|pdfUrl=https://ceur-ws.org/Vol-1859/bpmds-05-paper.pdf
|volume=Vol-1859
|authors=Barbara Weber,Jens Gulden,Andrea Burattin
|dblpUrl=https://dblp.org/rec/conf/emisa/WeberGB17
}}
==Designing Visual Decision Making Support with the Help of Eye-tracking==
https://ceur-ws.org/Vol-1859/bpmds-05-paper.pdf
Designing Visual Decision Making Support with
the Help of Eye Tracking
Barbara Weber1,3 , Jens Gulden2 , and Andrea Burattin3
1
Technical University of Denmark, Denmark
2
University of Duisburg-Essen, Germany
3
University of Innsbruck, Austria
Abstract. Data visualizations are helpful tools to cognitively access
large amounts of data and make complex relationships in data under-
standable. This paper shows how results from neuro-physiological mea-
surements, more specifically eye-tracking, can support justified design
decisions about improving existing data visualizations for exploring pro-
cess execution data. This is achieved by gaining insight into how vi-
sualizations are used for decision-making. The presented examination
is embedded in the domain of process modeling behavior analysis, and
the analyses are performed on the background of representative analyt-
ical questions from the domain of process model behavior analysis. We
present initial findings on one out of three visualization types we have
examined, which is the Rhythm-Eye visualization.
1 Introduction
Research on business process modeling has evolved a wide variety of reflections
on characteristics of process models with regard to their structural and dynamic
properties, their execution semantics, and their expressiveness [1, 2]. With re-
spect to the execution of business process instances, however, mainly the area
of process mining [3] has yet provided a systematic analytical approach.
Visual analysis tools allow to leverage capabilities of the human cognitive
apparatus which is capable of pattern-based processing of perceived stimuli on
multiple levels of granularity in parallel. These kinds of analyses allow to gain
insight into process event data by a projection of data into appropriate percep-
tual spaces, and thus offer a complementary perspective on existing statistical
approaches, e. g., process mining techniques. There have been suggestions for
time-line based views on event data [4], but additional work into this direction
is missing up to now.
In the course of examining cognitive aspects of human process modeling [5,
6], we have collected a series of experimental data about the behavior of hu-
man modelers during the solving of specific modeling tasks. The data consists
of recorded modeling phases of type “Comprehension”, “Modeling”, and “Rec-
onciliation”. Each phase has an exact start time and end time. In order to gain
an analytical understanding from this data collection, it is important to apply
analysis techniques which allow to exploratively navigate through the available
data, rather than to perform statistical analyses that pre-suppose an underlying
structure of the data.
48 Designing Visual Decision Making Support with the Help of Eye-tracking
For the purpose of performing exploratory analyses on our data collection re-
sulting from process modeling experiments, we are currently examining different
visualization types with respect to their usefulness for our analysis purposes. In
this paper, we focus on the Rhythm-Eye [7] visualization as a versatile process
instance data analysis instrument (cf. Sect. 2.2). This visualization type uses a
circular visual structure for plotting process log data over time, and claims to
be potentially more cognitively efficient than a traditional linear time-line pro-
jection. We analyze its suitability for our analytical purposes with the help of
an eye-tracking experiment which gets qualitatively evaluated.
The following section introduces the Rhythm-Eye and the application con-
text. Sect. 3 describes our experiment. Initial findings are summarized in Sect. 4.
Related work is discussed in Sect. 5, and the final Sect. 6 concludes the paper.
2 Backgrounds
2.1 Process Behavior Analysis
The creation of a process model (also denoted as process of process modeling)
is an iterative process involving three phases [5], i. e., comprehension, modeling
and reconciliation, that can be combined in a flexible way (i. e., phases can occur
repeatably and phases can be skipped in different) [8]. During comprehension
phases the modeler understands the problem at hand and builds an internal
representation of it (i. e., a mental model). During modeling phases the modeler
interacts with the modeling tool in order to externalize the mental model and to
create an actual representation. Finally, reconciliation phases represent actions
aiming at improving the understandability of the model by changing the layout
of the modeling. To analyze the process of process modeling and to support pro-
cess behavior analysis Cheetah Experimental Platform has been developed that
allows to collect all interactions of a modeler with the modeling environment [9].
Analyzing the modeler’s interactions with the modeling environment allows to
infer the sequence of modeling phases. In general, creation and deletion inter-
actions (e. g., creating a new task on the modeling canvas, or deleting an edge
connecting two tasks) are classified as modeling phase. Interactions to rename
modeling elements and move elements usually characterize reconciliation phases.
Comprehension phases, in turn, are phases without any interaction between the
user and the modeling tool. Techniques for the automatic identification of mod-
eling phases are reported in the literature [5].
2.2 The Rhythm-Eye Visualization
A well-known visualization for time-related data is its projection onto a linear
time-line, which from left to right displays the progress of time. Horizontal po-
sitions on the time-line represent points in time, and sections represent phases
with a beginning and an end time. The Rhythm-Eye visualization [7] uses a cir-
cular representation to display the temporal progress of processes. Points in time
and phases are projected onto a ring structure, rather than onto a linear time-
line, according to their time of occurrence during process execution (cf. Figures
1 and 2). The circular ring structure is interrupted by a gap that separates the
2
Designing Visual Decision Making Support with the Help of Eye-tracking 49
start and end points of displayed processes. In the same way as multiple lanes
of a time-line projection can be placed one below each other, the Rhythm-Eye
visualization allows to nest multiple rings inside each other. With this circular
projection, it can be expected that rhythmic patterns in processes can be made
visible at a glance and better be compared across boundaries of process types
[7]. This assumption is based on the consideration, that a ring structure avoids
the perception of periphery areas at the very start and end of the projection
space. Hence, a more homogeneous impression of the distribution of events and
phases over time would be achieved. The ring projection can also use space on a
display device in a more compact and efficient way than a time-line projection.
Our implementation of the Rhythm-Eye visualization is embedded into an
experimentation environment which, among others, allows to configure the num-
ber of rings and their size parameters, as well as to assign data from different
sources to be projected onto the rings. The configuration allows for a free choice
of combinations of phase data from either the same experiment subject (intra-
subject analysis) or different ones (inter-subject analysis). Types of events and
phases can be filtered individually per ring. As a consequence, a variety of config-
urations becomes possible, in which multiple rings may be used to differentiate
between different types, or events and phases of the same types are projected
onto multiple rings for comparison.
For the purpose of analyzing the detected modeling phases of our experi-
ments, this means that the experimentation environment is able to deal with
diverse analytical questions that can be asked towards the existing data, e. g.,
questions that focus on temporal relationships of different modeling phases in one
particular experiment, versus questions that compare distributions of modeling
phases from multiple experiments. Since the configuration of the visualization
is performed dynamically and the resulting rendering is immediately shown,
the experimentation environment also allows for a seamless navigation between
these different analytical perspectives. For example, an exploration can begin
with an inter-subject analysis that compares phases of one particular kind with
each other, then the analyst decides to drill-down into the details of one specific
subject to compare the individual modeling phases of this experiment with each
other, and later widens the focus again by re-incorporating other subjects to
investigate a particular constellation discovered. This type of explorative navi-
gation resembles “slicing & dicing” techniques from Online Analytical Processing
(OLAP) approaches in the field of data warehouse analysis [10, 11].
In order to differentiate between the starting point of a process and its end,
not the whole 360° circle can be used, but a gap between the start and the end
is inserted to distinguish both sides of the displayed process. We have examined
two settings of the visual projection. The first one uses a symmetric gap around
the top-most 0°/ 360° angle with the size of 30° (see Figures 1b and 2a). The
second one uses a larger asymmetric gap of 90° to the left of the 0° / 360° angle,
i. e., effectively 270° of the circle are used to project a process, beginning from
the top-most 0° “north” angle, and reaching to the 270° “west” angle (see Figures
1a, 1c, and 2b).
3
50 Designing Visual Decision Making Support with the Help of Eye-tracking
3 Methodological Approach
To analyze the effectiveness of the Rhythm-Eye visualization in the context of
Process Behavior Analysis we conduced an eye tracking session where we asked
two students from the Technical University of Denmark that are familiar with
process modeling and were introduced to process behavior analysis to answer
three different analytical questions that are relevant in the context of process
behavior analysis.
Analytical Questions
Q1 Is there a long reconciliation phase at the end of the process?
Q2 Is the modeling done in rather short or large chunks throughout the process?
Q3 Is the comprehension behavior of the subject changing over time?
Question Q1 is relevant since it allows to differentiate between modelers that
exhibit long reconciliation phases at the end and modelers that perform contin-
uous reconciliation activities [5]. Long reconciliation phases at the end indicate
that the modeler was paying attention to the actual layout of the model (and
improvements thereof as last operation done), which suggests some attention
towards future readability. Question Q2 is chosen since, from research on the
process of process modeling, we know that subjects differ in terms of model-
ing chunk size [5]. While some modeler exhibit behavior where numerous short
modeling phases exist, other modeler are able to mentally construct larger por-
tions of the process and to externalize it as larger chunks. Finally, Q3 is relevant
to differ modelers with large initial comprehension phase followed by several
shorter phases suggesting they created a full mental model of the entire process
from modelers that immediately start modeling and have comprehension phases
distributed over the whole session [5].
Design. For each analytical questions we presented participants with 6 vi-
sualizations (3 for each configuration of the Rhythm-Eye) that all depicted the
modeling behavior of a single modeler. Overall, this led to 18 measurements for
each participant. At the end of the session we asked the participants for feed-
back and their perception regarding the two configurations of the Rhythm-Eye
visualization.
Research Questions. From a cognitively effective visualization we would ex-
pect that it allows the user of the visualization to focus on only those parts of
the information that is relevant for answering. In our setting all three analytical
questions have in common that they can be answered by focusing on a single
phase type, i.e., comprehension, modeling, or reconciliation. For example, for Q1
we would expect that a cognitively effective visualizations guides subjects to fo-
cus mostly on the ring of the Rhythm-Eye that projects the reconciliation phase.
Moreover, since the question is about the ending part of the modeling session the
focus should be concentrated there. In turn, if we would observe that the visual-
ization does not guide the attention of the user to the relevant parts, it cannot
be considered cognitively effective for answering the posed analytical questions.
This results into our first research question RQ1: Does the Rhythm-Eye vi-
sualization support to focus the user’s attention on the parts of the
visualization that are relevant for answering the questions?
4
Designing Visual Decision Making Support with the Help of Eye-tracking 51
Moreover, concerning the two configurations of the Rhythm-Eye we would
expect that configuration 2 guides the user better to find the starting point of
the modeling session. The fact that the representation resembles a clock starting
at 12 o’clock presumably supports the user in understanding that this coincides
with the starting point of the modeling session and that the visualization is
to be read in a clock-wise direction. With configuration 1, in turn, we expect
less guidance from the visual representation and more difficulties of the users
to orientate themselves. This results into our second research question RQ2:
Does configuration 2 provide better initial orientation for their user
in finding the starting point when compared to configuration 1?
Instrumentation. To collect the data for answering the two above mentioned
research questions the participants’ eye movements were tracked while answering
the analytical questions using a Tobii Pro TX300 eye tracker. For designing the
eye tracking session and for analysis of the eye tracking data Tobii Pro Studio
3.4 was used.
4 Initial Findings
Results Research Question 1. To answer research question RQ1 we considered
eye fixations. A fixation is a time frame, typically lasting for about 200 millisec-
onds up to several seconds, in which a subject’s gaze stops on a stimulus. For
this reason, fixations represent the amount of time a specific area caught the
attention of the subject [12]. From the collected fixation data we then created
heatmaps showing the fixations of both participants in an integrated manner (cf.
Fig. 1). The heatmaps display the sum of fixation times of all subjects, while
answering the analytical questions. In line with our expectations, when subjects
were answering Q1 their cumulative focus was clearly on the end of the session
and mostly on the reconciliation ring (cf. Fig. 1a). When subjects were answer-
ing Q2, their focus was distributed throughout the whole session and mostly
on the modeling ring (cf. Fig. 1b). Finally, when answering Q3 subjects’ gaze
was very focused on the comprehension ring (cf. Fig. 1c). These results were
consistent over all visualizations presented. Based on the heatmaps it can be
concluded that the Rhythm-Eye visualization supports the users to focus their
attention on those parts of the visualization that are relevant for answering the
questions (while ignoring less relevant parts), i.e., the visualization seems to be
cognitively effective. This is supported by the qualitative feedback provided by
the participants. They both perceived answering the analytical questions using
the visualizations as an easy task. Moreover, they pointed the clear differentia-
tion of information positively out (i.e., one modeling phase type for each ring,
plus the different color coding for each modeling phase).
Results Research Question 2. To answer RQ2 we compared the two configu-
rations described Section 2.2. For our analysis we used so called gaze plots that
show the time sequence of where a participant looked (cf. Fig. 2). In gaze plots,
fixations are numbered in ascending temporal order and connected with a line to
the subsequent fixation. Since RQ2 is concerned with the question whether con-
figuration 2 better guides users to the starting point, we focused in our analysis
5
52 Designing Visual Decision Making Support with the Help of Eye-tracking
(a) Sum of all fixation times (b) Sum of all fixation times (c) Sum of all fixations
while answering Q1. while answering Q2. times while answering Q3.
Fig. 1: Heat maps with sum of fixation times for all subjects and questions.
(a) Gaze plot of the first configuration. (b) Gaze plot of the second configuration.
Fig. 2: Gaze plots of the two visualization configurations. Each colored circle represents a fixation
point whereby the size indicates the fixation duration, the number in the center is the ordering of
the fixations (i. e., 1 represents the first fixation), and its background color represents the subject.
on the first few fixations in the gaze plot. Our data revealed that configuration 2
immediately guided both users to start looking at the area on the screen where
the modeling session starts, i.e., approx. 12 o’clock in the visualization and to
read from there the visualization clock-wise (cf. Fig. 2b). Configuration 1, in
turn, provided less guidance for the user on where to start reading the visualiza-
tion and we observed less consistent gaze plots when compared to configuration
2 (cf. Fig. 2a). In many cases, participants started their exploration of the vi-
sualization its upper left-hand side, before thy moved there focus to the right
where the modeling session starts. Overall, our results suggest that configura-
tion 2 better guides the user in to effectively read the visualization. This is also
supported by the qualitative feedback where one of the participants pointed out
that the clock metaphor appeared very effective to immediately recognize the
beginning and the end of each session.4
5 Related Work
Several publications with different backgrounds discuss visualizations as means
to provide cognitive support for understanding information, e. g., [13–15]. In our
work, we share the view that advantages of the use of visualizations stem from
4
Video recordings are available at http://bpm.q-e.at/gaze_plots_rhythm_eye.
6
Designing Visual Decision Making Support with the Help of Eye-tracking 53
cognitive capabilities of the human visual perception apparatus, which to ef-
ficiently leverage is a relevant research challenge. The relevance of a reflected
use of visualizations is also increasingly recognized in the field of Information
Systems [16, 17]. With respect to process-aware information systems (PAIS), re-
search has identified the demand to more strongly focus on providing analytical
means which can give insight into the characteristics of process-related data
[18, 19]. As one direction to follow, it is suggested to develop visual analysis
techniques which take in specific perspectives on data to fulfill business-relevant
information needs. Some analysis techniques of this kind have already been de-
veloped [4, 20]. The examination at hand follows this path of research. Means
for performing in-depth process data analyses currently mostly originate from
the field of Process Mining [21]. While up to now the focus in this field lies
on the analytical reconstruction of structural relationships between events using
statistical methods, it is also argued from out this research area, that model
reconstruction alone does not provide a comprehensive set of methods to gain
valuable insight into process execution knowledge [18].
In the context of analyzing the process of process modeling, the most widely
used visualization tool currently available is the Modeling Phase Diagram (MPD)
[22]. A MPD is a line chart where the x axis reports the time and the y axis
indicates the number of items in the modeling canvas. To indicate the different
modeling phases, corresponding fractions of the line are shaped according to
different patterns (e.g., filled black for modeling, filled gray for reconciliation,
dotted black for comprehension).
6 Conclusion and Future Work
In this paper we analyzed the effectiveness of the Rhythm-Eye visualization
in the context of process modeling behavior analysis. We could demonstrate
that the Rhythm-Eye visualization is an effective mean for analyzing analyti-
cal questions on process modeling behavior. We could also show that different
configurations of the Rhythm-Eye differ in guiding users to the starting point of
the session to be analyzed. This work-in-progress is a first step in an attempt
to inform the design of visualizations for process modeling behavior through
neuro-physiological measurements and the analysis of eye movements.
The findings reported in this paper are subject to several limitations: they
are based on a very small sample size, i.e., only two participants. This could
be mitigated by performing repeated measurements. However, additional data
collection is certainly needed to increase external validity in terms of generaliz-
ability of results. Another limitation is that all three analytical questions in our
setting can be answered by focusing on a single phase type. While such questions
are certainly relevant for process behavior analysis, they only constitute a sub-
set of relevant questions. In the future we plan to look into questions that need
to integrate information from several phases and comparisons of the behavior
of different subjects. Another limitation is the focus on a single visualization.
While this was sufficient to demonstrate its usefulness for answering analytical
questions, a comparison with other visualizations for analysing process modeling
7
54 Designing Visual Decision Making Support with the Help of Eye-tracking
behavior is needed. As for future work we plan to systematically compare the
Rhythm-Eye projection with MPD and the piano roll projection (cf. Sect. 5).
References
1. Weske, M.: Business Process Management: Concepts, Languages, Architectures.
second edition edn. Springer, Berlin Heidelberg (2012)
2. Mendlng, J.: Metrics for Process Models: Empirical Foundations of Verification,
Error Prediction, and Guidelines for Correctness. Springer (2008)
3. van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement
of Business Processes. Springer, Berlin Heidelberg (2011)
4. Song, M., van der Aalst, W.M.P.: Supporting process mining by showing events
at a glance. In Chari, K., Kumar, A., eds.: Annual Workshop on Information
Technologies and Systems, Social Science Research Network (2007) 139–145
5. Pinggera, J.: The Process of Process Modeling. PhD thesis, University of Inns-
bruck, Department of Computer Science (2014)
6. Burattin, A., Kaiser, M., Neurauter, M., Weber, B.: Eye Tracking Meets the
Process of Process Modeling: a Visual Analytic Approach. In: TAProViz. (2016)
7. Gulden, J.: Visually comparing process dynamics with rhythm-eye views. In
Ross Brown, S.K., ed.: TaProViz, Springer (2016)
8. Pinggera, J., Soffer, P., Fahland, D., Weidlich, M., Zugal, S., Weber, B., Reijers,
H., Mendling, J.: Styles in business process modeling: an exploration and a model.
Software & Systems Modeling 14(3) (2015) 1055–1080
9. Pinggera, J., Zugal, S., Weber, B.: Investigating the Process of Process Modeling
with Cheetah Experimental Platform. In: Proc. ER-POIS ’10. (2010) 13–18
10. Codd, E. F., C.S.B.S.C.T.: Providing olap (on-line analytical processing) to user
analysts: An it mandate. (1993)
11. Berson, A., Smith, S.J.: Data Warehousing, Data Mining, & OLAP. McGraw-Hill,
New York (2004)
12. Riedl, R., Léger, P.M.: Fundamentals of NeuroIS. Springer (2016)
13. Chen, C.: Information visualization. Springer, London (2010)
14. Cairo, A.: The Functional Art. Voices That Matter. Pearson Education (2010)
15. Whitney, H.: Data Insights: New Ways to Visualize and Make Sense of Data.
Morgan Kaufman, Waltham (2013)
16. Gulden, J., Reijers, H.A.: Toward advanced visualization techniques for conceptual
modeling. In: CAiSE Forum, CEUR (2015)
17. Gulden, J., van der Linden, D., Aysolmaz, B.: Requirements for research on visu-
alizations in information systems engineering. In: ENASE Conference. (2016)
18. van der Aalst, W.M.P., de Leoni, M., ter Hofstede, A.H.: Process mining and
visual analytics: Breathing life into business process models. In Floares, A., ed.:
Computational Intelligence. Nova Science Publishers, New York (2011)
19. Mendling, J., Recker, J.: The state of the art of business process management
research. Business & Information Systems Engineering (BISE) 58(1) (2016) 55–72
20. Begole, J., Tang, J.C., Smith, R.B., Yankelovich, N.: Work rhythms: Analyzing
visualizations of awareness histories of distributed groups. In: CSCW, ACM (2002)
21. Aalst, W.M.v.d., Desel, J., Oberweis, A., eds.: Business Process Management:
Models, Techniques, and Empirical Studies. Springer, Berlin Heidelberg (2000)
22. Pinggera, J., Zugal, S., Weidlich, M., Fahland, D., Weber, B., Mendling, J., Reijers,
H.: Tracing the Process of Process Modeling with Modeling Phase Diagrams. In:
Proc. ER-BPM ’11. (2012) 370–382
Acknowledgments: This work is partially funded by the Austrian Science Fund project “The Mod-
eling Mind: Behavior Patterns in Process Modeling” (P26609).
8