Designing Visual Decision Making Support with the Help of Eye Tracking

Abstract

Data visualizations are helpful tools to cognitively access large amounts of data and make complex relationships in data understandable. This paper shows how results from neuro-physiological measurements, more speci cally eye-tracking, can support justi ed design decisions about improving existing data visualizations for exploring process execution data. This is achieved by gaining insight into how visualizations are used for decision-making. The presented examination is embedded in the domain of process modeling behavior analysis, and the analyses are performed on the background of representative analytical questions from the domain of process model behavior analysis. We present initial ndings on one out of three visualization types we have examined, which is the Rhythm-Eye visualization.

Introduction

Research on business process modeling has evolved a wide variety of re ections on characteristics of process models with regard to their structural and dynamic properties, their execution semantics, and their expressiveness [ 1, 2 ]. With respect to the execution of business process instances, however, mainly the area of process mining [ 3 ] has yet provided a systematic analytical approach.

Visual analysis tools allow to leverage capabilities of the human cognitive apparatus which is capable of pattern-based processing of perceived stimuli on multiple levels of granularity in parallel. These kinds of analyses allow to gain insight into process event data by a projection of data into appropriate perceptual spaces, and thus o er a complementary perspective on existing statistical approaches, e. g., process mining techniques. There have been suggestions for time-line based views on event data [ 4 ], but additional work into this direction is missing up to now.

In the course of examining cognitive aspects of human process modeling [ 5, 6 ], we have collected a series of experimental data about the behavior of human modelers during the solving of speci c modeling tasks. The data consists of recorded modeling phases of type \Comprehension", \Modeling", and \Reconciliation". Each phase has an exact start time and end time. In order to gain an analytical understanding from this data collection, it is important to apply analysis techniques which allow to exploratively navigate through the available data, rather than to perform statistical analyses that pre-suppose an underlying structure of the data. For the purpose of performing exploratory analyses on our data collection resulting from process modeling experiments, we are currently examining di erent visualization types with respect to their usefulness for our analysis purposes. In this paper, we focus on the Rhythm-Eye [ 7 ] visualization as a versatile process instance data analysis instrument (cf. Sect. 2.2). This visualization type uses a circular visual structure for plotting process log data over time, and claims to be potentially more cognitively e cient than a traditional linear time-line projection. We analyze its suitability for our analytical purposes with the help of an eye-tracking experiment which gets qualitatively evaluated.

The following section introduces the Rhythm-Eye and the application context. Sect. 3 describes our experiment. Initial ndings are summarized in Sect. 4. Related work is discussed in Sect. 5, and the nal Sect. 6 concludes the paper. 2 2.1

Backgrounds

Process Behavior Analysis

The creation of a process model (also denoted as process of process modeling) is an iterative process involving three phases [ 5 ], i. e., comprehension, modeling and reconciliation, that can be combined in a exible way (i. e., phases can occur repeatably and phases can be skipped in di erent) [ 8 ]. During comprehension phases the modeler understands the problem at hand and builds an internal representation of it (i. e., a mental model). During modeling phases the modeler interacts with the modeling tool in order to externalize the mental model and to create an actual representation. Finally, reconciliation phases represent actions aiming at improving the understandability of the model by changing the layout of the modeling. To analyze the process of process modeling and to support process behavior analysis Cheetah Experimental Platform has been developed that allows to collect all interactions of a modeler with the modeling environment [ 9 ]. Analyzing the modeler's interactions with the modeling environment allows to infer the sequence of modeling phases. In general, creation and deletion interactions (e. g., creating a new task on the modeling canvas, or deleting an edge connecting two tasks) are classi ed as modeling phase. Interactions to rename modeling elements and move elements usually characterize reconciliation phases. Comprehension phases, in turn, are phases without any interaction between the user and the modeling tool. Techniques for the automatic identi cation of modeling phases are reported in the literature [ 5 ]. 2.2

The Rhythm-Eye Visualization

A well-known visualization for time-related data is its projection onto a linear time-line, which from left to right displays the progress of time. Horizontal positions on the time-line represent points in time, and sections represent phases with a beginning and an end time. The Rhythm-Eye visualization [ 7 ] uses a circular representation to display the temporal progress of processes. Points in time and phases are projected onto a ring structure, rather than onto a linear timeline, according to their time of occurrence during process execution (cf. Figures 1 and 2). The circular ring structure is interrupted by a gap that separates the start and end points of displayed processes. In the same way as multiple lanes of a time-line projection can be placed one below each other, the Rhythm-Eye visualization allows to nest multiple rings inside each other. With this circular projection, it can be expected that rhythmic patterns in processes can be made visible at a glance and better be compared across boundaries of process types [ 7 ]. This assumption is based on the consideration, that a ring structure avoids the perception of periphery areas at the very start and end of the projection space. Hence, a more homogeneous impression of the distribution of events and phases over time would be achieved. The ring projection can also use space on a display device in a more compact and e cient way than a time-line projection.

Our implementation of the Rhythm-Eye visualization is embedded into an experimentation environment which, among others, allows to con gure the number of rings and their size parameters, as well as to assign data from di erent sources to be projected onto the rings. The con guration allows for a free choice of combinations of phase data from either the same experiment subject (intrasubject analysis) or di erent ones (inter-subject analysis). Types of events and phases can be ltered individually per ring. As a consequence, a variety of con gurations becomes possible, in which multiple rings may be used to di erentiate between di erent types, or events and phases of the same types are projected onto multiple rings for comparison.

For the purpose of analyzing the detected modeling phases of our experiments, this means that the experimentation environment is able to deal with diverse analytical questions that can be asked towards the existing data, e. g., questions that focus on temporal relationships of di erent modeling phases in one particular experiment, versus questions that compare distributions of modeling phases from multiple experiments. Since the con guration of the visualization is performed dynamically and the resulting rendering is immediately shown, the experimentation environment also allows for a seamless navigation between these di erent analytical perspectives. For example, an exploration can begin with an inter-subject analysis that compares phases of one particular kind with each other, then the analyst decides to drill-down into the details of one speci c subject to compare the individual modeling phases of this experiment with each other, and later widens the focus again by re-incorporating other subjects to investigate a particular constellation discovered. This type of explorative navigation resembles \slicing & dicing" techniques from Online Analytical Processing (OLAP) approaches in the eld of data warehouse analysis [ 10, 11 ].

In order to di erentiate between the starting point of a process and its end, not the whole 360° circle can be used, but a gap between the start and the end is inserted to distinguish both sides of the displayed process. We have examined two settings of the visual projection. The rst one uses a symmetric gap around the top-most 0°/ 360° angle with the size of 30° (see Figures 1b and 2a). The second one uses a larger asymmetric gap of 90° to the left of the 0° / 360° angle, i. e., e ectively 270° of the circle are used to project a process, beginning from the top-most 0° \north" angle, and reaching to the 270° \west" angle (see Figures 1a, 1c, and 2b).

Methodological Approach

To analyze the e ectiveness of the Rhythm-Eye visualization in the context of Process Behavior Analysis we conduced an eye tracking session where we asked two students from the Technical University of Denmark that are familiar with process modeling and were introduced to process behavior analysis to answer three di erent analytical questions that are relevant in the context of process behavior analysis.

Analytical Questions Q1 Is there a long reconciliation phase at the end of the process? Q2 Is the modeling done in rather short or large chunks throughout the process? Q3 Is the comprehension behavior of the subject changing over time?

Question Q1 is relevant since it allows to di erentiate between modelers that exhibit long reconciliation phases at the end and modelers that perform continuous reconciliation activities [ 5 ]. Long reconciliation phases at the end indicate that the modeler was paying attention to the actual layout of the model (and improvements thereof as last operation done), which suggests some attention towards future readability. Question Q2 is chosen since, from research on the process of process modeling, we know that subjects di er in terms of modeling chunk size [ 5 ]. While some modeler exhibit behavior where numerous short modeling phases exist, other modeler are able to mentally construct larger portions of the process and to externalize it as larger chunks. Finally, Q3 is relevant to di er modelers with large initial comprehension phase followed by several shorter phases suggesting they created a full mental model of the entire process from modelers that immediately start modeling and have comprehension phases distributed over the whole session [ 5 ].

Design. For each analytical questions we presented participants with 6 visualizations (3 for each con guration of the Rhythm-Eye) that all depicted the modeling behavior of a single modeler. Overall, this led to 18 measurements for each participant. At the end of the session we asked the participants for feedback and their perception regarding the two con gurations of the Rhythm-Eye visualization.

Research Questions. From a cognitively e ective visualization we would expect that it allows the user of the visualization to focus on only those parts of the information that is relevant for answering. In our setting all three analytical questions have in common that they can be answered by focusing on a single phase type, i.e., comprehension, modeling, or reconciliation. For example, for Q1 we would expect that a cognitively e ective visualizations guides subjects to focus mostly on the ring of the Rhythm-Eye that projects the reconciliation phase. Moreover, since the question is about the ending part of the modeling session the focus should be concentrated there. In turn, if we would observe that the visualization does not guide the attention of the user to the relevant parts, it cannot be considered cognitively e ective for answering the posed analytical questions. This results into our rst research question RQ1: Does the Rhythm-Eye visualization support to focus the user's attention on the parts of the visualization that are relevant for answering the questions?

Moreover, concerning the two con gurations of the Rhythm-Eye we would expect that con guration 2 guides the user better to nd the starting point of the modeling session. The fact that the representation resembles a clock starting at 12 o'clock presumably supports the user in understanding that this coincides with the starting point of the modeling session and that the visualization is to be read in a clock-wise direction. With con guration 1, in turn, we expect less guidance from the visual representation and more di culties of the users to orientate themselves. This results into our second research question RQ2: Does con guration 2 provide better initial orientation for their user in nding the starting point when compared to con guration 1?

Instrumentation. To collect the data for answering the two above mentioned research questions the participants' eye movements were tracked while answering the analytical questions using a Tobii Pro TX300 eye tracker. For designing the eye tracking session and for analysis of the eye tracking data Tobii Pro Studio 3.4 was used. 4

Initial Findings

Results Research Question 1. To answer research question RQ1 we considered eye xations. A xation is a time frame, typically lasting for about 200 milliseconds up to several seconds, in which a subject's gaze stops on a stimulus. For this reason, xations represent the amount of time a speci c area caught the attention of the subject [ 12 ]. From the collected xation data we then created heatmaps showing the xations of both participants in an integrated manner (cf. Fig. 1). The heatmaps display the sum of xation times of all subjects, while answering the analytical questions. In line with our expectations, when subjects were answering Q1 their cumulative focus was clearly on the end of the session and mostly on the reconciliation ring (cf. Fig. 1a). When subjects were answering Q2, their focus was distributed throughout the whole session and mostly on the modeling ring (cf. Fig. 1b). Finally, when answering Q3 subjects' gaze was very focused on the comprehension ring (cf. Fig. 1c). These results were consistent over all visualizations presented. Based on the heatmaps it can be concluded that the Rhythm-Eye visualization supports the users to focus their attention on those parts of the visualization that are relevant for answering the questions (while ignoring less relevant parts), i.e., the visualization seems to be cognitively e ective. This is supported by the qualitative feedback provided by the participants. They both perceived answering the analytical questions using the visualizations as an easy task. Moreover, they pointed the clear di erentiation of information positively out (i.e., one modeling phase type for each ring, plus the di erent color coding for each modeling phase).

Results Research Question 2. To answer RQ2 we compared the two con gurations described Section 2.2. For our analysis we used so called gaze plots that show the time sequence of where a participant looked (cf. Fig. 2). In gaze plots, xations are numbered in ascending temporal order and connected with a line to the subsequent xation. Since RQ2 is concerned with the question whether conguration 2 better guides users to the starting point, we focused in our analysis (a) Sum of all xation times (b) Sum of all xation times (c) Sum of all xations while answering Q1. while answering Q2. times while answering Q3. on the rst few xations in the gaze plot. Our data revealed that con guration 2 immediately guided both users to start looking at the area on the screen where the modeling session starts, i.e., approx. 12 o'clock in the visualization and to read from there the visualization clock-wise (cf. Fig. 2b). Con guration 1, in turn, provided less guidance for the user on where to start reading the visualization and we observed less consistent gaze plots when compared to con guration 2 (cf. Fig. 2a). In many cases, participants started their exploration of the visualization its upper left-hand side, before thy moved there focus to the right where the modeling session starts. Overall, our results suggest that con guration 2 better guides the user in to e ectively read the visualization. This is also supported by the qualitative feedback where one of the participants pointed out that the clock metaphor appeared very e ective to immediately recognize the beginning and the end of each session.4 5

Related Work

Several publications with di erent backgrounds discuss visualizations as means to provide cognitive support for understanding information, e. g., [13{15]. In our work, we share the view that advantages of the use of visualizations stem from 4 Video recordings are available at http://bpm.q-e.at/gaze_plots_rhythm_eye. cognitive capabilities of the human visual perception apparatus, which to efciently leverage is a relevant research challenge. The relevance of a re ected use of visualizations is also increasingly recognized in the eld of Information Systems [ 16, 17 ]. With respect to process-aware information systems (PAIS), research has identi ed the demand to more strongly focus on providing analytical means which can give insight into the characteristics of process-related data [ 18, 19 ]. As one direction to follow, it is suggested to develop visual analysis techniques which take in speci c perspectives on data to ful ll business-relevant information needs. Some analysis techniques of this kind have already been developed [ 4, 20 ]. The examination at hand follows this path of research. Means for performing in-depth process data analyses currently mostly originate from the eld of Process Mining [ 21 ]. While up to now the focus in this eld lies on the analytical reconstruction of structural relationships between events using statistical methods, it is also argued from out this research area, that model reconstruction alone does not provide a comprehensive set of methods to gain valuable insight into process execution knowledge [ 18 ].

In the context of analyzing the process of process modeling, the most widely used visualization tool currently available is the Modeling Phase Diagram (MPD) [ 22 ]. A MPD is a line chart where the x axis reports the time and the y axis indicates the number of items in the modeling canvas. To indicate the di erent modeling phases, corresponding fractions of the line are shaped according to di erent patterns (e.g., lled black for modeling, lled gray for reconciliation, dotted black for comprehension). 6

Conclusion and Future Work

In this paper we analyzed the e ectiveness of the Rhythm-Eye visualization in the context of process modeling behavior analysis. We could demonstrate that the Rhythm-Eye visualization is an e ective mean for analyzing analytical questions on process modeling behavior. We could also show that di erent con gurations of the Rhythm-Eye di er in guiding users to the starting point of the session to be analyzed. This work-in-progress is a rst step in an attempt to inform the design of visualizations for process modeling behavior through neuro-physiological measurements and the analysis of eye movements.

The ndings reported in this paper are subject to several limitations: they are based on a very small sample size, i.e., only two participants. This could be mitigated by performing repeated measurements. However, additional data collection is certainly needed to increase external validity in terms of generalizability of results. Another limitation is that all three analytical questions in our setting can be answered by focusing on a single phase type. While such questions are certainly relevant for process behavior analysis, they only constitute a subset of relevant questions. In the future we plan to look into questions that need to integrate information from several phases and comparisons of the behavior of di erent subjects. Another limitation is the focus on a single visualization. While this was su cient to demonstrate its usefulness for answering analytical questions, a comparison with other visualizations for analysing process modeling behavior is needed. As for future work we plan to systematically compare the Rhythm-Eye projection with MPD and the piano roll projection (cf. Sect. 5). Acknowledgments: This work is partially funded by the Austrian Science Fund project \The Modeling Mind: Behavior Patterns in Process Modeling" (P26609).

References

1. Weske , M. : Business Process Management: Concepts, Languages, Architectures. second edition edn . Springer, Berlin Heidelberg ( 2012 )

2. Mendlng , J.: Metrics for Process Models: Empirical Foundations of Veri cation, Error Prediction, and Guidelines for Correctness . Springer ( 2008 )

3. van der Aalst , W.M.P. : Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer, Berlin Heidelberg ( 2011 )

4. Song , M., van der Aalst , W.M.P. : Supporting process mining by showing events at a glance . In Chari, K. , Kumar , A., eds. : Annual Workshop on Information Technologies and Systems , Social Science Research Network ( 2007 ) 139 { 145

5. Pinggera , J.: The Process of Process Modeling . PhD thesis , University of Innsbruck, Department of Computer Science ( 2014 )

6. Burattin , A. , Kaiser , M. , Neurauter , M. , Weber , B. : Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach . In: TAProViz. ( 2016 )

7. Gulden , J.: Visually comparing process dynamics with rhythm-eye views . In Ross Brown, S.K., ed.: TaProViz , Springer ( 2016 )

8. Pinggera , J. , So er, P., Fahland , D. , Weidlich , M. , Zugal , S. , Weber , B. , Reijers , H. , Mendling , J.: Styles in business process modeling: an exploration and a model . Software & Systems Modeling 14 ( 3 ) ( 2015 ) 1055 { 1080

9. Pinggera , J. , Zugal , S. , Weber , B. : Investigating the Process of Process Modeling with Cheetah Experimental Platform . In: Proc. ER-POIS '10 . ( 2010 ) 13 { 18

10. Codd , E. F. , C.S.B.S .C.T.: Providing olap (on-line analytical processing) to user analysts: An it mandate . ( 1993 )

11. Berson , A. , Smith , S.J. : Data Warehousing, Data Mining, & OLAP . McGraw-Hill , New York ( 2004 )

12. Riedl , R. , Leger , P.M. : Fundamentals of NeuroIS. Springer ( 2016 )

13. Chen , C. : Information visualization. Springer, London ( 2010 )

14. Cairo , A. : The Functional Art. Voices That Matter . Pearson Education ( 2010 )

15. Whitney , H. : Data Insights: New Ways to Visualize and Make Sense of Data . Morgan Kaufman, Waltham ( 2013 )

16. Gulden , J. , Reijers , H.A. : Toward advanced visualization techniques for conceptual modeling . In: CAiSE Forum , CEUR ( 2015 )

17. Gulden , J., van der Linden, D. , Aysolmaz , B. : Requirements for research on visualizations in information systems engineering . In: ENASE Conference. ( 2016 )

18. van der Aalst , W.M.P., de Leoni, M., ter Hofstede , A.H. : Process mining and visual analytics: Breathing life into business process models . In Floares, A., ed.: Computational Intelligence . Nova Science Publishers, New York ( 2011 )

19. Mendling , J. , Recker , J.: The state of the art of business process management research . Business & Information Systems Engineering (BISE) 58(1) ( 2016 ) 55 { 72

20. Begole , J. , Tang , J.C. , Smith , R.B. , Yankelovich , N.: Work rhythms: Analyzing visualizations of awareness histories of distributed groups . In: CSCW , ACM ( 2002 )

21. Aalst , W.M.v.d., Desel , J. , Oberweis , A., eds.: Business Process Management: Models, Techniques, and Empirical Studies . Springer, Berlin Heidelberg ( 2000 )

22. Pinggera , J. , Zugal , S. , Weidlich , M. , Fahland , D. , Weber , B. , Mendling , J. , Reijers , H.: Tracing the Process of Process Modeling with Modeling Phase Diagrams . In: Proc. ER-BPM '11 . ( 2012 ) 370 { 382