Identifying Cohorts that Di er in their Behaviour: Tool Support

Abstract

Process mining is a specialised form of data analytics that aims to provide data-driven improvement recommendations, derived from event logs. These event logs contain information about the execution of real-world processes, which may be complex. Cohort identi cation recommends drill-down lters for process mining, based on di erences in process. In this paper, we describe its integration in three process mining tools: as a stand-alone ProM plug-in, as part of the visual Miner and (planned) as part of Course Insights.

Keywords

• Process Mining
• Feature Selection
• Filter Recommendation
• Stochastic Comparative Process Mining

-

Process mining, a specialised form of data analytics, provides techniques using which analysts can extract insights from recorded process behaviour in event logs. The insights are used to provide data-driven recommendations to improve business operations. Many real-life processes are complex in nature, and studying their process models is challenging [ 1 ]. Process mining techniques to deal with this complexity include ltering, slicing and dicing, and process cubes.

Cohort identi cation aims to identify and recommend sub-sets of the traces in the log (cohort s). These cohorts are de ned by attributes of traces (e.g. \claim amount", \gender" or \country"), and cohort identi cation recommends the attributes and values such that the traces that have the attribute and value (cohort) di er as much as possible from the traces that do not have the attribute or have a di erent value (anti-cohort) in terms of the process that is being followed, which includes the order of steps taken for traces, and how often di erent sequences appeared in the (anti-)cohort. The di erence between the cohort and the anti-cohort is expressed as a distance measure [ 4 ], where 1 means that their processes are completely di erent (i.e. no activity appears in both), and 0 means that their processes are no more di erent than a random division of the combined log.

That is, cohort identi cation nds groups of cases in event logs that follow a process that is di erent than the process of other groups. For instance, consider the following log consisting of 1000 cases of customers purchasing online and registering for an account, in which each trace is annotated with whether the customer was a Silver or Gold customer, from an East or West branch: [hregister; purchasei2S0E0, hregister; purchasei1S0W0 , hregister; purchasei5G0E , hregister; purchasei1G0W0 , hpurchase; registeri1S0E0, hpurchase; registeri5S0W , hpurchase; registeri2G0E0 , hpurchase; registeri1G0W0 ]. In this log, Gold customers executed the trace variant hregister; purchasei 145500 = 13 times, while for the other customers this is 1405000 = 0:45 times. Thus, the likelihood that Gold customers rst register is lower than for other customers. Cohort identi cation would assess this for all potential combinations of attributes and values, and provide a ranked list based on a quanti cation of such di erences.

Cohort identi cation has similar goals as other process comparison techniques such as trace clustering [ 10 ], concept drift detection [ 7 ], and event attribute clustering [ 2 ], however provides better explainable results: the output is a list of attribute-value pairs that denote sub-logs of interest. The details of our cohort identi cation technique are described in [ 6 ].

In this paper, we describe the integration of cohort identi cation into three existing open source data intelligence tools: as a plug-in of the ProM framework (Section 2), as an extension of the process mining tool visual Miner (Section 3), and as an extension of the learning analytics dashboard Course Insights (Section 4). The rst tool demonstrates the use of cohort identi cation as a stand alone technique, the second tool depicts the use of cohort identi cation in conjunction with other process mining techniques, and the third tool illustrates the embedding of cohort identi cation in a learning analytics context. A screen- Fig. 1: Cohort Identi cation in ProM. cast is available at3. 2

Stand-alone Plug-in of ProM

The ProM framework [ 3 ] is a state-of-art open-source process mining framework aimed at practitioners and academics. Cohort identi cation has been implemented as a plug-in of ProM enabling practitioners and academics to access the technique. Upon starting with as input an event log, two parameters can be set: the attribute that determines the activity being executed, and the maximum number of trace attributes that is exhaustively considered. Upon completion, 3 https://vimeo.com/442323972

rst a diversi ed set of cohorts is shown (with attribute, value, cohort size, and distance between the cohort and anti-cohort). Figure 1 shows a screenshot. Implementation. The implementation is exible, as it provides extension points to (1) elicit attribute value ranges, (2) measure distance between events, traces and logs, and (3) truncate cohorts based on size or other aspects. Furthermore, the implementation is multithreaded and for pruning stores a Map entry, an int[] and an AtomicBoolean for each attribute value range combination. Maturity & How to Access. Cohort identi cation is open source and is part of the ProM 6.10 release; see4. The plug-in has been successfully applied in three case studies [ 6 ]. 3

visual Miner

The visual Miner (vM) [ 5 ] is an existing process mining tool that enables endusers to combine advanced academic process mining techniques in an industrycapable and user-friendly package. The input of vM is an event log. First, vM applies a process discovery technique to the log to obtain a process model. Second, vM applies a conformance checking technique and visualises the di erences between log and the discovered model. Third, it computes detailed frequency and performance information, and visualises this on the model, amongst others using animation. Fourth, it allows the user to drill down and focus on parts of the event log that are of interest, by applying one of several lters. Settings to any of these techniques and lters can be changed at any time, and vM will update and redo the necessary steps automatically [ 5 ]. A screenshot is shown in Figure 2; for a complete overview of vM's features, please refer to5. Cohort Identi cation (new). While the vM makes it easy to drill down into parts of the log or process of particular interest, it was up to the user to manually analyse the visualisation available to discover potentially interesting parts.

4 http://promtools.org

5 http://leemans.ch/publications/ivmProM6.10.pdf

Selected Attributes Program, Residential Status

Minimum Coverage: 20%

Recommendations 2 Program = Tourism Program = Engineering and Residential Status = Domestic

Cohort identi cation suggests lters on attributes on the traces in an event log, such that applying the lter leads to the largest di erences in process. Figure 2 shows a screenshot of the integration of cohort identi cation in vM: the cohorts are computed automatically in the background, and the result is shown to the user. The rst column shows the trace attribute of the cohort, the second column the values of the attribute that are in the cohort, the third column the number of traces in the cohort, while the last column shows the distance between the cohort and the anti-cohort. Using a click (for the cohort) or shift+click (for the anti-cohort) one can quickly lter down the event log to the corresponding traces, in order to study the di erences in process in more detail. The identi ed cohorts can be exported to an Excel document for further analysis. Embedding cohort identi cation in vM makes it easy for end-users to conduct detailed process mining analysis using one plug-in.

Maturity & How to Access. vM has been applied to many process mining projects by industry partners and academics (see 6 for an overview). Cohort identi cation has been added in April 2020 and has been successfully applied in three case studies [ 6 ]. Visual Miner is open source, part of the ProM framework [ 3 ] and can be downloaded from6. 4

Course Insights

Course Insights is an instructor-facing learning analytics dashboard, developed at the University of Queensland, that empowers course coordinators to gain insights and act on student data to enhance student learning and experience across the course life-cycle at scale. It collates student data from a variety of learning systems and sources and displays it to instructors all in one simple and easy to use interface. An essential element is its comparative analysis functionality, which enables course coordinators to use lters to compare and contrast di erent student groups based on their demographics, enrolment, engagement and performance data. An observational study that analysed how the lters were used by 71 sta members found that commonly only a small subset of the features was used and lters were rarely applied on top of one another [ 8 ]. To overcome this challenge, we are implementing cohort identi cation in Course Insights to recommend insightful lters to instructors [ 9 ].

6 http://visualminer.org

Cohort Identi cation (planned). Figure 3 illustrates the proposed presentation of lter recommendations to instructors, including the attributes and values, coverage (fraction of students covered by the lter), and distance (insightfullness of the lter).

Maturity & How to Access. A case study based on data from a course with 875 students, with high demographic and educational diversity has explored the potential bene ts of applying cohort identi cation to Course Insights [ 9 ]. The cohort identi cation is planned to be implemented in Course Insights, which can be accessed via7. A remaining challenge is to make Course Insights fully process aware: with cohort identi cation, users can drill down into sub-groups with di erences in their process, however more support to study these di erences is necessary.

5 Conclusion

In this paper, we described how cohort identi cation, which recommends trace-attribute-based lters to maximise the di erences between processes, is implemented as a stand-alone ProM plug-in, is integrated in the visual Miner, and is being integrated in Course Insights. The technique lters sub-logs of traces (cohorts) de ned by trace attribute value ranges (features) to compare behavioural di erences in cohorts or to drill down into a particular cohort. The technique can be used to understand the di erences between two cohorts and answer questions related to a particular cohort. Cohort identi cation can be applied in reasonable time to event logs. In the future, we intend to focus implementing process infrastructure in Course Insights and on automated comparison techniques to compare the identi ed cohorts.

7 https://analytics.itali.uq.edu.au/dev/insights

References

1. van der Aalst , W.M.P. : Process Mining - Data Science in Action ( 2016 )

2. Bolt , A., van der Aalst , W.M.P., de Leoni, M.: Finding process variants in event logs (short paper) . In: CoopIS . vol. 10573 , pp. 45 { 52 ( 2017 )

3. van Dongen, B.F. , et al.: The ProM framework: A new era in process mining tool support . In: Petri Nets . pp. 444 { 454 ( 2005 )

4. Leemans , S.J.J. , Syring , A.F. , van der Aalst , W.M.P. : Earth movers' stochastic conformance checking . In: BPM forum . pp. 127 { 143 ( 2019 )

5. Leemans , S.J.J. , et al.: Process and deviation exploration with Inductive visual Miner. In: BPM demos . vol. 1295 , p. 46 . CEUR-WS.org ( 2014 )

6. Leemans , S.J.J. , et al.: Identifying cohorts: Recommending drill-downs based on di erences in behaviour for process mining . In: ER ( 2020 )

7. Maaradji , A. , Dumas , M. , Rosa , M.L. , Ostovar , A. : Detecting sudden and gradual drifts in business processes from execution traces . TKDE 29 ( 10 ), 2140 { 2154 ( 2017 )

8. Shabaninejad , S. , et al.: Automated insightful drill-down recommendations for learning analytics dashboards . In: LAK . p. 41 { 46 ( 2020 )

9. Shabaninejad , S. , et al.: Recommending insightful drill-downs based on learning processes for learning analytics dashboards . In: AIED . pp. 486 { 499 ( 2020 )

10. Weerdt , J.D., vanden Broucke, S.K.L.M. , Vanthienen , J. , Baesens , B. : Active trace clustering for improved process discovery . TKDE 25 ( 12 ), 2708 { 2720 ( 2013 )