=Paper=
{{Paper
|id=Vol-2703/paperDC7
|storemode=property
|title=Automated Segmentation of User Interface Logs Using Trace Alignment Techniques (Extended Abstract)
|pdfUrl=https://ceur-ws.org/Vol-2703/paperDC7.pdf
|volume=Vol-2703
|authors=Simone Agostinelli
|dblpUrl=https://dblp.org/rec/conf/icpm/Agostinelli20
}}
==Automated Segmentation of User Interface Logs Using Trace Alignment Techniques (Extended Abstract)==
https://ceur-ws.org/Vol-2703/paperDC7.pdf
Automated Segmentation of User Interface Logs
Using Trace Alignment Techniques
(Extended Abstract)
Simone Agostinelli
Sapienza Università di Roma, Rome, Italy
agostinelli@diag.uniroma1.it
Abstract—Robotic Process Automation (RPA) is a fast- user actions associated with the routine itself in well bounded
emerging automation technology that allows organizations to routine traces. A routine trace represents an execution instance
automate high volume routines. RPA tools are able to capture of a routine within a UI log. To be more precise, starting
in dedicated User Interface (UI) logs the execution of routines
previously performed by a human user on the UI of a computer from a UI log previously recorded by a RPA tool and an
system, and then emulate their enactment in place of the user interaction model representing the expected behaviour of a
by means of a software (SW) robot. The issue to automatically routine performed during an interaction session with the UI,
understand which user actions contribute to a specific routine we propose to leverage trace alignment in Process Mining [4]
inside the UI log is also known as segmentation. The proposed to automatically identify and extract the routine traces by the
research investigates how to leverage trace alignment techniques
in Process Mining to automatically derive the boundaries of a UI log. Such traces are finally stored in a dedicated routine-
routine by analyzing the UI log that keeps track of its execution, based log, which captures exactly all the user actions happened
thus tackling the segmentation issue. during many different executions of the routine, thus achieving
the segmentation task.
I. I NTRODUCTION
Robotic Process Automation (RPA) uses software robots II. S EGMENTATION U SING T RACE A LIGNMENT
(or simply SW robots) to mimic and replicate the execution In this section, after providing the relevant background on
of highly routine tasks (in the following, called routines) trace alignment (see Section II-A), we present a first approach
performed by humans in their application’s User Interface to tackle the segmentation issue (see Section II-B).
(UI). SW robots encode, by means of executable scripts,
sequences of fine-grained interactions with a computer system. A. Alignment between UI Logs and Interaction Models
Commercial RPA tools allow SW robots to automate a wide Trace alignment [4] is a conformance checking technique
range of routines in a record-and-replay fashion. The current within Process Mining that is employed to replay the content
practice for identifying the single steps of a routine is by of any trace of an event log against a process model repre-
means of interviews, walk-throughs, and detailed observation sented as a Petri net, one event at a time. For each trace in
of workers conducting their daily work. A recent approach the log, the technique identifies the closest corresponding trace
proposed by Bosco et al. [1] makes this identification less that can be parsed by the model, i.e., an alignment, together
time-consuming and error-prone, as it enables to automatically with a fitness value, which quantifies how much the trace
extract from a UI log, which records the UI interactions during adheres to the process model. The fitness value can vary from
a routine enactment, those routine steps to be automated with 0 to 1. A fitness value equals to 1 means a perfect matching
a SW robot. While this approach is effective in case of UI between the trace and the model.
logs that keep track of single routine executions, i.e., there In our context, we perform trace alignment by constructing
is an exact 1:1 mapping among a recorded user action and an alignment γ of a UI log U (note that we can consider
the specific routine it belongs to, it becomes inadequate when the entire content of the UI log as a single trace) and an
the UI log records information about several routines whose interaction model w as a Petri net, which allows us to exactly
actions are mixed in some order that reflects the particular pinpoint where deviations occur. To this aim, the events in U
order of their execution by the user. In addition, since the same need to be related to transitions in the model. Building this
user action may belong to different routines, the automated alignment requires to relate “moves” in the log to “moves”
identification of those user actions that belong to a specific in the model. However, it may be that some of the moves in
routine is far from being trivial. The challenge to automatically the log cannot be mimicked by the model and vice versa. A
understand which user actions contribute to which routines move in log for a transition t indicates that t occurred when
inside a UI log is also known as segmentation [2], [3]. not allowed; a move in model for a transition t indicates that
In this research, we investigate a technique for automatically t did not occur, when, conversely, expected. Many alignments
deriving the boundaries of a routine by analyzing the UI are possible for the same UI log and a Petri net. We aim
log that keeps track of its execution, in order to cluster all at finding a complete alignment γ opt of U and w with
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
minimal number of deviations (i.e., of moves in log/model), alignment. At the end of the iteration, the routine-based log
also known in literature as optimal alignments. For the sake of URw is stored into Uset , and the the next interaction model
simplicity, we are assuming here that all the deviations have contained in Wset can be analyzed. In conclusion, a number of
the same severity. However, the severity of a deviation can be routine-based logs equal to the number of interaction models
customized on a ad-hoc basis [5]. under study are computed.
B. A First Approach to Segmenting UI Logs III. D ISCUSSION , F UTURE W ORK AND C ONCLUSION
The proposed approach underlying our segmentation tech- Our first solution to the segmentation issue is a supervised
nique consists of two methodological phases, filtering and technique that leverages trace alignment to identify sequences
trace alignment, to be applied in sequence. The envisioned of user actions in a UI log that belong to specific routine
technique takes in input a UI log U , a set of interaction models executions, clustering them in well bounded routine traces.
Wset and returns a set of routine-based logs Uset . For each Differently from event abstractions techniques [6], which map
interaction model w ∈ Wset (one for each routine of interest) low-level event types to multiple high-level activities (while
represented as Petri nets, the following steps are performed: the event instances, i.e., with a specific timestamp in the log,
Filtering. The filtering phase is used to filter out noisy actions can be coupled with a single high-level activity), segmentation
from the UI log. Specifically, for each interaction model techniques must enable to associate low-level event instances
w ∈ Wset , a local copy of the UI log U w is created. Then, all (corresponding to our UI actions) to multiple routines. The
user actions that appear in U w but that can not be replayed complete knowledge of the interaction models’ structure is, of
by any transition t of w are removed from U w . The output course, the main limitation of the presented technique.
of this step is a model-based filtered UI log Uφw . Working As a future work, we aim at relaxing the supervised
with Uφw rather than with U w will allow us to apply the assumption in different ways: (i) by employing declarative
trace alignment technique neglecting all the potential moves rules rather than Petri nets to represent only a partial view of
in log with user actions that could never be replayed by w. the routines’ structure; (ii) by investigating sequential pattern
As a consequence, this will drastically reduce the number of mining techniques [7] to examine frequent sequences of UI
alignment steps required to find optimal alignments, and at the actions with common data attributes; (iii) by analyzing web log
same time optimize the overall performance. Before moving mining techniques [8], which are focused on an issue similar
to the next step, a new routine-based log URw is initialized. to the one of segmentation, as the input is a set of clickstreams
and the goal is to extract sessions where a user engages with
Trace Alignment. The second step consists of applying the
a web application to fulfill a goal; (iv) by employing machine
trace alignment discussed in Section II-A for any interaction
learning techniques to automatically identify routine traces
model w ∈ Wset and its associated model-based filtered UI
without any previous knowledge of the routines’ structure.
log Uφw . This enables to extract from Uφw all those user actions
Finally, we are going to perform a robust evaluation of
that match a distinguishable pattern with w in the form of an
the proposed technique against synthetic and real-world case
optimal alignment γ opt . Trace alignment allows to pinpoint
studies with heterogeneous UI logs. It is worth to notice that
the synchronous moves between Uφw and w. If they exist, the
for the computation of the trace alignment, we will rely on
user actions involved in synchronous moves are extracted and
opt the highly-scalable and performing planning-based alignment
stored into γsm . Note that focusing just on synchronous moves
techniques implemented in [5], [9], which we can customize
allows us to automatically exclude all redundant user actions
for our purposes. For this reason, our main target will be to
from the analysis. Then:
analyze the reliability and accuracy of our technique.
1) a trace τsm consisting of the user actions associated with
the synchronous moves stored in γsm opt
is created; R EFERENCES
w
2) a UI log Usm containing only τsm , which is required to [1] A. Bosco, A. Augusto, M. Dumas, M. La Rosa, and G. Fortino, “Discov-
properly run (again) trace alignment is created; ering Automatable Routines From User Interaction Logs,” in BPM’19,
w Forum track, 2019.
3) a new alignment between Usm and w with the goal to [2] S. Agostinelli, A. Marrella, and M. Mecella, “Research Challenges for
compute the fitness value is performed. Intelligent Robotic Process Automation,” in BPM’19 Workshops, 2019.
[3] V. Leno, A. Polyvyanyy, M. Dumas, M. La Rosa, and F. M. Maggi,
w
In case the fitness value is equal to 1, this means that Usm “Robotic Process Mining: Vision and Challenges,” BISE, 2020.
(and, consequently, τsm ) can be replayed from the start to the [4] A. Adriansyah, N. Sidorova, and B. F. van Dongen, “Cost-Based Fitness
in Conformance Checking,” in ACSD’11. IEEE, 2011.
final marking of w, making τsm a valid routine trace of w. In [5] M. de Leoni and A. Marrella, “Aligning Real Process Executions and
such a case, τsm is stored into URw and all the events associated Prescriptive Process Models through Automated Planning,” Expert System
to the synchronous moves in τsm are removed by Uφw . On the with Application, vol. 82, pp. 162–183, 2017.
[6] F. Mannhardt, M. de Leoni, H. A. Reijers, W. M. van der Aalst, and P. J.
contrary, a fitness value lower than 1 indicates the presence of Toussaint, “Guided process discovery – a pattern-based approach,” Inf.
at least one move in the model in τsm with respect to w, i.e., Syst., vol. 76, 2018.
τsm can not be completely replayed by w and is not a valid [7] G. Dong, Sequence Data Mining. Springer-Verlag, 2009.
[8] Web Usage Mining. Springer Berlin Heidelberg, 2007, pp. 449–483.
routine trace, meaning that we can discard it. [9] G. De Giacomo, F. M. Maggi, A. Marrella, and F. Patrizi, “On the
opt
The above two steps can be repeated until γsm is not Disruptive Effectiveness of Automated Planning for LTLf -Based Trace
empty, i.e., until there are synchronous moves in the computed Alignment,” in AAAI’17, 2017.