A common problem in domains, such as, Robotic Process Automation or Task Mining, is to obtain event logs that include screenshots, i.e., a User Interface (UI) log. This paper presents a data package consisting of synthetic UI logs and corresponding screenshots to facilitate the testing and validation of process analysis proposals. The UI logs are generated based on an easily adaptable configuration. They closely resemble real-world use cases in the administrative domain, including three distinct processes with diferent activities, variants, and decision points. Multiple scenarios with slight variations in the Graphical User Interface (GUI) components are generated for each log. This data package provides a valuable resource for evaluating and refining approaches in screen-based contexts.
challenges, we propose developing a collection of synthetic UI logs that can be easily customized by modifying the generation configuration. This synthetic log dataset will facilitate the testing and validating various proposals in the field, thereby addressing the aforementioned issues.
A data package has been published1 consisting of synthetic UI logs along with corresponding
screenshots. These UI logs closely resemble real-world use cases within the administrative
domain. They exhibit varying levels of complexity, measured by the number of activities, process
variants, and visual features that influence the outcome of decision points. For its generation, the
BPM Log Generator tool2 [
The data package comprises three distinct processes, P1, P2, and P3, for which their initial configuration is provided, i.e., a tuple of <SeedLog, Case-level variability configuration, Scenariolevel variability configuration> . They are characterized by the following: • P1. Client Creation: 5 activities and 2 variants. The decision point in this process revolves around the presence of an attachment in the reception of an email. • P2. Client Deletion. User’s presence in the system: 7 activities and 2 variants.
The decision in this process is based on the result of the user’s search in the Customer Management System (CRM), represented by a checkbox. • P3. Client Deletion. Validation of payments: 7 activities and 4 variants. The decision involves two conditions: (1) the presence of an attachment justifying the payment of the invoices in the email and (2) the existence of pending invoices in the user CRM profile. These problems depict processes with a single decision point, without cycles, and executed sequentially to ensure a non-interleaved execution pattern. Particularly, P3 is more complex since two visual characteristics determine its decision point.
For each problem, case-level variations have been applied to generate logs with diferent sizes, in the range of {10, 25, 50, 100} events. In cases where the log exceeds the desired size, the last instance is removed to maintain completeness. In addition, each log size has its associated balanced and imbalanced log, where balanced logs have an approximately equal distribution of instances across variants. In contrast, imbalanced logs have a frequency diference of more than 20% between the most frequent and least frequent variants. To ensure the results’ reliability, 30 scenarios are generated for each tuple <Problem, LogSize, Balanced?>. These scenarios exhibit
2Available at: https://canela.lsi.us.es/bpmloggenerator/public slight variations at the scenario-level, particularly in the look and feel and user interface of the applications depicted in the screenshots. Each scenario consists of UI logs corresponding to specific problems categorized by log size (10, 25, 50, 100) and balanced? (Balanced, Imbalanced). Here is an example from the file structure of P1. UI logs and their corresponding screenshots are organized in folders named as follows: sc{scenarioId}_size_{LogSize}_{Balanced?}.
P1/ sc0_size_10_Balanced/ screenshots/ 1_img.png 2_img.png ...
log.csv sc0_size_10_Imbalanced/ ...
sc30_size_100_Balanced/
... initial_generation_configuration/ seed.csv seed_screenshots.zip case-level_variabilty_conf.json scenario-level_variabilty_conf.json decision.json
Furthermore, each problem includes two additional artifacts. Firstly, the
initial_generation_configuration folder contains the data necessary for generating problem data
using the BPM Log Generator tool [
"compo2": [374, 230, 482, 250] }, "decision": { "V1": "exists(B_compo1) or exists(B_compo2)", // condition to be variant 1 "V2": "not(exists(B_compo1) or exists(B_compo2))" // condition to be variant 2 }
This testing oracle serves as a label for validating mined data. The file is divided in two main sections: "UICompos" and "decision". The "UICompos" section includes a key for each activity related to the decision, storing key-value pairs that represent the UI components involved, along with their bounding box coordinates. The "decision" section defines the condition for a case to match a specific variant based on the mentioned UI components.
This section is intended to provide an example of the use of this data package. Specifically, this example consists of performing a decision model discovery study using the provided data package. The analysis utilizes a decision mining tool3, which focuses on extracting information from events preceding the decision point (DP).
Table 1 presents the relevant properties of the UI logs associated with each problem, specifically concerning events before DP. These properties include the number of variants (#variants),
the average number of activities (avg #act), the number of activities occurring before the decision point (#act before DP), the average number of UI components per screenshot, i.e., UI compo. density prior to the decision point (#UICompo density before DP), and the average number of UI components influencing the decision point condition (avg #UICompo in DP).
Considering that the first two activities (A: view mailing list, and B: view mail) are common
to all three problems. It can be observed, as expected, that the density varies depending on the
nature of the activity. Where A and B are denser than the rest, since P1, which only considers
these before the DP, has the maximum UI components density. The preliminary analysis, in
this case, focuses on the "best" configuration where all UI components involved in the decision
point are discovered, as specified in the "decision.json" file. This configuration corresponds to
a balanced log of size 100. The results of applying the decision model discovery approach
proposed by [
Although the diferences between these problems influence their behavior under specific
conifgurations, this analysis emphasizes the optimal case. For a more comprehensive understanding
and in-depth analysis, please refer to a recent publication [
This paper introduces a synthetic UI log data package with screenshots, which can be easily
modified using BPM Log Generator tool [
This publication is part of the projects PID2019-105455GB-C31 and PID2022-137646OB-C31, funded by MCIN/ AEI/10.13039/501100011033/ and by the “European Union”; the FPU scholarship program, granted by the Spanish Ministry of Education and Vocational Training (FPU20/05984) and its mobility grants (EST23/00732).