Towards Experience-based Assistance for Personal Robotic Process Automation by Process-Oriented Case-based Reasoning

Abstract

Robotic process automation (RPA) uses software robots to interact with user interfaces like humans. It thus allows for automating processes across existing application systems without needing internal data interfaces. Personal RPA, also referred to as Robotic Desktop Automation (RDA), is the application of RPA on the desktop of human workers to automate so-called “swivel chair” processes in everyday work. Today personal RPA lacks integration of (procedural) experience and the assistance of human workers with appropriate workflows to be executed in the respective work context. Our work aims to provide experience-based assistance for personal RPA workflows. The support includes workflow execution by pro-active and context-related recommendations of suitable workflows and workflow modeling by fostering reuse. This paper discusses the application of process-oriented case-based reasoning for this purpose. We present the knowledge representation for the assistance system and examine the initial data basis of real RPA workflows as a foundation for developing the methods.

Keywords

• eol>process-oriented case-based reasoning
• robotic process automation
• workflows

1. Motivation

Robotic Process Automation (RPA) [ 1 ] enables the automation of processes across existing application systems without requiring internal data interfaces. RPA employs software robots that mimic human actions in graphical user interfaces (GUIs) to read, enter, and transfer data. It is a lightweight technology and easy to develop since many tools enable the visual programming of RPA workflows [ 2 ]. In general, the most suitable processes to be automated by RPA are repetitive, rule-based, and well-structured, with a high volume of tasks [ 3 ]. Typically, RPA bots run unattended in a server environment while sensors trigger their execution.

Robotic Desktop Automation [ 4, 5, 6 ], which we refer to as personal RPA, is the automation of individual processes of human workers locally on their desktop. A key diference to RPA is that a human worker triggers personal RPA bots. Consequently, the user can control the workflow execution in an agile manner, meaning they can stop the execution at any time and complete the remaining steps manually. Due to the attended execution, personal RPA implies a reduced number of tasks, reduced execution time of the workflows, and the need to give immediate feedback to the user [ 6 ]. Furthermore, personal RPA does not have its own identity as it runs in the desktop environment of the human worker with the same roles and authorizations [ 1 ]. The users themselves model personal RPA workflows and deploy them locally on their desktop. Figure 1 gives an example of a personal RPA workflow that automates the creation of a user account for an incoming request by email. The RPA bot executes all steps as long as the user does not intervene. The example illustrates how tasks can be modeled at diferent levels of abstraction. Here, the first-level tasks are composite tasks representing a partial workflow goal. The nested tasks are the atomic bot actions, e.g., for interacting with the GUI. The first-level tasks can contain nested tasks or execution calls, e.g., of scripts.

Go back to email program Press Win+Tab

Click on Window with Title “Inbox - .*”

Retrieve sender email address from opened email

Open application create new user account

A line of research (cf. [ 7, 8 ]) addresses the transition from rule-based RPA to intelligent RPA. Agostinelli et al. identify four goals for intelligent RPA as open research challenges [ 7 ]: 1) automatic understanding of which human tasks belong to which process and 2) which processes are good candidates for the automation, 3) automated generation of RPA workflows, and 4) the automated orchestration of multiple RPA workflows. Further primary challenges identified by Herm et al. are “the lack of training data, human bias in data, compliance issues with transfer learning, poor explainability of robot decisions, and job-security-induced fear of AI robots” [ 8 ].

For personal RPA, these challenges are also significant barriers. RPA in general and personal RPA, in particular, lacks integration of (procedural) experience, the assistance of human workers with appropriate RPA workflows to be executed in the respective work context, and assistance for modeling workflows. In contrast to RPA, personal RPA does not run unattended but is individually called by the users on their desktops. These environments might change quickly and therefore require frequent adaptation of the RPA workflows.

With the ongoing research work presented in this paper, we are addressing the challenges of rule-based personal RPA with experience-based assistance. We hypothesize that ProcessOriented Case-based Reasoning (POCBR) provides suitable methods for this purpose. To the best of our knowledge, there is no publication addressing the application of CBR or POCBR in the field of RPA.

The remainder of this paper is organized as follows: In the next section, we present the myRPA project on which this research work is based. In section 3, we present our approach to experience-based assistance in the domain of personal RPA with a focus on applying POCBR. We discuss the suitability of our initial data basis for our research in section 4. Section 5 concludes with a summary and outlook.

2. The Project myRPA

The project myRPA1 - Experience-based Robotic Process Automation for Knowledge-based Personal Assistants aims at developing a knowledge-based assistance system to support information and knowledge workers. This comprises process information through personal semantic support and recommendation of personal RPA workflows to automate processes on the desktop according to need and work context to free human workers from tedious routine tasks. Self-learning mechanisms are used to continuously develop and optimize the workflows, which can improve the eficiency and quality of information and knowledge work and reduce the cognitive load of employees regarding “tedious activities”, thus creating more mental freedom for non-repetitive work.

The project focuses on information and knowledge-intensive processes in the ofice environment. We use the corporate memory infrastructure CoMem2 developed by DFKI as a result of many research projects. CoMem allows the integration of distributed and heterogeneous sources, represents them in knowledge graphs, aggregates, enriches, and refines them. For this purpose, CoMem uses a knowledge description layer based on ontologies and knowledge graphs, which allows flexible adaptation for a wide range of domains and enables data use independent of source systems and formats. CoMem is already in productive use in the industry.3 Knowledge services are realized and can be integrated into the workplace through the approach of a semantic desktop [ 9 ]. The semantic desktop, in simplified terms, is a corporate memory on the user’s desktop. Here, a personal information model (PIMO) [ 10 ], which represents the user’s mental model and describes local resources such as files, documents, contacts, and calendars in an integrated and machine-understandable way, is integrated into the knowledge description layer and connected to the corporate knowledge graph. This enables various assistance functionalities for users, like context-sensitive search, context-specific proactive information delivery, and resource access in work contexts that consider a user’s subjective view [ 11 ]. The PIMO also helps to make the assistance explainable and transparent to the user.

The POCBR framework ProCAKE4 [ 12 ], developed at the University of Trier and recently also by the DFKI, is a result of various research projects targeting diferent domains. The domainindependent framework is tailored for implementing structural and particularly process-oriented applications. For instance, it provides diferent representation types for processes and workflows with corresponding similarity measures. We consider our previous work on retrieving and adapting workflows (cf. [ 13, 14, 15 ]) a suitable starting point for further developing the POCBR methods for the domain of personal RPA workflows.

The approach pursued in myRPA is based on the deep integration of RPA and knowledge-based assistance complemented by experience-based methods for process modeling and realization of self-learning behavior. The foundation for this integration is the semantic modeling of RPA workflows in an ontology concerning the workflow as a whole, the single steps (sensors, tasks, and control flow elements), and the application systems for which user interactions are to be 1myRPA is a joined project of DFKI and the industry partners AmdoSoft Systems GmbH, envia Mitteldeutsche Energie AG, and Licht- und Kraftwerke Helmbrechts GmbH. 2https://comem.ai 3https://comem.ai/home/showcase 4https://procake.uni-trier.de automated. This allows for integrating the RPA workflows into the knowledge model of the assistance system and enables both semantic search and modeling support by reusing “best practice” workflows collected in a case base using POCBR. Furthermore, the semantic modeling and the learning methods allow the results to become or remain explainable. The assistance system sees the user in control, who will ultimately decide on the use. Thus, explainability is crucial.

similarity-based recommendation of processes POCBR system

ProCAKE

RPA system AmdoSoft/b4

RPA workflows low level process model with metadata RPA portal high level process model with ontology references context-based recommendation

of processes corporate memory system CoMem legacy systems managdeomcuemntesnytstems net shares

wikis personal information management case base similarity model adaptation knowledge ontologies 3. Experience-based Assistance Approach Various research works have shown the applicability of POCBR for supporting the modeling [ 13 ] and execution [ 17 ] of processes. For the proposed approach, we intend to develop our previous works further [ 13, 14, 15 ] on retrieval and adaptation of workflows in POCBR. The following sections present the assistance’s knowledge representation and core features. Moreover, we discuss the suitability of POCBR methods for personal RPA workflows. 3.1. Representation of Workflow Structure The workflow graph is the central component in a case of the proposed POCBR approach. Today there is no standardized modeling language for RPA [ 18 ]. However, gaining a shared understanding of RPA with all stakeholders is desirable, independent of vendor-specific terminology [ 19 ]. Regarding a tool-independent specification of RPA workflows, there is ongoing research on using use-case descriptions with a controlled vocabulary or pseudo code [ 20 ]. We propose to use semantically enriched BPMN6 format as a vendor and tool-independent representation. In BPMN, a modeler can describe RPA workflows both conceptually (cf. Fig. 1) and technically by adding the specifications required for execution. Domain-specific extensions of the BPMN language can enable this. A high-level process description can also provide an abstract view of the desired steps of an RPA workflow (e.g., using collapsed sub-processes). It is conceivable that mapping high-level process steps to low-level workflow steps can facilitate retrieval for users since they can pose queries with abstract process steps. We follow the BPMN v2.0 standard but restrict the expression elements to a small subset. To obtain more easily readable and maintainable workflow models and to facilitate workflow reuse, we represent the workflows in a block-oriented manner. The block orientation may also mitigate the problem of complex decision models within the workflows that can easily lead to spaghetti-like control flow in standard BPMN [ 18 ]. Particularly for personal RPA workflows, we assume a smaller number of tasks compared to unattended RPA workflows [ 6 ].

process / sub-process task block

sub-process block task

sub-process block sequence block

LOOP block

LOOP block

XOR block

condition A condition B

XOR block

condition A condition B loop condition

AND block

+ loop condition + 3.2. Semantic Description of Workflow Elements We employ an ontology with tool-independent concepts for personal RPA workflow elements to assess the semantic similarity between workflow elements. The ontology is integrated into the knowledge model of the corporate memory system CoMem to allow for linking and reasoning with the personal information model of a knowledge worker. More recently, other work [ 19 ] has been focusing on the conceptualization of RPA. We plan to integrate further ontological knowledge as soon as it becomes available. Figure 4 depicts a small excerpt from the RPA subClassOf

subClassOf Process isPartOf isPartOf Controlflow

Node subClassOf subClassOf Loop Xor

subClassOf …

Thing subClassOf Algorithm subClassOf

Task subClassOf Collection subClassOf …

subClassOf isContainedInGroup Code

Folder

Function invokesFunction subClassOf

subClassOf subClassOf

subClassOf subClassOf subClassOf ServiceTask

KeyboardAction

MouseAction

FunctionAction subClassOf

isSubGroupOf

AlgorithmGroup subClassOf Process Group subClassOf FunctionGroup domain ontology. In addition to the concepts for typical process components including process, control flow node , and task, the ontology contains further domain-specific concepts. For example, specific concepts comprise the various actions of RPA bots. In addition to atomic actio2ns such as mouse and keyboard actions, we represent composite actions, e.g., for the execution of custom script-based functions. We further group processes and functions and organize them in a hierarchy to enhance the similarity assessment. Previous works [ 17 ] in the field of knowledgeintensive CBR have shown the benefits of augmenting cases with ontological knowledge. In our POCBR approach, we follow the local-global principle and apply similarity measures for object-oriented case representations to the entities and relations in the ontology representing the semantics of the RPA workflow elements. The obtained local element similarities are used within a graph mapping procedure (cf. [ 13 ]).

3.3. Workflow Recommendation

A core feature of knowledge-based assistance is recommending suitable RPA workflows to be executed in a given work context. Recently, checklist-based decision support has been proposed to help knowledge workers evaluate the suitability of a given process for RPA automation and develop an RPA bot [ 21 ]. Noppen et al. [ 22 ] propose guidelines for developing RPA in an organization, which may be (partially) transferred to personal RPA. For instance, one guideline suggests “to create an automation library for reusing modules”. One idea is to start with smaller and simpler RPA workflows and reuse parts to build more complex ones. Following the principles of reusability and modularity can help to facilitate the maintenance of the workflows.

For personal RPA, a case-based approach seems to be more suitable than, for instance, a rule-based approach since, as described in section 1, personal RPA workflows are assumed to be highly individual. They may change more frequently over time due to changes in the desktop environment. Moreover, the case-based assistant can enhance its problem-solving competence incrementally along with the knowledge worker using the assistance. Existing retrieval and adaptation methods in POCBR combined with the proposed block-oriented modeling of workflows can support the similarity-based search and, ultimately, the recommendation of workflows in a transparent and explainable manner. Due to the block orientation, composed workflow blocks can be extracted from the workflows in a learning phase. Such blocks can also be considered structural features of a workflow and used to guide the retrieval [ 14 ].

3.4. Workflow Modeling

Modeling RPA workflows is regarded as a comparatively simple task in the scientific literature due to the visual programming supported by many RPA tools [ 2 ]. This holds in particular for personal RPA workflows that comprise a relatively small number of tasks with a simpler control lfow. However, modeling is not considered a one-time task in the RPA domain. There is a need to continuously adapt the workflows to a changing environment (e.g., a changed GUI of an application program) [ 6 ]. Experience-based assistance can further improve the eficiency of the initial modeling phase and support future maintenance.

For instance, POCBR can foster the reuse of existing workflows or building blocks similar to other workflow domains (cf. [ 13 ]). A further advantage can be improved quality and robustness of the workflows. In particular, exception handling is crucial for robust RPA, but modeling these checks by hand can be tedious. Here, an auto-completion feature could support the modelers. Using adaptation methods in POCBR [ 13 ], workflow blocks fulfilling a specific goal, for instance, exception handling, can be learned from the existing workflows. With the help of the RPA modeler, blocks can be mapped to concepts and integrated into the knowledge model. POCBR can also capture manually performed adaptations of an RPA workflow to help the modeler apply it to other afected workflows and future similar situations. 4. Data Basis and Suitability for Reuse We have obtained a repository of RPA workflows from our project partner AmdoSoft. AmdoSoft uses most workflows for automated testing of the company’s RPA software user interfaces. Consequently, the workflows difer from the intended application area for personal RPA and correspond more to unattended RPA workflows. Nevertheless, the repository ofers a rich experimental basis for developing the required methods. Currently, 358 workflows are available, which we converted into block-oriented BPMN following the procedure sketched in the previous section. To get a first impression of the suitability of the workflows for reuse, we extract workflow blocks according to the following procedure and analyze the occurrences.

1. extract each single task or sub-process in a workflow 2. extract each longest continuous sequence of tasks with at least two tasks/sub-processes 3. extract each control-flow block (LOOP, XOR, AND) with all contained elements

We obtain a set of 1439 diferent workflow blocks based on the semantic similarity model. Since workflow blocks can be part of one another, we remove those blocks that are part of a larger block contained in the same set of workflows as the smaller blocks. The reduced set contains 470 blocks. Among these blocks, 120 blocks have an occurrence frequency greater than one, while 106 blocks are contained in more than one workflow. This shows that repetitive blocks occur within the same workflow.

Figure 5 depicts the two workflow blocks with the highest frequency of occurrence (FO) containing more than one task. The workflow blocks are very similar in structure and semantics.

yes Did all tasks finish OK?

no yes Did tasks end with WARNING?no break UP break WARNING break DOWN

yes Did all tasks finish OK?

no yes Did tasks end with WARNING?no

Report – write INFO Report – write WARNING Report – write ERROR (a) Workflow block A (FO = 275) (b) Workflow block B (FO = 159) Both blocks evaluate the execution state of previous tasks and either terminate the workflow execution with a corresponding state (5a) or write the execution state to a report file (5b). Due to the high frequency of occurrence of both blocks, we assume that suficient context dependencies can be learned from the control flow in which they are embedded. If this analysis indicates a high utility of a block, an assistant can recommend it to the user during the modeling phase. If both blocks have a high utility in a specific modeling situation, a combination of both blocks can be added to a workflow under construction supported by (interactive) adaptation. Here, dependencies must be considered as block A terminates the execution. Thus, no other blocks can be appended. number of first-level tasks per WF number of LOOP/XOR/AND per WF number of sub-processes per WF number of blocks per WF number of WFs per block frequency of occurrence (FO) per block number of first-level tasks per block number of LOOP/XOR/AND per block number of sub-processes per block

Table 1 gives an insight into the structure of the 358 workflows and the distribution and structure of the 470 workflow blocks. It indicates the large variety of workflows. On average, one workflow contains about ten blocks (including blocks that are part of a larger one), while one block is contained in six workflows. In the given workflows, the sub-processes always consist of a pure sequence of tasks. We distinguish first-level tasks from tasks within sub-processes. The former tasks execute built-in actions (cf. Fig. 1 and Fig. 5), execute scripts, or invoke other workflows. In contrast, the latter tasks perform atomic actions like controlling mouse and keyboard, arranging windows, processing text, or calculating values.

5. Summary and Outlook

This paper introduces our ongoing research on experience-based assistance for personal robotic process automation by POCBR. We highlighted the characteristics of personal RPA and presented our research project focusing on the role of POCBR. By employing POCBR methods, we intend to foster the reuse of RPA workflows. This includes the recommendation of suitable workflows that can be executed in a given context and the modeling support for adapting workflows or creating new workflows to automate the desired process. As a basis for this, we propose representing procedural knowledge in the form of semantically enriched and block-oriented BPMN. Our analysis of the initial data basis indicates that the workflows are suitable for reuse. In the next step, we will investigate methods for learning constraints, like dependencies between the workflow blocks, to guide the retrieval and adaptation as a basis for workflow recommendation and modeling assistance.

Acknowledgments

We want to thank all project partners for their contributions so far, particularly AmdoSoft Systems GmbH, for providing RPA workflows and the implementation work. The project is funded by the German Federal Ministry of Education and Research (BMBF) under grant number 01IS21030B.

References

[1] J. Wewerka , M. Reichert , Robotic Process Automation - A Systematic Literature Review and Assessment Framework , CoRR abs/ 2012 .11951 ( 2020 ).

[2] M. C. Lacity , L. P. Willcocks , Robotic Process Automation at Telefónica O2, MIS Quarterly Executive 15 ( 2016 ).

[3] R. Syed , S. Suriadi , M. Adams , W. Bandara , S. J. J. Leemans , C. Ouyang , Arthur H. M. ter Hofstede , I. van de Weerd, M. T. Wynn, H. A. Reijers , Robotic Process Automation: Contemporary themes and challenges , Computers in Industry 115 ( 2020 ) 103162 .

[4] S. Seasongood , NOT JUST FOR THE ASSEMBLY LINE: A Case for Robotics in Accounting and Finance , Financial Executive 32 ( 2016 ) 31 - 32 , 35 - 36 , 39 .

[5] G. Baranauskas , Changing Patterns in Process Management and Improvement: Using RPA and RDA in Non-Manufacturing Organizations , European Scientific Journal 14 ( 2018 ) 251 .

[6] R. Cabello , M. Escalona , J. G. Enríquez, Beyond the Hype: RPA Horizon for RobotHuman Interaction , in: Business Process Management: Blockchain and Robotic Process Automation Forum 2020 , Proceedings, volume 393 of LNBIP , Springer, 2020 , pp. 185 - 199 .

[7] S. Agostinelli , A. Marrella , M. Mecella , Towards Intelligent Robotic Process Automation for BPMers , CoRR abs/ 2001 .00804 ( 2020 ).

[8] L.-V. Herm , C. Janiesch , H. A. Reijers , F. Seubert , From Symbolic RPA to Intelligent RPA : Challenges for Developing and Operating Intelligent Software Robots , in: Business Process Management, BPM 2021 , Proc ., volume 12875 of LNCS , Springer, 2021 , pp. 289 - 305 .

[9] L. Sauermann , A. Bernardi , A. Dengel , Overview and Outlook on the Semantic Desktop , in: Proceedings of the ISWC 2005 Workshop on The Semantic Desktop , 2005 , volume 175 of CEUR Workshop Proceedings, CEUR-WS.org , 2005 , pp. 1 - 18 .

[10] L. Sauermann , L. van Elst , A. Dengel , PIMO - A Framework for Representing Personal Information Models, in: I-SEMANTICS , J.UCS , Know-Center , Austria, 2007 , pp. 270 - 277 .

[11] H. Maus , C. Jilek , S. Schwarz , Remembering and Forgetting for Personal Preservation , in: Personal Multimedia Preservation - Remembering or Forgetting Images and Video , Springer, 2018 , pp. 233 - 277 .

[12] R. Bergmann , L. Grumbach , L. Malburg , C. Zeyen, ProCAKE: A Process-Oriented CaseBased Reasoning Framework , in: Workshops Proceedings for the 27th ICCBR 2019 , volume 2567 of CEUR Workshop Proceedings, CEUR-WS.org , 2019 , pp. 156 - 161 .

[13] R. Bergmann , G. Müller, Similarity-Based Retrieval and Automatic Adaptation of Semantic Workflows, in: Synergies Between Knowledge Engineering and Software Engineering , volume 626 of Advances in Intelligent Systems and Computing , Springer, 2018 , pp. 31 - 54 .

[14] C. Zeyen , G. Müller, R. Bergmann , Conversational Process-Oriented Case-Based Reasoning, in: Case-Based Reasoning Research and Development, LNAI , Springer, 2017 , pp. 403 - 419 .

[15] C. Zeyen , L. Malburg , R. Bergmann , Adaptation of Scientific Workflows by Means of Process-Oriented Case-Based Reasoning , in: Case-Based Reasoning Research and Development, LNAI , Springer, 2019 , pp. 388 - 403 .

[16] C. Jilek , M. Schröder , S. Schwarz , H. Maus , A. Dengel , Context spaces as the cornerstone of a near-transparent and self-reorganizing semantic desktop , in: The Semantic Web: ESWC 2018 Satellite Events - ESWC 2018 , volume 11155 of LNCS , Springer, 2018 , pp. 89 - 94 .

[17] A. Martin , K. Hinkelmann , Case-Based Reasoning for Process Experience , in: Business Information Systems and Technology 4.0 , Springer, 2018 , pp. 47 - 63 .

[18] M. Völker , S. Siegert , M. Weske , Adding Decision Management to Robotic Process Automation , in: Business Process Management: Blockchain and Robotic Process Automation Forum , 2021 , Proceedings, volume 428 of LNBIP , Springer, 2021 , pp. 23 - 37 .

[19] M. Völker , M. Weske , Conceptualizing Bots in Robotic Process Automation, in: Conceptual Modeling - 40th International Conference, ER 2021, Virtual Event , 2021 , Proceedings, volume 13011 of LNCS , Springer, 2021 , pp. 3 - 13 .

[20] C. J. M. Correia , A. R. Da Silva , Platform-independent Specifications for Robotic Process Automation Applications , in: Proc. of the 10th Int. Conf. on Model-Driven Engineering and Software Development , MODELSWARD 2022 , SCITEPRESS, 2022 , pp. 379 - 386 .

[21] J. Wewerka , M. Reichert , Checklist-based Support of Knowledge Workers in Robotic Process Automation Projects , in: 23rd IEEE Conference on Business Informatics, CBI 2021 , Bolzano , Volume 1 , IEEE, 2021 , pp. 52 - 61 .

[22] P. Noppen , I. Beerepoot , I. van de Weerd, M. Jonker , H. A. Reijers , How to Keep RPA Maintainable? , in: Business Process Management - 18th International Conference, BPM 2020 , Proceedings , volume 12168 of LNCS , Springer, 2020 , pp. 453 - 470 .