Business process automation (BPA) has recently emerged as a multi-billion dollar industry, revolutionizing how businesses manage their processes 1. It develops solutions for automated decision making, data management, and workflow optimization. Many of these solutions leverage digital assistants that can speak the language of business users to make these automations more accessible to non-tech-savvy end-users [1, 2, 3]. In enterprise settings, conversational assistants may be used by teams of people to automate role-based tasks (e.g., in HR, sales, or accounting) and they may want to refine them over time to meet changing business needs. However, existing commercial oferings of digital assistants for process automation from companies like IBM, Microsoft, and UIPath provide a fixed set of automation capabilities. Expanding their functionality is often restricted to domain experts with programming skills. Hence, most end-users are unable to customize or create new process automation capabilities to suit their process management needs. This, in-turn, can result in user frustration and decreased adoption of process automation tools. One possible approach to increase adoption is to empower end-users to intuitively and naturally develop customized and improved automation capabilities from existing automations within these digital assistants by teaching them. Unlike the personal chatbots in [4], digital assistants in business automation settings may be more role-driven. Hence, they would be used by multiple end-users who need to customize them based on company processes, roles and team preferences. In this paper, we posit that the next generation of digital assistants within enterprise settings will be iteratively taught in-situ through intuitive multi-modal interactions dependent on the type of task and skill of the teacher(s). These assistants should be characterized by their ability to continually learn, customizability by end-users with minimal coding expertise, evolution with changing business processes, and generalizability to new process automation tasks by understanding their context. We also discuss some of the challenges of realizing these assistants, focusing on challenges in machine learning, process management, and interactive task learning.
Traditional offline refinement (separate Use and Build phases)
The traditional development cycle, in Figure 1 (left), splits the use and build phases of digital assistants for process automation. Run-time failures encountered by users are logged, and used by expert developers to eventually build and deploy new versions of the assistants with improved automation skills. This is a relatively longer process, which due to lack of developer resources, can result in developers prioritizing the creation of some skills and resolution of certain issues over others, leaving behind frustrated and disconnected users.
We instead envision an iterative in-situ refinement of digital assistants, as in Figure 1 (right), where users are empowered to address automation shortcomings by teaching the assistant new capabilities. There are three aspects to this approach that difer from the traditional way that assistants are built. First, failures while using a digital assistant interrupt the user’s task and the system enters a teaching mode. These failures may occur due to developer or business-defined guardrails that limit the execution bounds for the assistant, or because of the user explicitly indicating they were dissatisfied with the assistant’s actions. For example, an assistant used to book a trip may only know what to do for domestic trips, and a guardrail can be triggered when a user tries to book an international trip. Second, the user is repairing a specific failure in the context of the task they were in. Since teaching happens in the context of the failures, it enables the user to quickly address the issue. We believe this is an easier way for non-developers to teach and refine the assistant. Third, the same user interacting the assistant is responsible for teaching the assistant how to handle the failure. This empowers users to evolve and customize the assistant according to their needs.
There can be two variants of the iterative in-situ refinement approach: Mode 1 where the user resumes their task after repairing the failure; and Mode 2 where the user restarts their task with the new version of the automation. The latter may be appropriate if there have been no side efects so far, or if the automation can roll back or compensate the steps taken so far.
This in-situ teaching mode needs to be intuitive to non-tech-savvy end-users by leveraging
various input modalities tailored to automation tasks and users’ teaching abilities. For some
tasks or users, it may be easier to “show” the assistant how to perform the task and thus requires
screen recording capabilities. Other tasks may be more easily taught through natural language
instructions. While [
Realizing this next generation of digital assistants with multi-modal, iterative, in-situ teaching requires addressing various research challenges and integrating multiple technological advancements before successful deployment is possible.
Metacognition: An important pre-requisite for learning in digital assistants is their ability to
recognize and understand what they do, and do not know [
Natural language understanding: Understanding the underlying intentions of user inputs is
critical for assistants to execute the right skills or determine when to get more information and
learn. Analyzing inputs in isolation can result in misinterpretation since information may be
provided through diferent modalities, combined across multiple inputs, or be missing altogether.
Prior work [
Interactive and informative experience: The teaching process can result in assistants behaving diferently from user expectations (e.g., learning incorrect skills or misinterpreting key information), which can prove costly. Hence, it is important to provide informative responses reflecting the assistant’s understanding at every stage of the teaching process. This includes “failing gracefully” by defining mechanisms to provide interpretable error messages and seek appropriate user feedback, when users’ expectations are violated. The vast space of multi-modal responses makes generating the appropriate response a dificult problem.
Learning new skills: Automation skills vary in complexity, scope, and usage, challenging
assistants to learn new skills. Some may require incrementally updating to existing skills, or
learning a model through user-provided input/output examples, or even synthesising new code
from specifications, making it important to identify what and how to learn. Large teams of
users may teach overlapping skills, requiring assistants to disambiguate. Also, new skills must
be immediately available for use. Analyzing interaction logs to provide capabilities such as
auto-complete, and continually updating skills with changing business and user needs are
important capabilities that can make the assistant’s teaching model more intuitive.
Customization and generalization: Existing assistants often require users to provide
individual inputs to execute each skill and its parameters, incurring significant overhead for repeated
executions of large workflows. The ability to teach customizations such as macros to execute
groups of skills with a single input, pre-filling process parameters, and modifying process
execution should be enabled in assistants using eficient representations (e.g., user profiles).
However, since skills will be used by multiple users with diferent business needs, assistants
should be able to generalize the taught instructions and extrapolate to similar tasks. This will
enable them to perform an ever-increasing number of tasks, require fewer instructions, and
learn continually from subsequent interactions. Furthermore, new skills may be taught to the
assistant by multiple users (with diferent “teaching” abilities) which requires identifying similar
instructions and generalizing from them. While previous work focused on generalization in
specific domains [
Catastrophic forgetting: Teaching digital assistants new tasks runs the risk of conflicts
between new capabilities and old ones or even “forgetting” how to perform previously known
tasks. Existing literature [