Human-AI teams have the potential to produce improved outcomes in various tasks as opposed to each team member working alone. However, there are various factors that influence human-AI team performance which potentially difer from classical Human-Computer Interaction settings. Specifically, there is existing work indicating that it is beneficial for AI systems to automatically adapt their autonomy within the team and task in order to work towards achieving a shared goal more efectively. Thus, in this paper, we describe a concept of situational adaptive autonomy for human-AI cooperation in a shared workspace setting. We discuss that task complexity and models for the AI system's understanding of their human teammate, i.e. Theory of Mind models (ToMMs) might be helpful to implement situational adaptation of AI autonomy such that interaction and team performance can be improved. We present an experimental setup for a cooperative real-world robotic task and a corresponding approximation in a grid-world in which we plan to investigate situational adaptive autonomy within a shared workspace in an interactive human-AI team.
Recent artificial intelligence (AI) systems and robots port humans at work [1, 2, 3] or in everyday life tasks [4, 2, 5, 6, 7, 8]. However, improving task outcomes through human-AI collaboration is not trivial, often casespecific, and depends on the abilities and characteristics of each party [9]. While general design principles for classical human-computer interaction (HCI) have already been explored well, there is a need to update them for designing interactions with AI systems [10, 11, 12] such that human-AI teams can solve problems synergistically and improve their performance together. Hereby, it is important to consider factors where current and future AI systems might difer from classical HCI systems. omy that AI systems can possibly achieve during cooperative tasks [13]. When humans interact with technical systems as tools, they commonly automate a very specific sub-task to achieve their overall goals more eficiently by using the system [14, 15, 16]. In such cases, systems have a specific and limited purpose but in general no autonomy within the task and the team. On the other end of the autonomy spectrum lies a fully autonomous collaboration partner with whom the human outputs a This work was funded by German Federal Ministry of Education and Research (project IKIDA, 01IS20045). nEvelop-O collective result for a shared goal, i.e. human-human interaction, e.g. when colleagues collaborate with each other. Such collaboration depends on communication, interaction with AI agents falls somewhere in between the ends of the spectrum shown in Fig. 1.
Diferent concepts for autonomy levels have also been proposed in previous work [19, 13, 20, 21]. However, high(er) autonomy in systems does not necessarily increase team performance or is preferred by their human counterparts in every situation [22]. Previous work also indicated that the ability to slide along the autonomy scale and dynamically adapt autonomy levels is beneifcial [ 23, 21, 13, 24]. In more recent work, autonomy is often specified as a set of autonomy levels which an operator can switch (manually) [19, 20, 21]. Automatresults in specific use cases, e.g. multi-agent systems without human interaction [23], Unmanned Aerial Vehicle path-planning [21], settings where an operator remotely controls a robot in hazardous environments [13] or simulation-based evaluations for a cleaning and an inventory scenario with a mobile manipulator [24]. However, we see a lack of experimental real-world evaluations of concepts for situational adaptation of AI autonomy in cooperative shared workspace settings.
One distinguishing factor is the higher degree of auton- ically adjusting autonomy already showed promising © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License rate measures for task complexity as well as models for the AI system’s understanding of their human teammate, i.e. Theory of Mind models [25, 26, 27], to automatically where a human operator remotely controls one or multidecide when the AI system should switch its autonomy ple robots in hazardous or space environments [34, 34] level in collaborative interaction with their human part- and for path planning of Unmanned Aerial Vehicles [21]. ner. This should ensure that interaction is initiated and In particular, [21] shows that human-AI cooperation with expressed by the AI system only when it is appropriate autonomy adaptation can lead to better performance as and helpful for the human and the overall task goal. The opposed to either human or system completing the task proposed concept for situational adaptive autonomy re- alone. sults in an interactive AI system which can slide on the Generally, there is some evidence that humans profit autonomy spectrum shown in Fig. 1 in blue. from [35] and are in favor of systems having a high(er)
We advocate for investigating interaction paradigms level of autonomy [13] when it helps their goals. Humans arising when interactive AI systems slide along this au- also share their task load more when they perceive a systonomy scale. To this end, we propose an experimen- tem’s behavior as human-like [36]. Nevertheless, there tal task setup within a grid-world environment and a is also literature about how humans sometimes prefer real-world robotic task to gain deeper insights into the when they have control over systems [37, 10] or reduce interaction with a coordinating human-AI team. the systems’ autonomy [35]. Also, the interplay of a system’s autonomy level and the situational awareness of its user has been investigated [38]. Further work also ac2. Background knowledges that the design and evaluation of human-AI interaction (partly) depends on the autonomy spectrum There is a large body of previous work defining au- on which systems can lay from just being tools to being tonomy and its possible levels for AI or robotic agents counterparts or teammates [39]. [28, 22, 29, 30, 31, 32, 33]. For a more detailed overview Some research also demonstrates the potential of adapon the definition of autonomy that aligns well with how tive autonomy in shared workspace settings [40, 24]. In it is framed in our work we refer the interested reader to [40], the authors present a framework that incorporates [31, 32]. In this section, we focus on the discussion of re- situation assessment and planning together with human lated work that aims to enable AI agents to automatically intention prediction and reactive action execution. Their adjust their level of autonomy (Section 2.1) and subse- approach enables a robot to adapt to user preferences quently provide a short background on task complexity allowing the human partner to be more passive or acand Theory of Mind models, which we propose as two tive in giving commands. A Theory of Mind model for important factors to implement situational adjustment predicting temporary absence or inattention of the huof autonomy in shared workspace settings (Section 2.2). man is proposed in [24] to automatically adapt robot communication patterns during the execution of a coop2.1. Adaptive Autonomy erative table cleaning task. However, both approaches are only evaluated in virtual environments with simulated humans. We found a lack of evaluations for adjustable autonomy in shared workspaces with real human users and in real-world robotic scenarios.
There are various existing definitions for task complexity [41, 42, 43, 44]. The definitions often aim to describe how For our approach and all planned experiments, there is much cognitive processing capabilities, skills, informa- no debating or adjusting of the goal itself, i.e. the human tion, and knowledge availability are necessary to perform sets the overall task goal, which is to be achieved with a task [45, 41]. Furthermore, objective task complexity the help of the AI teammate [19] and the goal is known can comprise of number of task components, goals, or to both. In order to investigate efects on the interacpossible solution paths [46, 47]. Overall, [48] consolidate tion paradigms for situational adaptive autonomy in a existing definitions into one model for task complexity. shared workspace, we first of all require a concrete impleIt comprises of ten dimensions, e.g. the number of task mentation of autonomy levels for the AI agent. Existing components, their interdependency and time-related con- concepts for autonomy levels [22, 38, 59, 60, 32, 28, 29] straints. Furthermore, there are frameworks for task com- have often been proposed in a more theoretical framplexity for human-system integration [49] and for human ing. All of these concepts include at least 10 levels of teams [47]. Specifically, [ 47] distinguishes between com- autonomy for an AI agent. ponent complexity which considers task aspects for the For our concrete implementation we decided to follow individuals within the team and coordinative complexity the concept from [29, 28] but summarize their 10 levels regarding factors of interaction and teamwork such as into four. This is due to two reasons: First, while [29, 28] interdependencies and solution diversity. distinguish some levels by how and if the robot informs
Task complexity influences individual and group per- the human about its choice, we implement the AI sysformance in diferent tasks [ 48, 50]. While teams perform tem’s decision on how and when to notify or question worse compared to individuals when working on lower the human based on a ToMM. This model considers the complexity tasks, higher task complexity leads to interact- situation and current state of its human partner and thus ing groups outperforming individual performance [50]. only informs or asks them about anything when it asThis means, humans can profit from support when work- sumes that the human is able to comprehend and reply. ing on tasks with higher task complexity. This is also Thus, we do not separate such communication with the in line with [51] who found that humans rely more on human partner in distinct autonomy levels. Second, we support of systems when there is higher task complex- hypothesize that it is easier for the human teammate ity. While there are also other aspects which influence to understand in which autonomy level the AI partner the performance of human-AI teams [52], they might currently is when there are less levels overall. be most successful when humans and AI agents com- We implement the expression of the AI system’s autonplement each other [53, 54]. Complementarity requires omy level by how much initiative it takes to deviate from among other things that teammates have awareness of its currently assigned sub-task if it assumes a benefit for the situation [55, 56] and about what their teammate the team performance. We specify the following four knows and plans, which is known as Theory of Mind, diferent types of initiative. i.e. the modeling of mental states of others [25]. They No Initiative: the AI just continues with its current are also used computationally in various human-AI in- sub-task; makes no suggestions if it notices sub-tasks teraction settings [26, 27, 24, 57, 58] such that they allow with higher priority; goes into idle mode if it encounters for anticipation for human actions and planning with an a problem during its sub task execution. appropriate model of the human in the interaction. This Low Initiative: If during a sub-task execution the AI can ensure that systems can adjust for specific users and encounters a problem or notices a sub-task with a higher plan better (together) with them [24]. priority, it presents a list of possible alternative sub-tasks to the human and waits to see if they choose one. 3. Situational Adaptive Autonomy Moderate Initiative: If during a sub-task execution the AI encounters a problem or notices a sub-task with a for Cooperative Tasks in Shared higher priority it presents the option it assumes as the Workspaces best possibility and waits for confirmation (or rejection).
Full Initiative: If during a sub-task execution the AI Situational adaptive autonomy may influence the interac- encounters a problem or notices a sub-task with a higher tion and performance of human-AI teams in cooperative priority, the AI executes an alternative it considers the tasks. In this section, we discuss how we plan to realize best possibility for improvement; if the ToMM indicates a concept of situational adaptive autonomy in a shared human availability it informs the human and ask them if workspace setting (Section 3.1) and describe the task in the decision was okay. which we plan to evaluate our approach (Section 3.2). Overall, assigning sub-tasks can be based on access, competence and permission to execute them as also proposed in [31]. The AI system can lower its autonomy when uncertainties or problems arise or if a miss match between its own competence and (sub-)task requirements occurs. Additionally, humans can intervene in the AI’s actions or potentially delegate new sub-tasks. An increase in autonomy can be beneficial, e.g. when only execution of a sub-task with higher priority can prevent catastrophic failure of the overall task and the ToMM indicates current unavailability or missing situational awareness of the human. We theorize that when task complexity is low(er), autonomy and initiative can be low(er) as well while the team could profit from high(er) autonomy and initiative from the AI when task complexity is high(er), as in [50].
We consider a setting in a shared workspace where the overall task consists of sub-tasks that can be either performed by only the AI agent, only the human or both but in some cases with potentially diferent degree of eficiency. Additionally, we assume that there is always a set of sub-tasks that may result in or prevent failure of the overall task goal. Our setups for the robotic task and the corresponding abstracted grid-world, inspired by [61], are illustrated in Fig. 2. In the proposed task, the human and AI need to organize and process the contents of boxes which get delivered over time. These boxes contain various objects, such as books or documents that require individual handling before they can be stored in their designated spaces. Task performance can be measured e.g. by the number of completely sorted boxes over a predefined time. This task cannot only vary situationally in its complexity but also requires ToMM for successful collaboration.
For the development of interaction paradigms and a more common definition of situational autonomy adaptation in shared workspace settings, we consider it crucial to gain empirical insights from evaluations with real humans in real robotic tasks. To this end, we plan to implement the concept described in this paper and evaluate it with the discussed task setting. We strive to gain a deeper understanding of how factors such as task complexity or ToMM can possibly be used for situational autonomy adaptation in shared workspaces. Particularly, we consider it important to investigate the efects of the resulting situational autonomy adaptation on the performance of and interaction within human-AI teams. We plan to test whether our conceptualized and implemented autonomy levels and corresponding initiative types positively impact team performance in human-AI interaction for a collaborative task within a shared workspace.
Generally, we advocate that the successful deployment
of situational autonomy adaptation in human-AI
interaction requires more interdisciplinary exchange and
research to better understand the implications for both
humans and AI systems in the future.
ysis of the empirical literature, Human factors 64 rience, and operator trust, Theoretical Issues in
(
W. Taysom, A. Uszok, Dimensions of adjustable au- [45] D. J. Campbell, Task complexity: A review and tonomy and mixed-initiative interaction, in: Agents analysis, Academy of management review 13 (1988) and Computational Autonomy: Potential, Risks, 40–52.
and Solutions 1, Springer, 2004, pp. 17–39. [46] T. M. Brown, C. E. Miller, Communication networks
[32] S. A. Mostafa, M. S. Ahmad, A. Mustapha, Ad- in task-performing groups: Efects of task
complexjustable autonomy: a systematic literature review, ity, time pressure, and interpersonal dominance,
Artificial Intelligence Review 51 (2019) 149–186. Small group research 31 (2000) 131–157.
[33] C. Castelfranchi, Founding agents’” autonomy” on [47] E. H. Lazzara, D. Pavlas, S. Fiore, E. Salas, A
framedependence theory, in: ECAI, volume 1, 2000, pp. work to develop task complexity, in: Proceedings
353–357. of the Human Factors and Ergonomics Society
An[34] G. Dorais, R. P. Bonasso, D. Kortenkamp, B. Pell, nual Meeting, volume 54, SAGE Publications Sage
D. Schreckenghost, Adjustable autonomy for CA: Los Angeles, CA, 2010, pp. 2338–2342.
human-centered autonomous systems on mars, in: [48] P. Liu, Z. Li, Task complexity: A review and
conMars society conference, 1998. ceptualization framework, International Journal of
[35] S. S. Sundar, Rise of machine agency: A framework Industrial Ergonomics 42 (2012) 553–568.
for studying the psychology of human–ai interac- [49] M. Efatmaneshnik, H. A. Handley, Task complexity
tion (haii), Journal of Computer-Mediated Commu- measurement framework for human systems
intenication 25 (
ligibility in context-aware applications, in: Pro- [51] R. Parasuraman, V. Riley, Humans and automation:
ceedings of the 11th international conference on Use, misuse, disuse, abuse, Human factors 39 (1997)
Ubiquitous computing, 2009, pp. 195–204. 230–253.
[38] M. R. Endsley, Automation and situation awareness, [52] O. Vereschak, G. Bailly, B. Caramiaux, How to
in: Automation and human performance: Theory evaluate trust in ai-assisted decision making? a
and applications, CRC Press, 2018, pp. 163–181. survey of empirical methodologies, Proceedings of
[39] C. Wienrich, M. E. Latoschik, extended artificial the ACM on Human-Computer Interaction 5 (2021)
intelligence: New prospects of human-ai interac- 1–39.
tion research, Frontiers in Virtual Reality 2 (2021) [53] M. Steyvers, H. Tejeda, G. Kerrigan, P. Smyth,
686783. Bayesian modeling of human–ai complementarity,
[40] M. Fiore, A. Clodic, R. Alami, On planning and Proceedings of the National Academy of Sciences
task achievement modalities for human-robot col- 119 (
complacency for monitoring highly reliable sys- [56] J. Jiang, A. J. Karran, C. K. Coursaris, P.-M. Léger, tems: the role of task complexity, system expe- J. Beringer, A situation awareness perspective on