Robots acting in real-world and social environments usually face situations requiring tight, continuous interactions with humans. The presence of humans entails several research challenges under the control perspective of a robotic system. A human indeed represents a source of uncertainty a robot controller should deal with in order to synthesize and execute behaviors that are valid, safe, and acceptable.
Humans are usually not controllable and only partially predictable representing a source of uncertainty. With respect to Human-Robot Interaction (HRI), uncertainty about the behavior of a human concerns goals, beliefs, intentions, and expectations. A robot controller should be capable of reasoning about who is the human it interacts with, what are the objectives of the interactions, how to achieve them, and when to execute the needed actions. According to the different contexts and objectives, some assumptions can be made to reduce this uncertainty. In general, it is necessary to find suitable interaction strategies to carry out tasks in a reliable (safe) and effective way.
In addition, there is a social perspective to consider in order to meet the social expectation of a human in a given context and, thus, realize behaviors that are acceptable also under a social perspective. In (social) human-robot interaction scenarios, it is particularly important to reason about how tasks are carried out by a robot in order to comply with so-called social norms and, as a consequence, behave in an acceptable way (for the human user).
The need for implementing different "intelligent behaviors" requires investigating several research directions that lead to the integration of Robotics and Artificial Intelligence (AI) [1,2]. This integration is especially crucial to support personalized and adaptive social and assistive interactions with humans. General interaction capabilities of robotic platforms should be customized according to the specific features of the scenario as well as the preferences and needs of users [3,4,5]. It is fundamental to endow the robot with an "expressive" and wellstructured user model. On the one hand, such a model allows robots to personalize their general interaction/assistive capabilities (i.e., behaviors) to the specific needs of a user. On the other hand, it allows robots to adapt their behavior execution over time according to the changing or evolving states of users (e.g., worsening of impairments, changing health-related needs, or changing interaction preferences).
In this work, we propose a holistic model to be integrated within a task and motion planning approach to enhance the awareness of the social navigation skills of robots. The proposed approach relies on a motion planner, called CoHAN [6,7], which implements human-aware navigation skills and provides a number of parameters to tailor implemented motion behaviors. Within an integrated task and motion planning framework, the idea is to leverage human-related and contextual knowledge available at the task planning level to set COHAN motion parameters and dynamically contextualize navigation behaviors. To this aim, this paper proposes a holistic model to represent domain needs and a mapping from human-aware domain knowledge to the CoHAN's navigation primitives.
Endowing a robot with a well-structured model of humans is crucial to synthesize effective interaction strategies. There are several human and social-related variables that would affect the motions and interaction style of a robot in a given social context [8,9]. Works in social navigation usually focus on the motion task alone, without considering the context in which it is executed. In our opinion, it is important to consider human-related knowledge correlated to the execution of a particular motion [6] as well as contextual knowledge concerning the domain-level tasks being executed [10]. Depending on the specific domain/application needs, tasks requiring (social) navigation skills may entail different priorities, safety requirements, and different performance constraints. All this information impacts the interaction style of a robot and the way motions and navigation behaviors are actually implemented.
We propose a holistic model to characterize social navigation tasks from different synergetic perspectives. We aim at integrating this knowledge into a novel task and motion planning approach to enrich the navigation skills of a robot when performing tasks. Usually, task and motion planners work at two different levels of abstraction. At a higher level, a task planner focuses on the goal-oriented behavior of a robot and plans tasks to achieve high-level goals. At a lower level, a motion planner acts closer to the execution layer by concretely implementing the requested motions. In particular, a motion planner should take into account the perspectives, intentions, and physical motions of involved humans [6].
With respect to the adaptation of the physical motion of a robot, contextual knowledge about the task being executed and the qualities of involved humans could enhance the awareness of the robot. The idea is thus to leverage the contextual and abstract perspective of a task planner to provide a motion planner with contextual knowledge about performed tasks and involved humans in order to enhance the awareness of navigation skills. At the same time, a motion planner exposes a set of interaction parameters that a task planner would use to tailor the physical behavior of the robot to the known context when dispatching navigation actions.
Description of the synergetic perspectives considered to define a holistic model for integrated task and motion planning in human-aware social navigation.
To characterize the features of the tasks a robot should perform in a certain (social) environment to achieve desired goals. Tasks may have different priorities, performance requirements as well as safety constraints. This information would affect the navigation style of a robot and the way it actually moves within the environment and in relationship with humans.
To characterize the features of the humans involved in the execution of tasks and related motions. Humans have different interaction skills, intentions, goals, and preferences that may affect the behavior of a robot at different levels. Furthermore, there can be different expectations with respect to the reliability of their behavior. This information would thus elicit different interaction/navigation styles of a robot with respect to the known features of involved humans.
To characterize the types and qualities of the interaction skills of a robot as well as performance, and execution requirements. This information would in particular define the interaction parameters that would be used to tailor the interaction to the different contexts and expected behaviors of involved humans.
To characterize features of the environment in which a robot should act. This information is suitable to describe objects/obstacles (and their features) that are part of the environment as well as the structure of the environment (e.g., its topology). At this abstraction level, humans can be considered as "dynamic obstacles" of the environment. With respect to the definition of a holistic model, it is necessary to characterize geometric-related information e.g., motion intentions, perspectives, and acceleration that would affect the implemented physical motions of the robot [6].
Table 1 describes four perspectives contributing to the holistic model. For each perspective, the table shows the level of abstraction at which related knowledge affects the behavior of a robot. The next sections discuss in more detail how these perspectives contribute to the contextualization of human-aware navigation behaviors.
Task-level knowledge should characterize the motivations and objectives that lead a robot to act in a (social) environment. As shown in Table 1, the domain and human perspectives contribute to this level of abstraction. These perspectives characterize socially-relevant information about the tasks to be performed (i.e., the domain perspective) and the interacting features of involved humans (i.e., the human perspective).
From the domain perspective, it is useful to define also a number of variables characterizing the tasks a robot should perform with respect to the expected direct/indirect interactions with humans. According to these variables, it would be possible to dynamically "configure" the motion planner and constrain the resulting navigation behavior of the robot. Table 2 describes the variables defined to characterize the tasks requiring a robot to act in a particular social environment. The rationale behind these variables is to estimate to which extent a task is critical with respect to the social dimension of the resulting interaction. Each variable is associated with a score (min 0, max 3) assessing the task from a social perspective. The sum of the scores would estimate the level of the necessary social awareness.
The higher the cumulative value, the lower the need of taking into account human-related constraints when executing a task. For instance, let us consider a task to be executed in a robotic social context, with critical priority, low risk, and strict performance requirements. The motion planner would mainly focus on the technical constraints of the needed motions and execute the task in the most efficient way possible. Let us consider instead a task to be executed in a crowded social context, with low priority, critical risk, and none performance requirements. The motion planner would mainly focus on the social constraints of the needed motions and execute the task in the safest way possible. Considering these examples, we define some thresholds to categorize the social relevance of tasks.
• Technical-critical task have a cumulative score within the interval [9,12] and represent tasks whose execution can focus on the technical constraints mainly. The execution of these tasks would thus have a low impact on humans. The motion planner can thus "relax" underlying social constraints in order to be as much efficient as possible. • Interaction-critical task have a cumulative score within the interval [5,8] and represent tasks whose execution should find a trade-off between technical and social constraints. Namely, the execution of these tasks is expected to affect human behaviors and the motion planner should take into account human behaviors when executing needed motions.
• Social-critical task have a cumulative score within the interval [0, 4] and represent tasks whose execution should mainly focus on the social constraints. The execution of these tasks is expected to strongly affect humans. The motion planner should therefore mainly focus on the social constraints in order to be as much safe and reliable as possible.
Furthermore, it is necessary to estimate the (physical) interaction abilities of the humans that are directly (or indirectly) involved in the execution of a robot task. We rely on the International Classification of Functioning, Disability, and Health (ICF1 ) proposed by the World Health Organization (WHO). The ICF theoretical framework describes the level of functioning of a person from different points of view. Ontological models of the ICF have been proposed and integrated into robot architectures to personalize and adapt the assistance [11,10,12,13]. Moving along a surface on foot, step by step, so that one foot is always on the ground, such as when strolling, sauntering, walking forwards, backward, or sideways.
Table 3 shows the areas and variables modeling the interacting skills and qualities of humans. The rationale behind the considered variables of ICF is to estimate the physical reliability and uncertainty of the physical interactions that may occur between a human and a robot. Depending on the cumulative scores of the variables we define three categories of humans.
• Fragile. Humans falling into this category have a cumulative score within the interval [25, 44]. This category represents humans with limited interaction skills (e.g., low hearing or seeing functioning) and unstable motions (e.g., unstable walking, equilibrium issues, or low attention). This category should in general entail conservative/prudent robot behaviors since no assumptions can be made on the actual physical state/motions of the human (maximum uncertainty). • Average. Humans falling into this category have a cumulative score within the interval [13,25]. This category represents average humans with good interaction skills and sufficiently stable motions. This category allows the robot to make some assumptions about the expected behaviors of the interacting humans and thus perform some level of optimization and planning of motions (average uncertainty). • Reliable. Humans falling into this category have a cumulative score within the interval [0,12]. This category represents "efficient" humans able to reliably interact with robots and perform mutual adaptation to robot motions. In this case, the robot may achieve a higher level of optimization since the behavior of the human is predictable to some extent (minimum uncertainty).
Table 3 and Table 2 generally characterize categories of humans that could be involved in the execution of robotic tasks. To reliably and safely interact with humans it is also necessary to characterize behaviors and interaction qualities of humans from a motion (physical) perspective. Several works in the literature addressed the social navigation problem by taking into account e.g., emotional states and proxemics to adapt motions to humans [14,15,16]. The framework CoHAN generates flexible motion trajectories by taking into account observed intentions and perspectives of humans [6]. The primary objective of CoHAN is to support a higher level of human awareness by observing and evaluating human perspectives. This section describes the sets of motion parameters that could be used to constrain the generation of robot trajectories. These parameters are at the basis of the proposed task and motion planning integrated approach.
Table 4 shows the parameters of the motion planner determining the desired "qualities" of the implemented robot behaviors. These variables set the desired limits of velocity and acceleration of the robot. An interesting parameter is plan (Planning horizon). It determines the look-ahead of planned trajectories and can be set according to the expected uncertainty of involved humans. On the other hand, the robot may perform tasks with high priority (e.g., emergency assistance, and drug delivery) and thus implement trajectories as efficiently as possible finding a suitable trade-off between optimization and safety when encountering humans. In the case of patients, robot tasks would be executed considering Fragile humans but the high priority of the task would require "relaxing" social constraints in the generation of the trajectories. In the case of healthcare professionals, robot tasks would be executed considering Reliable humans and implementing efficient navigation behaviors.
This paper proposes a conceptual integration of task and motion planning aimed at contextualizing the navigation behaviors of robots. We leverage the domain-level knowledge of a task planner to dynamically contextualize navigation skills according to the needs/preferences of humans. This proposal relies on CoHAN, a motion planning framework exposing several motion parameters that can be used by a task planner to constrain trajectory generation. Future work concerns the development of the integrated task and motion planning framework and its evaluation on an assistive-inspired simulated environment. Then, we aim at considering real HRI experiments for evaluating the envisaged capabilities with real human users as well as integrating perception capabilities to dynamically infer human categories from perception.
Author Alessandro Umbrico from ISTC-CNR was supported by the Short-Term Mobility (STM) 2022 Program of the National Research Council of Italy (CNR).
Similarly, the parameter band (Band tightness) can be set according to the expected level of collaboration of humans (e.g., conflict resolution when moving in narrow spaces). Table 5 shows the set of motion parameters modeling the physical motions and states of humans. Parameters about velocity, are useful to infer the motion intentions of a human. The parameter hrad (Radius) specifies proxemics constraints. The parameter hfield (Field of vision) allows the motion planner to know whether the robot is visible to the human or not.
In addition to robot and human parameters, CoHAN supports social variables that can further contextualize robot behavior. The variables of Table 6 like st2c (Time to collision), svis (Visibility), sband (Hidden humans) are especially interesting to realize robot behaviors that are acceptable and close to human expectations.
We now propose patterns mapping the defined categories of humans and social tasks to the motion variables of CoHAN. This mapping allows a task planner to enrich dispatched motion tasks with information about human and task categories.
Table 7 shows how the defined human and task categories could be mapped to the motion variables characterizing the behaviors of the human and the robot. Non-reliable/fragile humans for example entail a motion model of the human characterized by a higher level of uncertainty about intentions and beliefs, limiting the assumptions of the robot when implementing its motions. Similarly, social-critical tasks are mapped to motion variables entailing a more Vice versa, efficient and highly reliable humans entail a motion model of the human characterized by less uncertainty allowing the robot to "optimize" its motions to some extent. Technical-critical tasks would for example push trajectory optimization in order to execute tasks as efficiently as possible. To set social motion variables of CoHAN we consider the synergetic combination of the human and task categories. In this case, indeed it is crucial to jointly reason about the task a robot is supposed to perform and the involved humans in order to find a suitable trade-off between, safety, reliability, and efficiency. Table 8 shows the defined social variable patterns.
We plan to develop an integrated task and motion planning framework and evaluate the desired contextualization capabilities in an assistive-inspired simulated environment. We in particular consider an in-hospital scenario where a socially interacting robot is deployed to support patients and healthcare personnel. In such a scenario a robot would perform different types of tasks (e.g., drug delivery, patient monitoring, technical support to healthcare professionals) each with different priorities, and should interact with different categories of humans (e.g., fragile patients and more reliable healthcare professionals). Figure 1 shows the designed environment. We will consider different scenarios by varying the human/task categories, and the resulting social constraint patterns.
On the one hand, the robot may perform tasks with low priority (e.g., monitoring patients in different rooms and general non-critical assistance) and implement social trajectories when encountering humans. In the case of patients, robot tasks would be executed considering Fragile humans and realize navigation behaviors as cautiously as possible. In the case of healthcare