We consider a scenario in which a human operator physically interacts with a lightweight robotic manipulator to accomplish structured co-manipulation tasks. We assume that these tasks are interactively executed by combining the plan guidance and the human physical guidance. In this context, the human guidance is continuously monitored and interpreted by the robotic system to infer whether the human intentions are aligned or not with respect to the planned activities. This way, the robotic system can adapt the execution of the tasks according to the human intention. In this paper we present an overview of the overall framework and discuss some initial results.
Safe and compliant physical human-robot interaction is a crucial issue in service robotics
applications [
Figure 1 illustrates the overall architecture. The High-Level Control System manages
task planning/replanning, monitoring and execution. The Human Operator can guide
the task execution by physically interacting with the manipulator. The robotic system
can estimate the forces Ft provided by the operator on the end-effector, while the
operator perceives the associated force feedback Fext. The operator’s forces are continuously
monitored during physical human-robot interaction to interpret the human guidance in
the context of to the current plan. Specifically, the High-Level Control System interacts
with the Operator Intention Estimation in order to define targets which are coherent
with both the human guidance and the planned activities. The selected target points Xt
and an associated control model b are provided to the Adaptive Shared Controller to
suitably generate the motion trajectory Xc which is compliant with the human
guidance. Finally, the outcome of the Adaptive Shared Controller is exploited by the
Position Controller that can directly generate positions and orientations for the manipulator
end-effector, delegating inner control loops to solve the associated inverse kinematics.
High Level Control System. The High Level Control System integrates plan
generation, plan monitoring, and execution (see Figure 2). The proposed framework is based
on an Executive System capable of continuously monitoring and orchestrating multiple
hierarchical tasks. It can exploit a hierarchical Task Planner for plan generation and
replanning, while a Target Selector is introduced to interpret the human guidance with
respect to the current tasks providing targets and control modes for the Adaptive Shared
Control. The proposed executive framework is inspired by the one proposed by [
LTM method(task1,[task2..]). method(task2,[task3..]). operator(task3,[..]). operator(task4,[..]). operator(task5,[..]). ..
alive WM alive task2 task1 task3 task4 task5
TRUE plan1 white-sugar add-sugar(coffee) sugar.added sugar.added take(coffee) coffee.taken
TRUE add-white-sugar(coffee) sugar.added
TRUE plan2 brown-sugar add-sugar(coffee) sugar.added
TRUE
add-brown-sugar(coffee)
sugar.added
sugar.added
take(coffee)
coffee.taken
that contains a declarative representations of the tasks and the actions the robotic system
is able to execute. Each task can be allocated, hierarchically decomposed, and
instantiated into the WM, which represents the executive state of the system: the WM collects
the set of activities currently under execution, including both complex and primitive
tasks. Additional details can be found in [
Plan Execution. Each task can be associated with set of alternatives P1; : : : ; Pk each representing a possible executable plan generated by a HTN planner. The alternative plans are associated with suitable nodes allocated in the WM and connected with the associated hierarchal task structure. When multiple conflicting behaviors are enabled, the human operator guidance can be exploited to implicitly overcome the produced impasse pointing the system towards the desired target. During the interaction, the WM maintains the hierarchical structure of the allocated plans.
The human interactive physical interventions are continuously interpreted in order to estimate the associated intention and to accordingly adjust the robot collaborative behavior at different levels of abstraction: trajectories, targets, and tasks. The interpretation of the human intention is obtained by the interaction of the High-level Control System and the Operator Intention Estimation modules (see Figure 1). The first one proposes possible targets for robotic manipulator which are consisted with the activities in the WM; each possible target is evaluated by the Operator Intention Estimation conisdering the human physical guidance. The interpreted targets are then provided to the Target Selector, whose outcome is sent to the Adaptive Shared Controller that suitably adapts the robot behavior: when the human guidance is coherent with respect to the tasks in WM and a shared target clearly emerges, the Adaptive Shared Controller provides a compliant robotic behavior. Otherwise, if the assessed intention is misaligned or the current target is ambiguous, the Adaptive Shared Controller switches in a passive mode to enable a comfortable human guidance.
Intention Classification. Given a target (and the trajectory to reach it) the human
interventions are classified into four possible (low-level) intentions. In a first case, We
have a Coinciding user guidance when it is coherent with both the target and trajectory.
The intervention is assessed as Deviating when the human aims at adjusting the robot
motion (e.g. in order to avoid an obstacle) without changing the final target. if the
human intention is to contrast the robot motion (e.g. to stop or suspend the execution) we
have an Opposite intention, while when the opposition is aimed at switching towards a
different task/target we assess an Opposite Deviating intention. The intention
classification mechanism is based on a three layered feed-forward Neural Network that classifies
the aim of the human physical interventions from three input data: the magnitude of the
contact forces provided by the operator; the distance between the current position of
the end effector and the closest point to the planned trajectory; the deviation between
the planned and human motions (i.e. the angle between the two movement vectors). The
network povides the outcome on four nodes, each associated with the classes introduced
above: Coinciding, Deviating, Opposite, Opposite Deviating. Additional details about
this network can be found in [
Target Selection. As already explained above, the multiple plans allocated in the WM are hierarchically decomposed till the primitive operators. Each allocated primitive operator, when enabled (i.e. all the associated preconditions satisfied), is associated with a possible target point and trajectory which is assessed by theOperator Intention Estimation and then sent to the Target Selector. More specifically, at each time stamp, all the enabled primitive operators produce a target point and the associated intention estimation, these are then exploited by the Target Selector to define both the current target position along with the interaction mode for a compliant interaction. The target selection process works as follows. Whenever there is only one target with the associated intention assessed as Coinciding or Deviating, that target is selected; otherwise, no target is selected. The Target Selector couples each target with an operation mode that coincides with the estimated intention in the case of Coinciding or Deviating; when no target can be selected, the operation mode is set as P assive leaving the lead to the human operator until a clear target is again selected in the next time stamps. Adaptive Shared Controller. The Adaptive Shared Controller receives target positions Xt and the operation mode (Coinciding, Deviating, P assive) from the High-Level Control System to generate the motion data Xd needed to reach the target. During the execution, the human exerts a force Ft on the end effector. Since the manipulator should be adaptive with respect to the operator physical guidance, we exploit an admittance controller, which is described by the second-order equation: Xci+1 =
M Xdi + D(X_ di
X_ ci ) + K(Xdi
M
Xci ) + Ft
:
(1)
with M , D and K representing, respectively, the desired virtual inertia, the virtual
damping and the virtual stiffness. The output of this module is the instant compliant
position Xc, representing the control command for the Position-Controlled System.
Depending on the estimated target and the human intention, the robotic manipulator may
set a passive or an active mode. In the first case, the manipulator is fully compliant to
the operator interaction without providing any contribution to the task execution. In the
second case, the robot can assist the operator during the execution of the cooperative
task. The switch from a passive to an active mode is obtained by removing the virtual
stiffness from Eq. 1 and by setting to zero the desired acceleration and velocity.
Instead, when the target is associated with a Coinciding or Deviating mode, the virtual
stiffness is set to a value higher than zero. In particular, when the operator intention is
interpreted as Coinciding the planned target point and the motion trajectories are
maintained, along with the admittance parameters for cooperative manipulation. Instead,
when the operation mode is Deviating, a more docile behavior for the robot is needed.
In order to achieve this effect, while the operation mode is Deviating, the Adaptive
Shared Controller not only sets specific admittance parameters, but also generates
intermediate target points between the final target position Xt and the closest point to the
planned path Cp. This intermediate target is updated until the operative mode changes
in order to smoothly guide the user towards the planned trajectory. When the
manipulator is guided back to the planned trajectory a Coinciding mode is activated again.
Similarly to [
The system has been assessed considering a simple assembly task that involves the human and the robot in the building of a small pyramid of objects as illustrated in in Figure 3 (left). As for the robotic platform, we exploited a KUKA LWR IV+, equipped with a WSG50 2-fingers gripper in a table-top workspace of 50 70 cm. In this experiment,
Alive
TRUE assemble(b1,base) base(b1)
TRUE assemble(b2,base) base(b2)
TURE plan1 base(b1) assemble(b2,base) base(b2) TURE plan1 base(b2) assemble(b1,base) base(b1) base(b1) ∧ base(b2) assemble(b3,base)
base(b3)
base(b1) ∧ base(b2)
assemble(b3,base)
base(b3)
two alternative plans can be executed to accomplish the task (Figure 3) and the actual
plan/action selection process depends on the users physical guidance during the
interaction. Our aim was to test whether the proposed plan guidance supports cooperative task
execution and enhances the accuracy of intention estimation. We involved three users
in the tests; each tester executed four times the task with or without the plan guidance.
Despite the simplicity of the task, we observed a clear impact of the plan guidance on
the intention estimation accuracy (0:962% with plan guidance vs 0:575% without) and
an associated speed-up in task execution (up to 59:6% of speed-up). Additional details
about the experimental results can be found in [
We presented a framework that integrates interactive plan execution and physical humanrobot interaction in order to enable the execution of complex co-manipulation tasks. We assume that system is endowed with hierarchically represented tasks that can be executed exploiting the human physical guidance. In contrast with alternative approaches to physical human-robot interaction, in the proposed framework the operator physical guidance is interpreted in the context of a structured collaborative task. In this setting, during the interactive manipulation, the user interventions are continuously assessed with respect to the possible alternative tasks/activities proposed by the plan in order to infer trajectory deviations and task switches. The robotic compliance is then suitably regulated. The proposed framework has been demonstrated in a real world testing scenario in which a user interacts with a lightweight manipulator in order to accomplish a simple assembly tasks. The collected results suggest that the system is more effective when the plan guidance is active, with a positive impact on both the time to executed the task and the classification performance.
Acknowledgement. The research leading to these results has been supported by the ERC AdG-320992 RoDyMan, H2020-ICT-731590 REFILLs, MISE ROMOLO, and DIETI MARsHaL.