=Paper=
{{Paper
|id=Vol-2054/paper8
|storemode=property
|title=Planning and Execution with Robot Trajectory Generation in Industrial Human-Robot Collaboration
|pdfUrl=https://ceur-ws.org/Vol-2054/paper8.pdf
|volume=Vol-2054
|authors=Amedeo Cesta,Lorenzo Molinari Tosatti,Andrea Orlandini,Nicola Pedrocchi,Stefania Pellegrinelli,Tullio Tolio,Alessandro Umbrico
|dblpUrl=https://dblp.org/rec/conf/aiia/CestaTOPPTU17
}}
==Planning and Execution with Robot Trajectory Generation in Industrial Human-Robot Collaboration==
https://ceur-ws.org/Vol-2054/paper8.pdf
Planning and Execution with Robot Trajectory
Generation in Industrial Human-Robot Collaboration
Amedeo Cesta1 , Lorenzo Molinari Tosatti2 , Andrea Orlandini1 , Nicola Pedrocchi2 ,
Stefania Pellegrinelli2 , Tullio Tolio2 , and Alessandro Umbrico1
1
Institute of Cognitive Science and Technology, ISTC-CNR, Italy
2
Institute of Industrial Technologies and Automation, ITIA-CNR, Milan
Abstract. The co-presence of a robot and a human sharing some activities in
an industrial setting constitutes a challenging scenario for control solutions, re-
quiring highly flexible controllers to preserve productivity and enforce human
safety. Standard methods are not suitable given the lack of methodologies able
to evaluate robot execution time variability, caused by the necessity to continu-
ously modify/adapt robot motions to grant human safety. This paper presents a
novel dynamic planning system for Human-Robot Collaboration (HRC) which
leverages an offline motion planning technique and deploys planning and execu-
tion features dealing with temporal uncertainty and kinematics both at planning
and execution time. The proposed system is deployed in a manufacturing case
study for controlling a working cell in which a robot and a human collaborate to
achieve a shared production goal. The approach has been shown to be feasible
and effective in a real case study.
1 Introduction
During the last decade, industrial robotic systems have entered assembly cells in order
to support human worker in repetitive and physical demanding tasks. However, Human-
Robot Collaboration (HRC) scenarios present several issues that must be addressed
to realize flexible and effective controllers [1]. From a low-level control perspective, a
generic HRC task can be accomplished through many robot trajectories where each tra-
jectory could be executed concurrently to different human tasks and its execution time
depends on the need to modify motion speed in order to grant human safety. Indeed,
the robot speed can be modified based on robot direction of motion, human position,
velocity and direction and motion (i.e.,speed variation monitoring). From a high-level
control perspective, a coordinated task plan should be generated, continuously updated
and concurrently performed by the human and the robot aiming at increasing the ef-
ficiency (i.e., maximize throughput), supporting the human in a timely manner (i.e.,
robot tasks synchronized with human tasks) and, again, always keeping the human safe.
Literature shows how robot motion and task planning are computationally complex,
making difficult their integration in an unified approach, without relying on limiting
hypothesis and applicability contexts [2, 3]. To overcome such issue, some authors
(e.g., [4, 5, 6, 7]) pursued a hierarchical integrated approach that, however, rely on a
clear distinction between task and motion planning features. Indeed, typically the task
plan is constructed at an abstract, high and discrete level and recursively evaluated just
before execution in order to verify the feasibility with respect to spatial/geometric fea-
tures of the domain. Moreover, these works do not consider temporal information and
concurrent execution of human and robot tasks at planning time. Plan-based controllers
such as, e.g., T-R EX [8] or I X T E T- E X E C [9], rely on temporal planning mechanisms
(exploiting respectively EUROPA [10] and I X T E T [11]) capable of dealing with coor-
dinated task actions and temporal flexibility. Unfortunately, these systems do not have
an explicit representation of uncontrollability features in the domain. Thus, in applica-
tion scenarios like HRC, the resulting controllers do not endow the flexibility needed
to cope with uncontrollable time-varying features and robustly execute plans without
strongly relying on replanning mechanisms. Here, we are pursuing an innovative ap-
proach for integrating task and motion planning capable of dealing with both temporal
and spatial constraints and addressing the uncertainty introduced by the human behavior
variability. The approach leverages recent research results, i.e., [12] and [13], to provide
temporal and geometric models of the human and the robot. This paper presents a plan-
ning and execution system fully integrated in such an approach. A key feature of this
work consists in modeling the expected behavior of the human at different levels of
abstractions dealing with temporal uncertainty and kinematics both at planning and ex-
ecution time. A system is deployed to realize a flexible plan-based controller capable
of dynamically adapting the robot behavior to the human tasks as well as guaranteeing
her safety. The system has been applied in a manufacturing case study for controlling a
working cell in which a robot and a human collaborate to achieve a shared production
goal. A set of experiments have demonstrated both the feasibility and effectiveness of
the proposed approach.
2 Human-Aware Control Approach
This paper proposes a human-aware control approach integrating task and motion plan-
ning solutions. Moreover, both the solutions, analyzed singularly, represent an advance-
ment with respect to the state of the art.
Figure 1 shows the main 1. Motion 0. Collaborative tasks definition
2. Domain definition
modules constituting the pro- Temporal trajectories
posed framework and the se- Task Planner
quence of steps implementing 3. Task (re)planning &
the integrated control approach: trajectory selection
Motion Planner
5. Motion
requests
a motion planner, a temporal Plan Executive
task planner and a plan execu-
4. Task plan execution & Monitoring 6. Motion execution & Monitoring
tive. First, the considered indus-
trial process is analyzed to iden- Fig. 1: Task and motion planner integration.
tify the possible human-robot collaboration scenarios (Step 0). Such step identifies the
(collaborative) tasks needed to realize the assembly processes, the resources that can
perform the tasks (human, robot or both), and the operational constraints (e.g., prece-
dence or synchronization constraints). Then, for all the possible collaborative tasks of
the robot i.e., human and robot simultaneous tasks, a set of robot trajectories is com-
puted (Step 1). Specifically, the motion planner is responsible for generating and exe-
cuting robot trajectories and guaranteeing the safety of the human operator adapting the
robot speed. It relies on an offline and statistical analysis able to identify the volume
occupied by the human with a certain probability during the execution of tasks (i.e., the
Human Occupancy Volume - HOV) [12]. Given the HOVs characterized by a different
occupancy probability, the motion planner generates, for each couple of simultaneous
human-robot task, a set of possible trajectories. The different trajectories that the robot
can follow will enter the volume occupied by the human at different levels, thus be-
ing characterized by a different execution time and confidence interval. The identified
robot and human tasks coupled with the related temporal information (time execution
and its variability) are encoded in a temporal planning model (Step 2). This information
allows the task planner to characterize the temporal uncertainty concerning the actual
duration of human and robot tasks. Considering this model, a task planner generates
temporally flexible plans (Step 3) coordinating the operations of the robot and the hu-
man and selecting the most suitable trajectories according to the expected collaborative
context by taking into account operational constraints and safety settings characterizing
the possible collaboration scenarios [14]. The task planner relies on a temporal planning
formalism capable of synthesizing flexible plans by dealing with temporal uncertainty.
Then, the plan executive executes the plan (Step 4) by properly dealing with the tem-
poral variability of the robot and the human. Robust plan execution is achieved through
temporal flexibility and a replanning mechanism that allow the controller to adapt/mod-
ify the plan and robot behavior according to the actual behavior of the human [15]. The
selected robot trajectories (Step 5) are executed by the motion planner which also im-
plements low-level speed separation and variation monitoring to avoid collisions with
the human (Step 6).
3 Dynamic Task Planning
The dynamic task planning system is in charge of (i) deciding the tasks the human and
the robot must perform; (ii) selecting the most suitable trajectory for robot motions
among the set of trajectories generated by the motion planner; (iii) dealing with tem-
poral uncertainty during plan generation and execution; (iv) monitoring the execution
and, in case of need, managing possible failures through replanning. Fig. 2 shows a de-
tailed view of the integrated control architecture. It describes the interactions between
the deliberative and the executive processes and the role of a ROS-based middleware
during the execution of a plan.
The Task Planner and 1. buffered 3b.1. rePlanning
Motion Planner
the Plan Executive in Fig. 2 feedback
Robot Operating System
Robot
have been implemented us- Failure send
ing PLATINUm (PLanning Task Planner Manager command
and Acting with TImeliNes 3b. failure
under Uncertainty) [14], a 2. planned
framework which complies feedback
Plan Exec
with the formalism intro- Monitor
Human
<>
3a. executed
duced in [13] and signifi- send
cantly extends EPSL [16] Dispatcher command
by introducing the capabil-
ity of dealing with temporal Fig. 2: The dynamic task planning control architecture.
uncertainty at both planning
and execution time. Thus, leveraging the timeline-based approach envisaged in [13],
PLATINUm characterizes the planning domain concerning a HRC scenario by consid-
ering the human as a uncontrollable element and the robot as a partially controllable
element. Then, following the structured approach described in [17], the HRC process
can be hierarchically characterized in three abstraction levels. The supervision level
models the overall processes (e.g., a collaborative assembly process) in terms of tasks
needed to realize them. The coordination level models the possible behaviors of the
human and the robot in terms of tasks they can perform. The (temporal) behavior of
the human is modeled as uncontrollable with lower and upper bounds on task durations
according to the information gathered by the offline analysis of the motion planner.
Similarly, the (temporal) behavior of the robot is modeled as partially controllable due
to the co-presence of the human which may affect robot task execution (e.g., robot mo-
tions can be slowed down or suspended according to the position of the human during
execution). Finally, the implementation level models the internal constraints that allow
a robot to actually execute assigned tasks. Again, the temporal characterization of robot
motion tasks leverages information gathered by the motion planner (step 1 in Fig. 1) and
encapsulates information about the available trajectories the execution time variability.
Synchronization rules model possible assignments of tasks to the human and the robot
and the related operational requirements. Such rules specify the possible collaborations
between the human and the robot and the temporal constraints that must be satisfied.
The proposed task and motion planning integration approach allows a task planner
to reason about the particular collaboration scenario and the related interaction modality
by deciding the "most suited" modality of execution of robot tasks in order find a good
tradeoff between the safety of the human and the throughput of the production process.
4 Deployment in a Real Scenario
PLATINU M has been deployed in a manufacturing case study integrating the task plan-
ning technology described above with a motion planning system for industrial robots
[12]. The reference selected application is a human-robot collaborative environment for
the preparation of the load/unload station (LUS) of a flexible manufacturing system
(FMS) (Fig.3). At the LUS, machined parts and raw parts have to be unmounted and
mounted on ad-hoc fixturing systems, called pallet, by a worker and a robot in order
to be machined by the FMS. With the aim to increase productivity and grant human
ergonomics and safety, robot trajectories and task allocation have to be respectively
adequately designed and planned.
Thus, PLATINU M and
its features are leveraged
to implement an integrated
task and motion planning
system capable of selecting
different execution modali-
ties for robot tasks accord-
ing to the expected collab-
oration of the robot with
a human operator. This is
the result of a tight integra-
tion of PLATINU M with a
motion planning system. In-
deed, the pursued approach Fig. 3: Experimental environment
realizes an offline analysis of the production scenarios in order to synthesize a num-
ber of collision-free robot motion trajectories for each collaborative task with different
safety levels. Each trajectory is then associated with an expected temporal execution
bound and represents a tradeoff between "speed" of the motion and "safety" of the hu-
man. The integrated system has been deployed and tested in laboratory on an assembly
case study similar to collaborative assembly/disassembly scenario described above. In
[15], an empirical evaluation is provided in order to assess the overall productivity of
the HRC cell while increasing the involvement of the robots. The idea is to gradually
make free a set of tasks originally preallocated to the human, so to increase the number
of degrees of freedom of PLATINU M during the minimization of the assembly time.
The results show the effectiveness of PLATINU M in finding well suited distribution of
tasks between the human and the robot in different scenarios with an increasing work-
load for the control system. Indeed, the total assembly time was reduced of 65% (from
259s to 169s) and the percentage of tasks assigned by PLATINU M to the robot moved
from 25% to 65%. Thus, PLATINU M instance resulted as capable of increasing the
productivity of the production process without affecting the safety of the operator. A
wider experimental campaign is now undergoing and will constitute an important pillar
for a future longer report.
5 Conclusions
This paper introduces a novel framework in which robot motion planning and task plan-
ning are integrated and their synergisms are exploited to cope with the variability of an
environment in which an industrial robot is acting together with a human worker. After
introducing the integration idea we have described the planning and execution feature
that guarantee robustness in coping with temporal uncertainty. The experiment in the
real case demonstrates the ability of the system to impact the reduction of the makespan,
and to demonstrate time constants able to cope with domain uncertainties.
Acknowledgment. The CNR authors are supported by the European Commission within the
H2020 research and innovation programme, FourByThree project, grant agreement No. 637095.
References
1. Freitag, M., Hildebrandt, T.: Automatic design of scheduling rules for complex manufactur-
ing systems by multi-objective simulation-based optimization. {CIRP} Annals - Manufac-
turing Technology 65(1) (2016) 433 – 436
2. Michalos, G., Kaltsoukalas, K., Aivaliotis, P., Sipsas, P., Sardelis, A., Chryssolouris, G.:
Design and simulation of assembly systems with mobile robots. {CIRP} Annals - Manufac-
turing Technology 63(1) (2014) 181 – 184
3. Pellegrinelli, S., Pedrocchi, N., Tosatti, L.M., Fischer, A., Tolio, T.: Multi-robot spot-welding
cells: An integrated approach to cell design and motion planning. {CIRP} Annals - Manu-
facturing Technology 63(1) (2014) 17 – 20
4. Wolfe, J., Marthi, B., Russell, S.J.: Combined task and motion planning for mobile manip-
ulation. In: Proceedings of the 20th International Conference on Automated Planning and
Scheduling, ICAPS 2010, Toronto, Ontario, Canada, May 12-16, 2010. (2010) 254–258
5. Kaelbling, L.P., Lozano-Pérez, T.: Hierarchical task and motion planning in the now. In:
2011 IEEE International Conference on Robotics and Automation. (2011) 1470–1477
6. Srivastava, S., Fang, E., Riano, L., Chitnis, R., Russell, S., Abbeel, P.: Combined task and
motion planning through an extensible planner-independent interface layer. In: 2014 IEEE
International Conference on Robotics and Automation. (2014) 639–646
7. de Silva, L., Lallement, R., Alami, R.: The hatp hierarchical planner: Formalisation and an
initial study of its usability and practicality. In: 2015 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS). (2015) 6465–6472
8. Py, F., Rajan, K., McGann, C.: A systematic agent framework for situated autonomous
systems. In: AAMAS. (2010) 583–590
9. Lemai, S., Ingrand, F.: Interleaving Temporal Planning and Execution in Robotics Domains.
In: AAAI-04. (2004) 617–622
10. Barreiro, J., Boyce, M., Do, M., Frank, J., Iatauro, M., Kichkaylo, T., Morris, P., Ong, J.,
Remolina, E., Smith, T., Smith, D.: EUROPA: A Platform for AI Planning, Scheduling,
Constraint Programming, and Optimization. In: ICKEPS 2012: the 4th Int. Competition on
Knowledge Engineering for Planning and Scheduling. (2012)
11. Ghallab, M., Laruelle, H.: Representation and control in ixtet, a temporal planner. In: 2nd
Int. Conf. on Artificial Intelligence Planning and Scheduling (AIPS). (1994) 61–67
12. Pellegrinelli, S., Moro, F.L., Pedrocchi, N., Tosatti, L.M., Tolio, T.: A probabilistic approach
to workspace sharing for human-robot cooperation in assembly tasks. {CIRP} Annals -
Manufacturing Technology 65(1) (2016) 57 – 60
13. Cialdea Mayer, M., Orlandini, A., Umbrico, A.: Planning and execution with flexible time-
lines: a formal account. Acta Inf. 53(6-8) (2016) 649–680
14. Umbrico, A., Cesta, A., Cialdea Mayer, M., Orlandini, A.: Platinum: A new framework for
planning and acting. In: AI*IA 2017, Advances in Artificial Intelligence. (2017) To appear
15. Pellegrinelli, S., Orlandini, A., Pedrocchi, N., Umbrico, A., Tolio, T.: Motion planning and
scheduling for human and industrial-robot collaboration. {CIRP} Annals - Manufacturing
Technology (2017) –
16. Umbrico, A., Orlandini, A., Cialdea Mayer, M.: Enriching a temporal planner with re-
sources and a hierarchy-based heuristic. In: AI*IA 2015, Advances in Artificial Intelligence,
Springer International Publishing (2015) 410–423
17. Cesta, A., Orlandini, A., Bernardi, G., Umbrico, A.: Towards a planning-based framework
for symbiotic human-robot collaboration. In: 21th IEEE International Conference on Emerg-
ing Technologies and Factory Automation (ETFA), IEEE (2016)