A Timeline-based Planning System for Human-Robot Collaboration in Manufacturing Domains

Abstract

Industrial robots have demonstrated their capacity to meet the needs of many applications, offering accuracy and efficiency. However, when robotworker collaboration is needed, safety represents a key aspect and needs to be enforced in a comprehensive way. In this regard, seamless and safe human-robot collaboration still constitutes an open challenge in manufacturing. FourByThree is an ongoing research project funded by the European Commission and aimed to design, build and test pioneering robotic solutions able to collaborate safely and efficiently with human operators in industrial manufacturing companies. The paper presents the ongoing work in the project related to a task planning framework specifically designed and tailored to address the challenges related to humanrobot collaborative production processes.

-

Industrial robots have demonstrated their capacity to meet the needs of many applications, offering accuracy and efficiency. However, when robot-worker collaboration is needed, safety represents a key aspect and needs to be enforced in a comprehensive way. In this regard, seamless and safe human-robot collaboration still constitutes an open challenge in manufacturing. FourByThree [ 1 ] is an ongoing research project3 aimed to design, build and test pioneering robotic solutions able to collaborate safely and efficiently with human operators in industrial manufacturing companies. Its overall aim is to respond to the above challenge by creating a new generation of robotic solutions, based on innovative hardware and software, which present four main characteristics: modularity, safety, usability and efficiency And considers three different actors: humans, robots and the environment.

The resulting robotic solutions of the project will be tested in four pilot implementations, which correspond to real industrial needs and are representative of the two possible robot-human relationships in a given workplace without physical fences: coexistence (human and robot conduct independent activities) and collaboration (they work collaboratively to achieve a given goal). During the project, two different categories of pilot studies are considered. Three pilots correspond to production industries related to different realistic scenarios in which robotic co-workers are considered to perform 3 http://www.fourbythree.eu assembly/disassembly tasks, conventional production tasks (e.g., deburring, welding, etc.) and working processes involving large parts. The fourth pilot study will be used as a living lab for experimenting with a big number of subjects, mainly during the development process.

This extended abstract presents the ongoing work in the project related the definition of safety strategies and control mechanisms generated by means of a task planning framework [ 2 ] specifically designed and tailored to address the challenges related to human-robot collaborative production processes. 2

Human-Robot Collaborative Scenarios

A human-robot collaboration workcell can be considered as a bounded connected space with two agents located in it, a human and a robot system, and their associated equipment [ 3 ]. A robot system in a workcell consists of a robotic arm with its tools, its base and possibly additional support equipment. The workcell also includes the workpieces and any other tool associated with the task and dedicated safeguards (physical barriers and sensors such as, e.g., monitoring video cameras) in the workcell space. In such workcell, different degrees of interaction between a human operator and the robot can be considered [ 4 ]. In all these cases, it is assumed that the robot and the human may need to occupy the same spatial location: Independent, the human and the robot operate on separate workpieces without collaboration, i.e., independently from each other. Synchronous, the human and the robot operate on sequential components of the same workpiece, i.e., one can start a task only after the other has completed a preceding task. Simultaneous, the human and the robot operate on separate tasks on the same workpieces at the same time. Supportive, the human and the robot work cooperatively in order to complete the processing of a single workpiece, i.e., they work simultaneously on the same task. Different interaction modalities requires the robot endowed with different safety settings while executing tasks. 3

Dynamic Task Planning for Safe Human-Robot Collaboration As part of the overall FourByThree (ROS-based) control architecture, a dynamic task planner is to provide continuous task synthesis features, safety critical properties at execution time, and user modeling ability for adapting tasks to the particular human at work. The integration of plan synthesis and continuous plan execution has been demonstrated both for timeline based planning (e.g., [ 5 ]) and PDDL based (e.g., [ 6 ]). In scenarios of human robot interaction important problems have been addressed: (a) ”human aware” planning has been explored for example in [ 7 ], (b) the interaction of background knowledge for robotic planning in rich domain (addressed for example in [ 8 ], (c) synthesis of safety critical plans to guarantee against harmful states (relevant in co-presence with humans) is addressed in [ 9 ] and [ 10 ]). Within the FourByThree project, a timelinebased planning approach is pursued relying on the APSI-TRF software infrastructure [ 11 ], made available by European Space Agency, and improved from the initial proposal [ 12 ] and its test in several missions. Then, a FourByThree planning framework HR Coll.

Process

HMI Worker

Worker Preferences/ Commands

Task Planning Model

Knowledge Eng. Services V&V Services

4x3 Task Planner

Knowledge Engineer Task Plan Execu1ve System 4x3 Archi (ROS) has been designed to deploy a continuous task planning and adaptation system with humans in the loop.

Production Engineer

Task Decomposition

The overall framework is depicted in Figure 1. A Production Engineer is in charge of defining the Human-Robot collaborative (HRC) production process characterizing each task according to specific HRC settings (i.e., interaction modalities). Then, a Knowledge Engineer is to encode such information in a task planning model following a hierarchical decomposition and leveraging the features provided by an environment for Knowledge Engineering of Planning with Timelines, called KEEN [ 13 ], that integrates “classical” knowledge engineering features with Verification and Validation (V&V) formal techniques to perform domain model validation, planner validation, plan verification, etc. The integration of Planning and Scheduling (P&S) technology with V&V techniques is key to synthesize a safety critical controller for the robot. The Task Planning Model can be, then, adapted also according to the preferences of the Human Worker that is supposed to interact with the robot during the production process. A FourByThree Task Planner then generates a temporally flexible task plan to be dispatched to the robot through an Executive System (integrated in the ROS-based architecture). During the production process, the Executive System is also in charge of monitoring the plan execution and, in case of need (e.g., a specific command issued by the human worker), ask the task planner to dynamically face modifications of the production environment. 4

Conclusions

The dynamic task planning framework briefly described above is to provide the control architecture with suitable deliberative features relying on the control model generated by the Knowledge Engineering according to the definition provided by the Production Engineer and the preferences of the Human Worker. An off-the-shelf planning and execution system based on APSI-TRF is then deployed to synthesize a suitable set of actions (i.e., in this work a timeline-based plan) that when executed controls the mechatronic device.

Acknowledgment. The CNR authors are supported by the European Commission within the H2020 research and innovation programme, FourByThree project, grant agreement No. 637095.

References

1. Maurtua , I. , Pedrocchi , N. , Orlandini , A. , de Gea Fernandez , J. , Vogel , C. , Geenen , A. , Althoefer , K. , Shafti , A. : Fourbythree: Imagine humans and robots working hand in hand . In: ETFA 2016. The 21st IEEE International Conference on Emerging Technologies and Factory Automation , IEEE ( 2016 )

2. Cesta , A. , Bernardi , G. , Orlandini , A. , Umbrico , A. : Towards a planning-based framework for symbiotic human-robot collaboration . In: ETFA 2016. The 21st IEEE International Conference on Emerging Technologies and Factory Automation , IEEE ( 2016 )

3. Marvel , J.A. , Falco , J. , Marstio , I. : Characterizing task-based human-robot collaboration safety in manufacturing . IEEE Trans. Systems, Man, and Cybernetics: Systems 45 ( 2 ) ( 2015 ) 260 - 275

4. Helms , E. , Schraft , R.D. , Ha¨gele, M.: rob@work: Robot assistant in industrial environments . In: 11th IEEE International Workshop on Robot and Human Interactive Communication , IEEE ( 2002 ) 399 - 404

5. Py , F. , Rajan , K. , McGann , C. : A Systematic Agent Framework for Situated Autonomous Systems . In: AAMAS-10. Proc. of the 9th Int. Conf. on Autonomous Agents and Multiagent Systems . ( 2010 )

6. Cashmore , M. , Fox , M. , Larkworthy , T. , Long , D. , Magazzeni , D. : AUV mission control via temporal planning . In: ICRA 2014. The IEEE International Conference on Robotics and Automation , Hong Kong, China, May 31 - June 7, 2014 . ( 2014 ) 6535 - 6541

7. Sisbot , E.A. , Alami , R.: A human-aware manipulation planner . IEEE Trans. Robotics 28 ( 5 ) ( 2012 ) 1045 - 1057

8. Lemaignan , S. , Alami , R.: Explicit knowledge and the deliberative layer: Lessons learned . In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2013 . ( 2013 ) 5700 - 5707

9. Abdellatif , T. , Bensalem , S. , Combaz , J., de Silva, L. , Ingrand , F. : Rigorous design of robot software: A formal component-based approach . Robotics and Autonomous Systems 60 ( 12 ) ( 2012 ) 1563 - 1578

10. Orlandini , A. , Suriano , M. , Cesta , A. , Finzi , A. : Controller synthesis for safety critical planning . In: ICTAI 2013. The IEEE 25th International Conference on Tols with Artificial Intelligence , IEEE ( 2013 ) 306 - 313

11. Cesta , A. , Fratini , S.: The Timeline Representation Framework as a Planning and Scheduling Software Development Environment . In: PlanSIG-08. Proc. of the 27th Workshop of the UK Planning and Scheduling Special Interest Group, Edinburgh, UK , December 11 - 12 . ( 2008 )

12. Cesta , A. , Cortellessa , G. , Fratini , S. , Oddi , A. : Developing an End-to-End Planning Application from a Timeline Representation Framework . In: IAAI-09. Proc. of the 21st Innovative Application of Artificial Intelligence Conference , Pasadena, CA, USA. ( 2009 )

13. Orlandini , A. , Bernardi , G. , Cesta , A. , Finzi , A. : Planning meets verification and validation in a knowledge engineering environment . Intelligenza Artificiale 8 ( 1 ) ( 2014 ) 87 - 100