In this paper we introduce a method for human-robot cooperation, speci cally for assembly-like tasks. Novel approaches for cooperation are needed if we want to enable intuitive and natural interaction between humans and robots in future factories. Our contribution is twofold: i) a framework for the representation of the cooperation task, which allows for run-time adaptation; ii) a dynamic procedure to monitor task execution based on AND/OR graphs. The framework has been experimentally validated in a cooperation scenario in which a Baxter robot and a human perform a screwing task together.
Human-robot cooperation (HRC) is expected to enhance and broaden the role of robots in many real-world scenarios. A possible example is manufacturing, in which humans and robots can play di erent roles in performing certain tasks, depending on their capabilities. Our objective is to explore the use of HRC for assembly-like tasks.
Given the high demand for adaptability and exibility in production lines,
one possibility is to avoid any a priori de ned task structure (i.e., the sequence
of operations), and to allow humans to decide which operation to perform on
the y [
The robot co-worker must: i) recognise human actions, and speci cally their
gestures, using a sort of gesture sensor ; ii) represent all the allowed sequences
of operations, and their mutual relationships, which we model using AND/OR
graphs [
We propose a framework for HRC (in particular for assembly) tasks allowing for task representation and the run-time adaptation to human actions, by means of the execution of proper robot actions. Such adaptation capabilities are enabled by the robot's ability to assess the cooperation context, the AND/OR graph based task representation formalism and the employed graph traversal procedure to adapt to human actions. 2
System's Architecture and Task Representation The system's architecture, shown in Figure 1 on the left hand side, includes a Human Action Recognition (HAR) module, a Controller module, a Planner module and a Task Representation (TR) module.
TR encodes the cooperation task structure as an AND/OR graph.
Furthermore, it interacts with Planner to determine which actions have been performed
and to provide alternatives for next actions to carry out for the human or the
robot, respectively. HAR is a sort of gesture sensor to recognise human actions,
introduced in [
The Controller is based on a task priority based architecture [
As we anticipated, TR uses an AND/OR graph to encode the cooperation process. An AND/OR graph G(N; H) is a structure made up of a set N of n nodes and a set H of h hyper-arcs. Nodes in N de ne reachable states in the graph. Hyper-arcs in H de ne transition relationships among nodes. In particular, hyper-arcs de ne many-to-one transition relationships between many child nodes and one parent node. The relationship between the transitions of a hyper-arc is considered to be in logical and, while the relationship between di erent hyper-arcs of a parent node is in logical or. Both nodes and hyper-arcs are associated with costs.
In our case, each hyper-arc hi models a set of actions Ai, where an action aj 2 Ai can be performed either by a human or a robot during the cooperation process. If the order in which to execute actions in Ai is important, we treat them as a strict sequence. Initially, all the actions in Ai are labelled as un nished ; when they are executed we label them as nished. If all the actions in Ai are nished, then hi is done. Nodes can be either solved or unsolved. A node nk 2 N is solved if there is at least one hyper-arc hi to this node, hi is done and all its child nodes are solved. The leaves in G are initialised as solved or unsolved at the beginning, depending on the initial state of the cooperation. This procedure iterates going upward to the root node of G. When the root is solved, then G is labelled as solved. During graph traversal, nk is feasible if there is at least one hi to it such that all its child nodes are solved. Otherwise, nk is unfeasible and hi is labelled as active. At all times, we have a set of active hyper-arcs Ha H in G.
The temporal task representation state S is the set of all the labelled nodes and hyper-arcs in G. S de nes the possible action alternatives for the human or the robot in cooperation. We de ne as cooperation context a sequence of actions performed by humans or robots during the cooperation, corresponding to an allowed traversal path in G. When a new human action an is detected by HAR, Planner interacts with TR to determine whether an belongs to an Ai such that the latter corresponds to an hyper-arc in Ha. If this does not happen, i.e., an does not belong to the current cooperation context, the robot enters a null mode and waits for further knowledge to determine which traversal path in G is involved. If there are multiple active hyper-arcs possibly involving an, the robot enters an ambiguous mode. Otherwise, the next action to suggest or perform is de ned in G so that the overall cost of the traversal path is minimised.
Examples We propose a framework allowing for the representation of cooperative tasks using AND/OR graphs, and their execution leaving a human free to choose the speci c sequence of operations to perform among a number of alternatives. The developed architecture is able to adapt to human actions at run-time, which are detected as gestures using wearable devices worn at the human's right wrist. The framework has been validated by modelling a cooperative assembly task, in which a human and a Baxter dual-arm manipulator perform turn-taking actions in any allowed sequence. (d) (h) (l) (d) (h) (l) (a) (e) (i) (a) (e) (i)
Darvish et al.