- Moving robots from their carefully designed and encapsulated work cells into the open, less structured human workspace for collaboration with workers requires robust error detection and recovery strategies. Foreseeing all possible uncertainties and unexpected events and to program in recovery actions at setup time is unfeasible. Online learning of nominal execution behaviour and automatic detection of anomalies using an Extended Markov Model, combined with interactively trained Bayesian networks for mapping anomalies to error causes and recovery actions, enables automatic recovery from previously experienced errors. A three-layered user-friendly model of errors-causesresponses and a simple GUI allows non-expert user to define new recovery activities and error causes when not yet handled anomalies occur.
Today’s robot systems for industrial applications rely on a structured environment to avoid errors. Parts, fixtures, tools and stations have defined positions and the workspace is encapsulated to avoid intruders that could possibly endanger this defined environment. Expected exceptions from the nominal case that were either foreseen during the planning of the robot system, or occurred during the setup phase of the system are coped with by integrating additional sensors, adapting tool-, fixture and part geometries and adding additional branches to the robot program to cope with these deviations. Furthermore, as many robotic systems are complicated, any exceptions and breakdowns occurring after system setup often require external technicians or engineers to diagnose and solve problems.
Such strictly controlled and carefully designed work cells are only economically feasible if the designed robot system will run unobstructed for a long time. Small and midsized enterprises (SMEs) are often characterized by a much more agile production style and consequently rely on human workspaces. Moving robots out of their strictly controlled and carefully designed spaces into human workspaces, which are by nature unstructured environments with a high degree of uncertainty, requires significantly enhanced robustness towards unforeseen events and geometric or other uncertainties. (The additional need for safety measures to protect the human co-worker from injuries is out of scope of this work, see e.g. [22], [12] and many others.) A SME suitable robot system therefore needs semi automatic exception handling and error recovery capabilities that allow non-expert users to manage exceptions (internally and externally triggered) occurring in daily operation. We propose a novel skill-based exception handling and error recovery approach that allows non-robot expert users to operate a robotic system embedded in a human-centric workspace. We briefly introduce our execution model, detail the Extended Markov Chain based Situation Awareness, which forms the base for Exception Handling, and the Error Recovery module employing a Bayesian network and betabinomial inference algorithm. The prosed system has been implemented in a pick & place and in an assembly work cell, which are finally presented. Research in exception handling is related to the area of error or fault recovery [17]. Error recovery has been defined as “the process by which the system returns to a state where production can restart after an abnormal and disruptive condition has occurred” [23]. For a robot coworker to effectively handle an exception, whether through informing the human worker or resolving the problem by itself, the types of faults that typically occur in the manufacturing robotic assembly cases needs to be understood. Fault taxonomies have been presented in other related fields, including mobile robots [7], computing [3], autonomous robots in RoboCup [21], workflow systems [16], serviceoriented architecture [6], and web service [8]. Reports show that many errors in manufacturing systems, including CNC machines, are hardware related and that approximately 60% of all stoppages are due to tool breakdown [23]. However, there has been a lack of study on the likelihood of common errors and exceptions occurring during assembly tasks involving collaborative robots. One of the reasons can be that robot coworkers have not yet proven to be robust enough for industry application to be studied and generalized based on real assembly cases [14].
At the base of the system is a Skill Execution Engine, which allows a more goal-oriented task description than strict motion based programming or planning. Without going into details of the skill-model [1], we assume skills to be independent, sensor-based motion or handling primitives that adapt themselves to position uncertainties and other deviations from an ideal state using build-in sensing and monitoring as well as (limited) internal error recovery. Robot tasks are constructed by chaining skills and control flow instructions, forming a state machine [2] based on SCXML1. While skills detect deviations from their expected performance and report these, the skill executor by itself does not provide any error recovery functionality. Features
1 Apache Commons SCXML executor, http://commons.apache.org/proper/commons-scxml/. Page 5 of the skill executor that allow the implementation of error recovery functionality at higher layers are:
The skill executor knows and publishes the current state of the system at any time. This allows an error recovery module to relate errors on the one hand to specific skill models and on the other hand to specific application steps and therefore to draw conclusions like “this is an error that is very typical for a pick operation” or “this is an error that occurred already in the past at this specific execution step of the application”.
The skill executor has an interface for an error recovery module to stop and later continue the execution of the skill based application program thereby allowing worker interaction to recover from errors detected by an error recovery module.
The skill formalism used by the skill executor is built on the concept of reusable hierarchical skills that are easy to enhance or adapt. It is therefore easily possible to include additional mechanisms into an existing skill model to cope with errors that could be detected by the system but just have not been considered yet.
The Situation Assessment (SA) constantly monitors the overall situation (robot task execution) using data published by the skills executor as well as by additional sensors dedicated for situation assessment. Deviations flagged by the SA are further examined by the Exception Handling (EH), which devises a possible cause and corrective measure, potentially involving user interaction. The whole system of skill executer, situation assessment and exception handling is collectively referred to as “Exception Handling Framework” or “EHF”.
The role of Situation Assessment (SA) is to learn and
monitor the (correct) skill execution and detect non-nominal
conditions. Deviations from the learned, nominal behaviour
are interpreted as Anomalies, which are passed on to the
Exception Handler (section V). Our implementation of SA
is based on prior work by [4] and [5], where SA was applied
to mobile robotics. We implemented and expanded SA to
learn skill based execution in a collaborative robotic system.
To learn how to perform a skill correctly, SA captures the
essence of the skill by learning the timing and sequence of
events that make up the skill. Our approach is to generate
one parameterized model that includes parameters in the
space and time domain. SA learns the sequence of events
within a skill execution by learning a set of parameters with
a temporal component, recording the transition from one
instantiation of the parameters to the next:
p( ) = ( 1, 2, … , )
(
(
Situation Assessment uses a Situation Model as a
template description to fuse together the different data points
for learning a Situation. The components of the Situation
Model ( in (
, x , y , , , , ] (
The component of a is a data point that
uniquely identifies the current primitive being executed in
the skill. In this case, the unique primitive ID provides the
understanding of time while the understanding of space is
provided by the F/T data. By using the primitive ID we can
learn a skill time invariantly. This means that SA will only
learn the sequence of the events and is invariant towards the
duration of the execution of specific primitives. We have
found this feature particularly useful when the duration of
the primitives or skills is stochastic. Should it be necessary
to catch anomalies in relation to when events occur (e.g. too
early or late), the primitive ID in (
The Situation Model serves as a template describing
which sensors SA should fuse together into one single state.
In general, all data points in the Situation Model have to be
real numbers. This allows the computation of one single
metric for each in (
SA can autonomously learn a skill without the user having to manually specify the states of a skill. We have implemented a spatiotemporal model that allows for online dynamic learning of states over time. For this purpose, we are currently using the Extensible Markov Model (EMM) as it is useful for online learning of sequences of states [10]. An example of a dynamically learned model using the EMM algorithm can be seen in Figure 1. In this example, a robot is picking up a nut from a table and placing it on a pipe in a single nonrecurring operation (therefore an open-ended chain). The EMM is also useful in learning looped tasks. Page 6
SA has two modes of operation: learning and detection
during execution. In the learning mode, SA monitors the
data points specified in the Situation Model and builds the
Situation for the skill that is being learned. During execution
of the same skill, SA loads the saved Situation and applies
the same clustering process as during learning. However,
should the clustering of the data result in a new state in (
The task of EH is to receive an Anomaly from SA and provide a suggested solution that is most likely to solve the problem. For each robotic system, EH maintains a hierarchical four-layered Bayesian network with all exceptions and solutions relevant to that cell. The hierarchical structure allows EH to reason about the most suitable solution to a problem. EH provides the suggested solution to the user along with all other possible solutions. The user is free to select the suggested solution, any other solution or to create a new solution. The selection is stored in EH as a sample of user solution preference. Such samples are used in priming the network for inference with future anomalies. With the feedback of user samples, a closed preference-learning loop is formed to provide suggestions for solutions to future anomalies. In this section, we provide a detailed description of EH and begin with the role of the Exception Scenario ES in EH.
The Exception Scenario (ES) is designed as a
fourlayered model consisting of Anomaly, Error, Fault and
Response, inspired by work in [18]. The hierarchy is a
fourlayered binary Bayesian network that facilitates inferring the
most likely Response (solution) to an Anomaly (a
deviation), an example is shown in Figure 2. At the lowest
level of the network, Anomaly nodes model anomalies
detected by SA. Each Anomaly node corresponds to a data
component (di of (
Figure 2 is an example of a Bayesian network with three
Exception Scenarios for the two Anomaly nodes of the
sensors and in (
The process of inferring a Response to an Anomaly in
the Bayesian network, is the inference process of the EH.
This process is an implementation of Bayes’ theorem:
∝
∙
ℎ
(
A prerequisite for performing inference is the calculation
of prior probabilities. As described in section IV, a feature
of the EHF is to learn the user-preferred solution of a given
anomaly. When the user selects a specific Exception
Scenario (i.e. an Error, a Cause and a Solution) to solve a
problem, it is fed back to the database as a sample of the user
selection, thus learning the preference of selecting this
Exception Scenario for a specific Anomaly. The sample data
is used to calculate the prior probability for each node of the
network. We treat calculating the node’s prior probability as
an inference process that adds another layer of Bayesian
inference as described in (
~Binom( , ) (
( | , ) = Be( , )
(
In (
In Figure 3, examples of Beta distributions for different
values of and are shown. Distribution 1: e( = 1, =
1) is a uniform distribution offering an uninformative prior
with a mean, = 0.5 and a high variance (uncertainty) due
to the low sample size. In this case, the posterior will largely
be determined by the data. Distribution 4: e( = 30, = 5)
has = 0.86 and a smaller variance, thus providing a
comparably less uncertain estimate of the user selection
preference, . The sequence of graphs 1-4 in Figure 3, can
be seen as an example of a continuous learning cycle,
starting with no knowledge of user selection (a uniform
distribution with no samples) towards more informative
distributions 2-4 as the sample size increases. When a new
node is created with no samples available ( = = 1), the
Beta distribution is uniform. However, to avoid the
uninformative uniform distribution we propose to query the
user to provide a subjective estimate of the selection (the
mean) of this node along with a confidence level (the
variance). Using the equations for the mean (
The Bayesian network described in section V.B and Fig. 2 receives evidence in the form of Anomaly information gathered by SA. In the example shown in Figure 2, SA has detected that the gripper was unexpectedly fully open (thus providing evidence that Gripper Open = true, Gripper Closed = false. The evidence is in practice introduced to the network by clamping the two nodes to their respective values. Page 8
After introducing evidence, posterior probabilities for all nodes are calculated. We have used the SMILE reasoning engine [9] for inference. The Response having the highest posterior probability is selected as the suggested solution. For each Response, the tree is descended towards the root Anomaly nodes, thus mapping out each possible path towards the root. The resulting list will have the most probable ES listed first with all other less likely alternative ES following in descending probability.
Whenever an anomaly is detected, the error layer classifies it into an error cause. If no cause is found the user is inquired and given the option to assign an existing cause, dismiss the anomaly as not indicating an error or to create a new cause (including a resolution, if known). Figure 4 shows the dialog box after successfully mapping an anomaly to an error and further to a recovery action. The user can accept this solution or add a new solution. Figure 5 shows the corresponding dialog box for adding a new triplet of error, error cause and recovery action. The dialog boxes shown in Fig. 4 and 5 are designed for use at system runtime and therefore as simplistic as possible. A more elaborated interface for managing the entire network of anomalies, errors, causes and recovery actions is also provided and targeted at specifically trained users that setup a new robot application.
Within the scope of SMErobotics, this framework has been intensively evaluated using various experiments. A detailed example is the failure to grasp as described in the following section. We have tested the system’s ability to learn the preference of selecting a solution by manually introducing the Gripper Open (GO) Anomaly, shown in Fig. 6, during the execution of a skill. In this test, we have tested the system’s ability to learn the user preference of selecting the Repair Actuator (RA) Response over the Replace Pneumatics (RP) Response. For the purpose of the test, the system had initially no knowledge of user selections (samples), except for five samples confirming the choice of the RP Response as the user preferred solution to the GO
Anomaly. Hereafter, we introduced the GO Anomaly repeatedly, selecting the RA Response as the solution each time. This process was repeated until EH started to suggest the RA Response, thus demonstrating EHF’s ability to learn the user preference of selecting the RA Response over the RP.
Test results are shown in Fig. 6. Initially, EH has five samples confirming the selection of the RP Response for the
GO Anomaly. Thus, when the GO Anomaly is introduced, EH suggests RP as the most suitable Response to the GO Anomaly with probability ~ 0.553. However, the user ignores the EH suggested RP Response and instead selects RA. Thus, when the GO Anomaly is introduced again, EH now has six samples (five for RA and one for RP), computing the most likely Response to be RP with probability ~ 0.545, and so on. At RA sample 5, EH computes the probability for each Response being identical (~ 0.526). Again, the GO Anomaly is introduced and this time EH suggests the RA Response with posterior probability ~ 0.535. Thus, with five samples confirming RP, it took six samples of RA for EH to suggest RA.
Through the test results in section VII, we showed that EH is able to learn the user preference of selecting a solution, even when it had learned a different preference earlier. As the user selects a specific solution to an Anomaly, the solution becomes more probable for future selection. This is normally helpful, but can be problematic if the user wishes the system to select a different solution, since learning a new preference can take several iterations, as the test results showed. This is especially true when the sample count for the prior solution is high. A possible future solution could be Page 9 to introduce additional information in the Error Layer, e.g. condition the error cause not only on the counting of user selections, but also on the state of various system variables at time of the user selection.
The system is currently being integrated in further demonstrators in the context of the SMErobotics project and will see more in-depth testing and possibly enhancements in these demonstrators. Concept videos of these showing the SMErobotics vision of future industrial robotics are available at http://video.smerobotics.org; especially the D2 and D3 videos are relevant in the context of this work.