We address the problem of imitation learning in interactive robots which learn from task demonstrations. Many current approaches to interactive robot learning are performed over a set of demonstrations, where the robot observes several demonstrations of the same task and then creates a generalized model. In contrast, we aim to enable a robot to learn from individual demonstrations, each of which are stored in the robot's memory as source cases. When the robot is later tasked with repeating a task in a new environment containing a di↵erent set of objects, features, or a new object configuration, the robot would then use a casebased reasoning framework to retrieve, adapt, and execute the source case demonstration in the new environment. We describe our ongoing work to implement this case-based framework for imitation learning in robotic agents.
Imitation is an essential process in human social learning and cognition [
The goal of Learning from Demonstration is to enable imitation learning in
robots through interactive demonstrations, provided through methods such as
teleoperation or kinesthetic teaching [
However, a challenge of this process is that the resulting action model is dependent on the number of demonstrations that were provided for that particular task. We also assume that the robot has been exposed to enough variations of the initial configuration such that its generalized model can be applied to a wide range of related initial configurations. As such, the generalized model is restricted to application in environments which are similar to those demonstrated.
We describe our preliminary work toward defining an alternate approach to imitation learning in robotics, one which takes a case-based approach in which the robot stores demonstrations individually in memory. We define a case-based framework which enables the full imitation learning process, from observing a task demonstration to transfer and execution. We also define a case representation which encodes task demonstrations for storage in source case memory. 2
Case-based reasoning has been used to address the problem of transfer in robotics
domains. Floyd, Esfandiari & Lam [
While these approaches do address knowledge transfer for robotic and
simulated agents, they primarily operate over input and output represented at a
higher level of abstraction, such as actions at a strategic level. The goal of our
work is to enable transfer to generate action at a lower level of control and in
response to real-world perceptual input, where we transfer the demonstrated
action trajectory used to achieve a task. We expand on our previous work [
We previously provided a general outline for imitation in [
We have revised our case-based approach to transfer (originally summarized
in [
Our eventual goal is to enable transfer for imitation learning in scenarios such as the following. A human teacher guides the robot to complete a task such as scooping the contents of one container into another. During the demonstration, the robot records the demonstrated trajectories and object features. At a later time, the robot is asked to repeat the scooping task, but in a new, target environment. Thus, the robot must use a di↵erent set of object features to parameterize and execute the scooping task than those observed in the original, source environment. Next, the robot transfers its representation of the scooping task to accommodate for the di↵erences between the source and target environments. The transferred task representation is then executed in the target environment.
Rather than generalize over a set of demonstrations as in current Learning
from Demonstration methods (surveyed in [
Demonstration and Learning We have implemented the first step in the
Case Storage process, where the robot records and stores each task
demonstration as a source case in memory. We define each case as the tuple C =
<L, D, T, O, Si, Sf >, where:
– L represents the label of the task which was demonstrated, e.g. ”scooping”.
– D represents the set of action models which encode the demonstrated motion,
represented as Dynamic Movement Primitives as defined in [
At a later time, the robot may be asked to repeat the task in a new, target environment. We are currently implementing the Case Adaptation process shown in Figure 1.
Observation will begin when the robot is asked to address a target problem. We assume that the robot has been provided a relevant source case which it can retrieve from memory to address the given target problem. The robot will then observe the target environment by viewing the objects located in the tabletop environment using an overhead camera. This will provide it with the target case’s initial state Si.
Retrieval must be performed to select a source case from memory containing the demonstration that is most relevant to the current target problem. Case retrieval will prioritize (i) similarity of task goals, (ii) similarity of salient objects, and finally, (iii) similarity of initial states. Once a relevant source case has been retrieved, the Mapping step must encode the di↵erences between the source and target environments. This mapping will be later used to transfer the source case such that di↵erences in the target environment are addressed.
Given a source case and mapping which encodes the di↵erences between the source and target cases, the Transfer step adapts the source case. We take a similarity-based approach to transfer, where we consider the similarity between the source case and target environments when defining transfer processes. As we encounter transfer problems in which the source and target problems become less similar, the source case is transferred at a di↵erent level of abstraction, such that only high-level features of that case are transferred. The adapted case is then executed in the target environment.
We have implemented three methods which implement the Transfer step, each of which operates by transferring the source case at a di↵erent level of abstraction. Once the source case has been transferred, it is used to plan and execute a new action trajectory. In preliminary experiments, we have evaluated each method separately such that we selected the level of abstraction at which transfer occurred in each target problem. These experiments have shown us that by changing the level of abstraction at which a case is transferred, a robot can use a single source demonstration to address target environments of varying similarity to the source environment. 4
We have implemented the Case Storage process and the last two steps of the Case Adaptation process, the Transfer and Execution steps. Currently, we manually provide the robot with the most relevant source case demonstration and a mapping between objects in the source and target environments. Thus, our next steps are to identify a method for autonomously determining this object mapping. Furthermore, future work will involve defining a process for identifying and retrieving an appropriate source case demonstration that is most applicable to a given transfer problem.
This work was supported by NSF Graduate Research Fellowship DGE-1148903.