=Paper=
{{Paper
|id=Vol-1520/paper5
|storemode=property
|title=A Case-Based Framework for Task Demonstration Storage and Adaptation
|pdfUrl=https://ceur-ws.org/Vol-1520/paper5.pdf
|volume=Vol-1520
|dblpUrl=https://dblp.org/rec/conf/iccbr/FitzgeraldG15
}}
==A Case-Based Framework for Task Demonstration Storage and Adaptation==
https://ceur-ws.org/Vol-1520/paper5.pdf
53
A Case-Based Framework for Task
Demonstration Storage and Adaptation
Tesca Fitzgerald, Ashok Goel
School of Interactive Computing,
Georgia Institute of Technology,
Atlanta, Georgia, 30332
{tesca.fitzgerald,goel}@cc.gatech.edu
Abstract. We address the problem of imitation learning in interactive
robots which learn from task demonstrations. Many current approaches
to interactive robot learning are performed over a set of demonstrations,
where the robot observes several demonstrations of the same task and
then creates a generalized model. In contrast, we aim to enable a robot
to learn from individual demonstrations, each of which are stored in the
robot’s memory as source cases. When the robot is later tasked with re-
peating a task in a new environment containing a di↵erent set of objects,
features, or a new object configuration, the robot would then use a case-
based reasoning framework to retrieve, adapt, and execute the source
case demonstration in the new environment. We describe our ongoing
work to implement this case-based framework for imitation learning in
robotic agents.
Keywords: Case-based agents, imitation learning, robotics
1 Introduction
Imitation is an essential process in human social learning and cognition [11, 10].
Imitation learning occurs when a learner observes a teacher demonstrating some
action, providing knowledge of (i) how the action was performed and (ii) the
resulting e↵ects of that action. This interaction-guided learning method allows us
to learn quickly and e↵ectively. As a result of its importance in human cognition,
it follows that imitation learning has become an area of increasing focus in
interactive robotics research as well.
The goal of Learning from Demonstration is to enable imitation learning in
robots through interactive demonstrations, provided through methods such as
teleoperation or kinesthetic teaching [1, 2]. Regardless of which demonstration
method is used, the following process is often used. First, the human teacher
provides several demonstrations of a skill. Between demonstrations, the teacher
may adjust the environment such that the skill is demonstrated in a variety of
initial configurations. The robot then creates an action model which general-
izes over the provided demonstrations. Lastly, the robot applies the generalized
action model to plan a trajectory which is executed in a new environment.
Copyright © 2015 for this paper by its authors. Copying permitted for private and
academic purposes. In Proceedings of the ICCBR 2015 Workshops. Frankfurt, Germany.
54
However, a challenge of this process is that the resulting action model is de-
pendent on the number of demonstrations that were provided for that particular
task. We also assume that the robot has been exposed to enough variations of
the initial configuration such that its generalized model can be applied to a wide
range of related initial configurations. As such, the generalized model is restricted
to application in environments which are similar to those demonstrated.
We describe our preliminary work toward defining an alternate approach to
imitation learning in robotics, one which takes a case-based approach in which
the robot stores demonstrations individually in memory. We define a case-based
framework which enables the full imitation learning process, from observing a
task demonstration to transfer and execution. We also define a case representa-
tion which encodes task demonstrations for storage in source case memory.
2 Related Work
Case-based reasoning has been used to address the problem of transfer in robotics
domains. Floyd, Esfandiari & Lam [7] describe a CBR approach to learning
strategies for RoboCup soccer by observing spatially distributed soccer team
plays. Their approach represents each case as an encoding of a single agent’s
perception and resulting action at a given time. Thus, they transfer the behav-
ior of an agent when it perceives a situation similar to that of the observed
agent. More recently, Floyd & Esfandiari [6] describe an approach for case-based
learning by observation in which strategy-level domain-independent knowledge
is separated from low-level, domain-dependent information such as the sensors
and e↵ectors on a physical robot. Ontañón et al. [8] describe their approach to
observational learning for agents in real-time strategy games. They use a case-
based approach to online planning, in which agents adapt action plans which are
observed from game logs of expert demonstrations.
While these approaches do address knowledge transfer for robotic and sim-
ulated agents, they primarily operate over input and output represented at a
higher level of abstraction, such as actions at a strategic level. The goal of our
work is to enable transfer to generate action at a lower level of control and in
response to real-world perceptual input, where we transfer the demonstrated
action trajectory used to achieve a task. We expand on our previous work [3] de-
scribing a case-based approach to interpretation and imitation in robotic agents.
We discussed two separate processes: (i) interpreting new skill demonstrations
by comparing it to previously observed demonstrations using a case based pro-
cess (further described in [5]), and (ii) a related process for imitating a task
demonstration. This paper expands on the latter process, case-based imitation.
We previously provided a general outline for imitation in [3] in which four
steps occur: representation of the task demonstration at multiple levels of ab-
straction, retrieval of the most relevant source case from memory, adaptation
of the source case to address the target problem, and execution of the adapted
case in the target problem. In this paper, we describe our more recent work pro-
viding (i) a revised, complete process of imitation beginning with observation
55
Fig. 1. Case-Based Process for Task Demonstration Transfer
of the task demonstration and ending with task transfer and execution, (ii) a
Mapping step which bridges the gap between the Retrieval and Transfer steps,
and (iii) a revised case representation for storing task demonstrations (iterating
on preliminary work introduced in [4]).
3 Approach Overview
We have revised our case-based approach to transfer (originally summarized
in [3]) to consist of two separate processes, as shown in Figure 1: the Case
Storage process in which the robot receives demonstrations of a task and stores
each demonstration as a case in source memory, and a Case Adaptation process
which is used at a later time when the robot is asked to repeat a task in a target
environment.
3.1 Why a CBR approach?
Our eventual goal is to enable transfer for imitation learning in scenarios such
as the following. A human teacher guides the robot to complete a task such as
scooping the contents of one container into another. During the demonstration,
the robot records the demonstrated trajectories and object features. At a later
time, the robot is asked to repeat the scooping task, but in a new, target environ-
ment. Thus, the robot must use a di↵erent set of object features to parameterize
and execute the scooping task than those observed in the original, source en-
vironment. Next, the robot transfers its representation of the scooping task to
accommodate for the di↵erences between the source and target environments.
The transferred task representation is then executed in the target environment.
Rather than generalize over a set of demonstrations as in current Learning
from Demonstration methods (surveyed in [1, 2]), using a case-based approach
allows us to: (1) operate under the assumption that the human teacher will
provide a limited number of demonstrations, (2) represent demonstrations as
individual experiences in the robot’s memory, and (3) utilize a complete frame-
work for transferring skill demonstrations, which includes the steps of retrieving,
analyzing, transferring, and executing a relevant source case demonstration in
an unfamiliar, target environment.
3.2 Case Storage Process
Demonstration and Learning We have implemented the first step in the
Case Storage process, where the robot records and stores each task demon-
56
stration as a source case in memory. We define each case as the tuple C =
, where:
– L represents the label of the task which was demonstrated, e.g. ”scooping”.
– D represents the set of action models which encode the demonstrated motion,
represented as Dynamic Movement Primitives as defined in [9].
– T is the set of parameterization functions which relate the set of action
models to the locations of objects in the robot’s environment. For example,
a parameterization function may be used to represent how the robot’s hand
must be located above a bowl prior to completing a pouring action.
– O is the set of salient object IDs which are relevant to the task.
– Si and Sf are the initial and final states, respectively, which represent the
set of objects observed in an overhead view of the robot’s environment.
3.3 Case Adaptation Process
At a later time, the robot may be asked to repeat the task in a new, target
environment. We are currently implementing the Case Adaptation process shown
in Figure 1.
Observation will begin when the robot is asked to address a target problem.
We assume that the robot has been provided a relevant source case which it
can retrieve from memory to address the given target problem. The robot will
then observe the target environment by viewing the objects located in the table-
top environment using an overhead camera. This will provide it with the target
case’s initial state Si .
Retrieval must be performed to select a source case from memory containing
the demonstration that is most relevant to the current target problem. Case
retrieval will prioritize (i) similarity of task goals, (ii) similarity of salient objects,
and finally, (iii) similarity of initial states. Once a relevant source case has been
retrieved, the Mapping step must encode the di↵erences between the source and
target environments. This mapping will be later used to transfer the source case
such that di↵erences in the target environment are addressed.
Given a source case and mapping which encodes the di↵erences between the
source and target cases, the Transfer step adapts the source case. We take a
similarity-based approach to transfer, where we consider the similarity between
the source case and target environments when defining transfer processes. As we
encounter transfer problems in which the source and target problems become
less similar, the source case is transferred at a di↵erent level of abstraction, such
that only high-level features of that case are transferred. The adapted case is
then executed in the target environment.
We have implemented three methods which implement the Transfer step,
each of which operates by transferring the source case at a di↵erent level of
abstraction. Once the source case has been transferred, it is used to plan and
execute a new action trajectory. In preliminary experiments, we have evaluated
each method separately such that we selected the level of abstraction at which
transfer occurred in each target problem. These experiments have shown us that
57
by changing the level of abstraction at which a case is transferred, a robot can
use a single source demonstration to address target environments of varying
similarity to the source environment.
4 Future Work
We have implemented the Case Storage process and the last two steps of the
Case Adaptation process, the Transfer and Execution steps. Currently, we man-
ually provide the robot with the most relevant source case demonstration and
a mapping between objects in the source and target environments. Thus, our
next steps are to identify a method for autonomously determining this object
mapping. Furthermore, future work will involve defining a process for identifying
and retrieving an appropriate source case demonstration that is most applicable
to a given transfer problem.
Acknowledgments
This work was supported by NSF Graduate Research Fellowship DGE-1148903.
References
1. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning
from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)
2. Chernova, S., Thomaz, A.L.: Robot learning from human teachers. Synthesis Lec-
tures on Artificial Intelligence and Machine Learning 8(3), 1–121 (2014)
3. Fitzgerald, T., Goel, A.: A case-based approach to imitation learning in robotic
agents. Intl. Conf. on Case-Based Reasoning Workshop on Case-Based Agents
(2014)
4. Fitzgerald, T., Goel, A.K., Thomaz, A.L.: Representing skill demonstrations for
adaptation and transfer. AAAI Symposium on Knowledge, Skill, and Behavior
Transfer in Autonomous Robots (2014)
5. Fitzgerald, T., McGreggor, K., Akgun, B., Thomaz, A.L., Goel, A.K.: Visual case
retrieval for interpreting skill demonstrations. International Conference on Case-
Based Reasoning (2015)
6. Floyd, M.W., Esfandiari, B.: A case-based reasoning framework for developing
agents using learning by observation. In: 2011 23rd IEEE International Conference
on Tools with Artificial Intelligence (ICTAI). pp. 531–538. IEEE (2011)
7. Floyd, M.W., Esfandiari, B., Lam, K.: A case-based reasoning approach to imitat-
ing robocup players. In: FLAIRS Conference. pp. 251–256 (2008)
8. Ontañón, S., Mishra, K., Sugandh, N., Ram, A.: Case-based planning and execution
for real-time strategy games. In: Case-Based Reasoning Research and Development,
pp. 164–178. Springer (2007)
9. Pastor, P., Ho↵mann, H., Asfour, T., Schaal, S.: Learning and generalization of
motor skills by learning from demonstration. In: Robotics and Automation, 2009.
ICRA’09. IEEE International Conference on. pp. 763–768. IEEE (2009)
10. Piaget, J., Cook, M.T.: The origins of intelligence in children. (1952)
11. Tomasello, M., Kruger, A.C., Ratner, H.H.: Cultural learning. Behavioral and brain
sciences 16(03), 495–511 (1993)