=Paper=
{{Paper
|id=Vol-1520/paper30
|storemode=property
|title=Toward a Case-Based Framework for Imitation Learning in Robotic Agents
|pdfUrl=https://ceur-ws.org/Vol-1520/paper30.pdf
|volume=Vol-1520
|dblpUrl=https://dblp.org/rec/conf/iccbr/Fitzgerald15
}}
==Toward a Case-Based Framework for Imitation Learning in Robotic Agents==
https://ceur-ws.org/Vol-1520/paper30.pdf
265
Toward a Case-Based Framework for Imitation
Learning in Robotic Agents
Tesca Fitzgerald
School of Interactive Computing,
Georgia Institute of Technology,
Atlanta, Georgia, 30332
tesca.fitzgerald@cc.gatech.edu
1 Introduction
Imitation learning is a skill essential to human development and cognition [6,
5]. Naturally, imitation learning has become a topic of focus for robotics re-
search as well, particularly in interactive robots [1, 2]. In imitating the actions
of a teacher, a cognitive agent learns the demonstrated action such that it may
perform a similar action later and achieve a similar goal. Thus, we expect that
a cognitive robot that learns from imitation would reuse what it has learned
from one experience to reason about addressing related, but di↵erent, problem
scenarios.
The eventual goal of this work is to use a case-based approach to enable
imitation learning in interactions such as the following. A human teacher guides
the robot to complete a task, such as scooping the contents of one container
into another. The robot records the demonstrated actions and observed objects,
saving the demonstration as a source case in its case memory. At a later time,
the robot is asked to repeat the scooping task, but in a new, target environment
containing a di↵erent set of object features to parameterize and execute the
task. Next, the robot would transfer its representation of the scooping task to
accommodate for the di↵erences between the source and target environments,
and then execute an action based on the transferred representation to achieve
the goal state in the target environment.
Using a case-based framework to address this problem allows us to repre-
sent demonstrations as individual experiences in the robot’s case memory, and
provides us with a framework for identifying, transferring, and executing a rele-
vant source case demonstration in an unfamiliar target environment. The main
research questions we plan to address are as follows:
– How should task demonstrations be represented in case memory?
– How do we determine which features of a robot’s environment are relevant
to completing a task, and thus should be stored in the source case?
– What features should be considered in retrieving a source case demonstration
for reuse in a target environment? How should these features be prioritized
during source case retrieval?
Copyright © 2015 for this paper by its authors. Copying permitted for private and
academic purposes. In Proceedings of the ICCBR 2015 Workshops. Frankfurt, Germany.
266
Toward a Case-Based Framework for Imitation Learning in Robotic Agents
Fig. 1. Case-Based Process for Task Demonstration Transfer
2 Research Plan and Progress
We have defined a case-based approach to transfer for enabling imitation in
robotic agents, consisting of two separate processes (as shown in Figure 1): the
Case Storage process in which the robot receives demonstrations of a task and
stores each demonstration as a case in source memory, and a Case Adaptation
process which is used at a later time when the robot is asked to repeat a task in
a target environment.
2.1 Case Storage Process
Demonstration and Learning We have implemented the first step in the
Case Storage process, where the robot records and stores each task demon-
stration as a source case in memory. We define each case as the tuple C =
, where:
– L represents the label of the task which was demonstrated, e.g. ”scooping”.
– D represents the set of action models which encode the demonstrated motion,
represented as Dynamic Movement Primitives as defined in [4].
– T is the set of parameterization functions which relate the set of action
models to the locations of objects in the robot’s environment. For example,
a parameterization function may be used to represent how the robot’s hand
must be located above a bowl prior to completing a pouring action.
– O is the set of salient object IDs which are relevant to the task.
– Si and Sf are the initial and final states, respectively, which represent the
set of objects observed in an overhead view of the robot’s environment.
2.2 Case Adaptation Process
At a later time, the robot may be asked to repeat a learned task in an unfamiliar
target environment. Using the framework shown in Figure 1, the robot may
address a target environment using the following steps.
Observation The robot is given a target problem to address, under the as-
sumption that it has a relevant source case in memory which can be used to
address the target problem. The robot observes the target environment by view-
ing the objects located in the table-top environment using an overhead camera,
providing it with the initial state Si of the target case.
267
Toward a Case-Based Framework for Imitation Learning in Robotic Agents
Retrieval and Mapping The robot must then choose a source case from
memory containing the demonstration that is most relevant to the current target
problem. Once a relevant source case has been retrieved, a mapping must be gen-
erated that encodes the di↵erences between the source and target environments.
This mapping is later used to transfer the source case such that di↵erences in the
target environment are addressed. We have not yet implemented the Retrieval
and Mapping steps, but will be addressing them in upcoming work.
Transfer and Execution We have implemented the last two steps of the Case
Adaptation process, the Transfer and Execution steps. Currently, we manually
provide the robot with the most relevant source case demonstration and a map-
ping between objects in the source and target environments.
We take a similarity-based approach to transfer, where we consider the sim-
ilarity between the source case and target environments when defining transfer
processes. As we encounter transfer problems in which the source and target
problems become less similar, the source case is transferred at a di↵erent level of
abstraction, such that only high-level features of that case are transferred. We
have implemented three transfer methods, each of which operates by transfer-
ring the source case at a di↵erent level of abstraction (further described in [3]).
Once the source case has been transferred, it is used to plan and execute a new
action trajectory to address the target problem. Preliminary experiments have
evaluated each method under the assumption that we select the approach, and
thus the level of abstraction at which transfer occurs, to be used for a given
transfer problem.
3 Future Work
Our current implementation assumes that we manually provide a mapping be-
tween equivalent objects in the source and target environments. We plan to iden-
tify (i) a method for autonomously determining this object mapping and (ii) a
process for identifying and retrieving an appropriate source case demonstration.
References
1. Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning
from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)
2. Chernova, S., Thomaz, A.L.: Robot learning from human teachers. Synthesis Lec-
tures on Artificial Intelligence and Machine Learning 8(3), 1–121 (2014)
3. Fitzgerald, T., Goel, A.K., Thomaz, A.L.: A similarity-based approach to skill trans-
fer. Women in Robotics Workshop at Robotics: Science and Systems (2015)
4. Pastor, P., Ho↵mann, H., Asfour, T., Schaal, S.: Learning and generalization of
motor skills by learning from demonstration. In: Robotics and Automation, 2009.
ICRA’09. IEEE International Conference on. pp. 763–768. IEEE (2009)
5. Piaget, J., Cook, M.T.: The origins of intelligence in children. (1952)
6. Tomasello, M., Kruger, A.C., Ratner, H.H.: Cultural learning. Behavioral and brain
sciences 16(03), 495–511 (1993)