Toward a Case-Based Framework for Imitation Learning in Robotic Agents

Introduction

Imitation learning is a skill essential to human development and cognition [ 6, 5 ]. Naturally, imitation learning has become a topic of focus for robotics research as well, particularly in interactive robots [ 1, 2 ]. In imitating the actions of a teacher, a cognitive agent learns the demonstrated action such that it may perform a similar action later and achieve a similar goal. Thus, we expect that a cognitive robot that learns from imitation would reuse what it has learned from one experience to reason about addressing related, but di↵erent, problem scenarios.

The eventual goal of this work is to use a case-based approach to enable imitation learning in interactions such as the following. A human teacher guides the robot to complete a task, such as scooping the contents of one container into another. The robot records the demonstrated actions and observed objects, saving the demonstration as a source case in its case memory. At a later time, the robot is asked to repeat the scooping task, but in a new, target environment containing a di↵erent set of object features to parameterize and execute the task. Next, the robot would transfer its representation of the scooping task to accommodate for the di↵erences between the source and target environments, and then execute an action based on the transferred representation to achieve the goal state in the target environment.

Using a case-based framework to address this problem allows us to represent demonstrations as individual experiences in the robot’s case memory, and provides us with a framework for identifying, transferring, and executing a relevant source case demonstration in an unfamiliar target environment. The main research questions we plan to address are as follows: – How should task demonstrations be represented in case memory? – How do we determine which features of a robot’s environment are relevant to completing a task, and thus should be stored in the source case? – What features should be considered in retrieving a source case demonstration for reuse in a target environment? How should these features be prioritized during source case retrieval? Copyright © 2015 for this paper by its authors. Copying permitted for private and academic purposes. In Proceedings of the ICCBR 2015 Workshops. Frankfurt, Germany. We have defined a case-based approach to transfer for enabling imitation in robotic agents, consisting of two separate processes (as shown in Figure 1): the Case Storage process in which the robot receives demonstrations of a task and stores each demonstration as a case in source memory, and a Case Adaptation process which is used at a later time when the robot is asked to repeat a task in a target environment. Demonstration and Learning We have implemented the first step in the Case Storage process, where the robot records and stores each task demonstration as a source case in memory. We define each case as the tuple C = <L, D, T, O, Si, Sf >, where: – L represents the label of the task which was demonstrated, e.g. ”scooping”. – D represents the set of action models which encode the demonstrated motion, represented as Dynamic Movement Primitives as defined in [ 4 ]. – T is the set of parameterization functions which relate the set of action models to the locations of objects in the robot’s environment. For example, a parameterization function may be used to represent how the robot’s hand must be located above a bowl prior to completing a pouring action. – O is the set of salient object IDs which are relevant to the task. – Si and Sf are the initial and final states, respectively, which represent the set of objects observed in an overhead view of the robot’s environment. 2.2

Case Adaptation Process At a later time, the robot may be asked to repeat a learned task in an unfamiliar target environment. Using the framework shown in Figure 1, the robot may address a target environment using the following steps.

Observation The robot is given a target problem to address, under the assumption that it has a relevant source case in memory which can be used to address the target problem. The robot observes the target environment by viewing the objects located in the table-top environment using an overhead camera, providing it with the initial state Si of the target case. Retrieval and Mapping The robot must then choose a source case from memory containing the demonstration that is most relevant to the current target problem. Once a relevant source case has been retrieved, a mapping must be generated that encodes the di↵erences between the source and target environments. This mapping is later used to transfer the source case such that di↵erences in the target environment are addressed. We have not yet implemented the Retrieval and Mapping steps, but will be addressing them in upcoming work. Transfer and Execution We have implemented the last two steps of the Case Adaptation process, the Transfer and Execution steps. Currently, we manually provide the robot with the most relevant source case demonstration and a mapping between objects in the source and target environments.

We take a similarity-based approach to transfer, where we consider the similarity between the source case and target environments when defining transfer processes. As we encounter transfer problems in which the source and target problems become less similar, the source case is transferred at a di↵erent level of abstraction, such that only high-level features of that case are transferred. We have implemented three transfer methods, each of which operates by transferring the source case at a di↵erent level of abstraction (further described in [ 3 ]). Once the source case has been transferred, it is used to plan and execute a new action trajectory to address the target problem. Preliminary experiments have evaluated each method under the assumption that we select the approach, and thus the level of abstraction at which transfer occurs, to be used for a given transfer problem. 3

Future Work

Our current implementation assumes that we manually provide a mapping between equivalent objects in the source and target environments. We plan to identify (i) a method for autonomously determining this object mapping and (ii) a process for identifying and retrieving an appropriate source case demonstration.

References

1. Argall , B.D. , Chernova , S. , Veloso , M. , Browning , B. : A survey of robot learning from demonstration . Robotics and Autonomous Systems 57 ( 5 ), 469 - 483 ( 2009 )

2. Chernova , S. , Thomaz , A.L. : Robot learning from human teachers . Synthesis Lectures on Artificial Intelligence and Machine Learning 8 ( 3 ), 1 - 121 ( 2014 )

3. Fitzgerald , T. , Goel , A.K. , Thomaz , A.L.: A similarity-based approach to skill transfer . Women in Robotics Workshop at Robotics: Science and Systems ( 2015 )

4. Pastor , P. , Ho↵mann, H., Asfour , T. , Schaal , S. : Learning and generalization of motor skills by learning from demonstration . In: Robotics and Automation , 2009 . ICRA'09. IEEE International Conference on. pp. 763 - 768 . IEEE ( 2009 )

5. Piaget , J. , Cook , M.T.: The origins of intelligence in children . ( 1952 )

6. Tomasello , M. , Kruger , A.C. , Ratner , H.H. : Cultural learning . Behavioral and brain sciences 16 ( 03 ), 495 - 511 ( 1993 )