This paper discusses potential limitations in learning in autonomous robotic systems that integrate several specialized subsystems working at di erent levels of abstraction. If the designers have anticipated what the system may have to learn, then adding new knowledge boils down to adding new entries in a database and/or tuning parameters of some subsystem(s). But if this new knowledge does not t in predened structures, the system can simply not acquire it, hence it cannot \think out of the box" designed by its creators. We show why learning out of the box may be di cult in integrated systems, hint at some existing potential approaches, and nally suggest that a better approach may come by looking at constructivist epistemology, with focus on Piaget's schemas theory.
The typical approach for designing intelligent robots is Divide and Conquer : A team of experts with di erent domains of competence is formed, each of which shall develop the individual components (perception, actuation, reasoning) required for the system to achieve global intelligent behavior. Then, these subsystems need to be integrated, which requires some sort of interface between subsystems and global coordination mechanisms.
As compared to disembodied AI, there are several reasons why robotic systems need to integrate several subsystems. For functional reasons (e.g., the robot needs vision, path planning, dialogue), for engineering reasons (reusing existing software modules), or for reasoning upon di erent types of knowledge (causal, spatial, temporal) for which speci c representations and reasoners have been developed. One may for instance represent causal relations by some action language and reason upon it with a satis ability solver, while spatial matters may be represented by transformation matrices and reasoned upon with graph search.
These specialized subsystems perform well in their own domains, and for most of them, learning \variants" have been devised: vision systems can learn new categories, motion planners can learn from previous queries or by imitation, task planners can learn heuristics, rule-based systems can learn by chunking, etc. This paper points out the issue of drawing meaningful relations between what is individually learned by the di erent subsystems of integrated systems and, furthermore, questions the capacity of current learning methods for robots to develop new representations and skills. 2
Let us consider the following example: a robot that should learn to manipulate
various objects. The robot is standing in front of a table and its task is to clear the
table, i.e, picking up any object from the table and releasing it in a nearby trash
bin. We assume a classical sense-plan-act architecture [
Then a novel object is introduced, e.g., a credit card, and the system should learn how to complete the task. The credit card cannot be grasped directly from the table, i.e., it has to be slid to the edge of the table before it can be grasped (see Fig. 1). We do not assume any particular learning methods, and suppose that after su cient training, the subsystems have learned as follows: { the deliberative subsystem has learned a new operator grasp 34 (the system does not know it is a slide-and-grasp) for objects of type type 23 (the system does not know it is a credit card); { the perception subsystem has been trained to recognize a new class of objects (type 23 ); { the actuation subsystem has added a new motion primitive to its database for grasp 34.
The system can now deal with credit cards or (similar objects), but it has not learned anything about why grasp 34 is appropriate for objects of type type 23. Therefore if it is presented with a novel at but di erent object, for instance a coin, it will not be able to reuse what it has learned about credit cards. Assuming that the perceptual subsystem has learned a feature related to \ atness", and given previous experiences with at and non- at objects, the system could infer through statistical methods a correlation between atness and a particular grasping strategy, hence using this knowledge to grasp unforeseen at objects. But since the subsystems work {by construction{ in di erent domains, this may simply not happen. What is learned by one subsystem is not necessarily relevant for other subsystems. Unless a human designer has anticipated which features may be of interest and built them in. 3
Learning is a general function that may take a variety of forms. For subsequent discussion, we introduce an informal distinction between two types of learning processes: horizontal and vertical learning.
We denote by horizontal learning the type of learning commonly found in arti cial systems (supervised/unsupervised learning, Reinforcement Learning). Horizontal learning takes place in prede ned structures, which have been set up for that end. For instance, rules in a logic program, weights in an ANN, spline parameters of a motion primitive. During the learning process, new knowledge is created by tuning existing knowledge or appending the existing one with new instances. The knowledge acquired through horizontal learning can be subsequently used by the system without modifying its core algorithm, since the data structures and/or semantics are the same as for previous knowledge.
Vertical learning is a more fundamental type of learning which involves
modi cation of the system itself as knew knowlege is acquired. This is what humans
(and probably other evolved species) do as they grow up. As they develop,
infants gradually acquire representations about causality, space, time, quantity,
and other concepts [
We believe that both types of learning are necessary as a basis for intelligent robots: horizontal learning for adapting to new objects/environments, and vertical learning for being able to solve problems that have not been anticipated by their designers. Next, we examine some approaches to address the vertical learning problem. To our knowledge, there exists no automated system capable of updating its core reasoning process through learning.
One way to circumvent the problem is to learn new representations. Learning
new representations allows to see the world from a new perspective, therefore
it is a key ability for solving unforeseen problems [
Another approach to tackle vertical learning is to come up with a form of
knowledge representation which can represent everything. If causal, perceptual
and motor knowledge could be represented seamlessly with the same language,
learning could take place in a single system, thereby avoiding the issue of
learning in integrated systems addressed in Section 2. Ontologies are good
candidates to this end. Some systems have been developed both for perception, e.g.,
SceneNet [
Deep end-to-end learning allows to learn perceptual features, deliberation
rules, and motor control parameters within a single process. But the training
process is data and computationally intensive, even for narrow tasks such as
object grasping [
Integrated systems have issues for relating knowledge learned across di erent subsystems, while monolithic systems have computational issues or heavily rely on designer's knowledge. In the next section, we question (and wish to foster discussions on that theme during the workshop) the possibility of drawing inspiration from a constructivist psycholgy for addressing our problem from a di erent perspective. 5
AI/Robotics essentially tries to reproduce cognitive and sensorimotor skills of humans adults. This approach has been successful for solving variety of problems, even outperforming humans in narrow domains. In the constructivist paradigm, the question of interest is not \How do humans grasp di erent objects?" but rather how a system who is initially barely aware of itself {the infant{ acquires knowledge and skills which allow him to grasp di erent objects.
The logician and psychologist Jean Piaget has long studied how knowledge
is constructed, particularly in infants. His main contribution his the discovery
of universal developmental stages in cognitive development, which may occur at
di erent times, but always in the same sequence, regardless of cultural or
social environment [
Piaget's ideas have been implemented and tested in micro-world simulations
or simple systems [
In order to investigate the application of Piaget's theory to robotics, we propose a model which could be used as a basic building block for a schemabased learning robotic controller (see Fig. 2). Unlike previous attempts, this schema model operates in the continuous time domain, i.e., inputs and outputs are multidimensional time-dependent signals. The schema continuously learns a forward model, which predicts sensory signals (S') as a function of motor control signals (M). The di erence between predictions and actual sensory input is used by the controller for adjusting control parameters in face of disturbances, which corresponds to the assimilation mechanism. When prediction (S') and actual sensory input (S) diverge beyond a certain threshold, a warning signal is issued to trigger the accomodation mechanism at a higher level.
In the initial stage, the system creates sensorimotor schemas through random exploration and motor babbling. These schemas are reinforced as they are re-enacted. When a su cient number of sensorimotor schemas has been reached, they produce patterns of activation which can be assimilated by higher level schemas. Higher-level schemas follow the same principles as sensorimotor schemas do, except that their input and output come from other schemas instead of sensors and actuators. Hence, assimilation and accomodation take place with the same mechanism, but at higher level of abstraction.
More details about the envisioned system will be presented at the workshop and, given the preliminary status of our proposal, rather than presenting results, we hope to foster discussions and to get inspiring ideas and suggestions from the community.