Mixed-Initiative in Interactive Robotic Learning Julia Peltason, Ingo Lütkebohle, Britta Wrede, and Marc Hanheide Applied Informatics, Bielefeld University, Germany {jpeltaso,iluetkeb,bwrede,mhanheid}@techfak.uni-bielefeld.de Abstract. In learning tasks, interaction is mostly about the exchange of knowledge. The interaction process shall be governed on the one hand by the knowledge the tutor wants to convey and on the other by the lacks of knowledge of the learner. In human-robot interaction (HRI), it is usually the human demonstrating or explicitly verbalizing her knowl- edge and the robot acquiring a respective representation. The ultimate goal in interactive robot learning is thus to enable inexperienced, un- trained users to tutor robots in a most natural and intuitive manner. This goal is often impeded by a lack of knowledge of the human about the internal processing and expectations of the robot and by the inflex- ibility of the robot to understand open-ended, unconstrained tutoring or demonstration. Hence, we propose mixed-initiative strategies to al- low both to mutually contribute to the interactive learning process as a bi-directional negotiation about knowledge. Along this line this paper discusses two initially different case studies on object manipulation and learning of spatial environments. We present different styles of mixed- initiative in these scenarios and discuss the merits in each case. 1 Introduction Mixed-initiative human-robot communication is long being studied in rescue and space mission tasks with the objective of finding the optimal balance between teleoperation and full autonomy (see [1] for a survey). Under conditions where the robot needs assistance, the human could take temporary control of the robot putting it into a teleoperation mode until it is able to act autonomously again. It has been shown that such mixed-initiative control solutions can decrease the task completion time and the varying performance among users [2]. Mixed-initiative is typically employed to resolve abnormal situations, and communication is mostly realized via input devices such as keyboard or joystick, thus targeting at expert users. In contrast, the presented work focuses on lay persons interacting with robots using natural human modalities. Moreover, we regard mixed-initiative as an integral interaction strategy within the robot’s regular operation process. We present two case studies demonstrating that also within natural human robot interaction, mixed-initiative has the potential to enhance both task performance and usability. Both case studies illustrated in Fig. 1 follow the “learning by interacting” paradigm [3] arguing that interaction needs to play a major role in robotic learning. The first scenario is discussed under the aspect of enhancing the learning process by mixed-initiative interaction. For the second scenario, we 2 Mixed Initiative in Human-Robot Communication: Two Case Studies focus on how to provide appropriate guidance for the user and thus facilitating interaction by applying a mixed-initiative strategy. 2 The Home Tour: Mixed-initiative facilitates learning Our first scenario [4] is embedded in the so-called Home Tour where a mobile robot assistant has to become acquainted with human’s living environment by interacting with a human during a guided tour. Here, a basic requirement for the robot is being able to learn a spatial model of the environment and in- tegrate human and robotic representation as introduced in previous work [5]. Originally our system for this scenario employed a pure human initiative model, where the human user had to explicitly teach the robot. As a consequence of the user’s initiative modeling, the robot could only acquire a sparse model of the environment as only those pieces of information have been included which haven explicitly been presented by the user. To overcome this limitation, the system was enhanced to comprise a more mixed-initiative interaction strategy. This strategy enables active learning where the learner actively asks to reduce its lack of knowledge. In the home tour the robot continuously observes the spatial situation and asks if it changes significantly according to a novelty measure [4]. This obviously has the potential to speed up and optimize the learning process, because the robot can ask for missing knowledge in a goal-directed manner. It furthermore assures a most comprehensive and complete representation of space. Hence, the robot can verify existing information, request new information and resolve uncertainty as illustrated in table 1. Thus, this form of robotic learning supports not only extension but also continuous revision of acquired knowledge. This is especially important as the scenario targets at inexperienced users with- out knowledge about the internal mode of operation, possibly representing sparse or erroneous models. Table 1: Dialog excerpts for the Home Tour. Human initiative Robot initiative Room teaching Monitoring question Verifying hypothesis Resolving uncertainty H: This is the kitchen H: Where are we? R: We just entered R: Are we still R: Kitchen. R: We are in the kitchen the hallway, right? in the living room? I will have a look at it . H: No! H: No! R: (looking). R: What room is it? R: What room is it? R: OK! H: The kitchen. H: The kitchen. R: Kitchen. OK. R: Kitchen. OK. Fig. 1: The Mixed-initiative Home Tour and the Curious Robot scenario. Shaping the Interaction with Lay Users 3 Interaction is modeled based on the principle of grounding [6] assuming that both interaction partners aim to establish mutual understanding or common ground during their interaction. As to the Home Tour, human and robot build up a common understanding of the environment during a mixed-initiative ne- gotiation for which grounding provides the basis. Combining the two concepts enables both interaction partners to engage in interaction in order to make the robot’s spatial representation as correct, complete and consistent as possible. This idea has successfully been demonstrated in a test-run ([5], [4]) where de- spite limited classification quality a consistent environment representation could be achieved due to the robot’s mixed-initiative capabilities. 3 The Curious Robot: Mixed-initiative facilitates interaction Our second scenario is an interactive object learning and manipulation scenario with a humanoid robot exploring objects that are interesting or salient for it, assisted by the human tutor [7]. However, from experiences with the original Home Tour, we observed that untrained users require a significant amount of prior training to complete the task because the robot’s interaction model is not immediately obvious. Therefore, their behaviors and interaction strategies are varying enormously which is almost impossible for a system to cope with. Within this scenario, we consequently focus on how mixed-initiative allows to structure interaction for the users and thus make their behavior more pre- dictable. Based on bottom-up visual salience [8] that we here consider as percep- tual context the robot takes initiative and asks the user about objects and how to grasp them, as shown in table 2. By asking about objects on its own instead of leaving it to the user to demonstrate them the robot provides guidance within interaction which in particular unexperienced users could benefit from. Besides, it communicates what is interesting for it which unexperienced users might also not be aware of. Last but not least, with using robot initiative to determine the objects to learn we avoid error-prone visual analysis of human demonstration behavior which makes interaction even more robust. In a first evaluation con- ducted as video study, we observed that in situations where the robot provided guidance, participants answered quicker and required less clarification from ex- perimenter compared to situations with less guidance [7]. Further, in guided situations, user behavior was more consistent and therefore more predictable. Table 2: Dialog excerpts for the Curious Robot scenario. Robot initiative Human initiative Acquire label Acquire grip Grasp Command grasping Interrupt system R: What is this? R: How can I R: I am going to . H: Take the apple! R: (grasping) R: (pointing) grasp the banana? grasp the banana. R: OK! I start H: No, stop! H: A banana. H: With the two R: OK! I start. grasping now. R: OK, I’ll stop. finger grip. grasping now. R: (grasping) R: (stops) R: (grasping) R: OK! R: OK! R: OK! 4 Mixed Initiative in Human-Robot Communication: Two Case Studies 4 Discussion & Outlook We briefly presented two case studies with initially different style of initiative taking. Most of the interaction in the Home Tour was designed to be initiated by the human interaction partner. But an extension towards more mixed-initiative as presented in Sec. 2 already proved to be beneficial for the acquisition of consistent representations. The latter scenario in contrast was following a robot initiative paradigm from the very beginning. The robot strives to complete its knowledge and itself decides what to learn. It controls the tasks and assures a very well-shape interaction. For subjects in the studies the structure was easily accessible. However, they had only limited abilities to decide and control what’s going on. It is a consequent advancement to bring both mixed-initiative styles closer together. For the Curious Robot scenario, this would mean to allow for more human intervention, for instance to correct or modify grasps. The Home Tour in contrast would benefit from more robot curiosity, for instance by adding a bottom-up attention for interesting areas triggering robot initiative, not only exploiting the spatial context as a data-driven cue. It shall be emphasized that despite the different initiative styles, both scenar- ios use the same dialog framework and rely on the same principles considering mixed-initiative and grounding as guide lines for system architecture [4]. For both scenarios, the same negotiation protocol between components is used, reflecting the grounding process between human and robot, making the presented infor- mation consistent and thus establishing common ground. Moreover, the same mechanisms for initiative taking are used. Components provide context informa- tion, for instance about the spatial or perceptual context, triggering interaction and the learning process. References 1. J. Wang and M. Lewis, Mixed-initiative multirobot control in USAR. I-Tech Education and Publishing, September 2007. 2. C. W. Nielsen, D. A. Few, and D. S. Athey, “Using mixed-initiative human-robot interaction to bound performance in a search task,” in International Conference on Intelligent Sensors, Sensor Networks and Information Processing. IEEE, 2008, pp. 195–200. 3. M. Hanheide and G. Sagerer, “Active memory-based interaction strategies for learning-enabling behaviors,” International Symposium on Robot and Human Interactive Communication (RO- MAN), 01/08/2008 2008. 4. J. Peltason, F. H. K. Siepmann, T. P. Spexard, B. Wrede, M. Hanheide, and E. A. Topp, “Mixed- initiative in human augmented mapping,” in International Conference on Robotics and Automa- tion. Kobe, Japan: IEEE, 2009. 5. E. Topp, “Human-robot interaction and mapping with a service robot: Human augmented map- ping,” Doctoral Thesis, KTH School of Computer Science and Communication (CSC), Stockholm, Sweden, Sept. 2008. 6. H. H. Clark and S. A. Brennan, “Grounding in communication,” in Perspectives on socially shared cognition, L. B. Resnick, J. M. Levine, and S. D. Teasley, Eds., 1991. 7. I. Lütkebohle, J. Peltason, L. Schillingmann, C. Elbrechter, B. Wrede, S. Wachsmuth, and R. Haschke, “The curious robot - structuring interactive robot learning,” in International Con- ference on Robotics and Automation. Kobe, Japan: IEEE, 2009. 8. Y. Nagai, K. Hosada, A. Morita, and M. Asada, “A constructive model for the development of joint attention,” Connection Science, vol. 15, no. 4, pp. 211–229, December 2003. This work was partially funded as part of the research project DESIRE by the German Federal Ministry of Education and Research (BMBF) under grant no. 01IME01N and the Cluster of Excellence “Cognitive Interaction Technology”.