Service robots that are employed in human-centered environments with a high degree of complexity, unpredictability, and dynamicity must be able to learn new tasks autonomously, i.e. when only the goal of the task is given. Furthermore, robots must be able to understand natural language instructions to accurately identify the requested tasks, which requires connections between symbols, i.e. words, and their meanings, i.e. percepts. There exist many studies in the literature that investigate action learning or grounding, but few consider both simultaneously. Additionally, action learning studies have been limited to learn a single action while only varying the initial position of the gripper [3,4]. Furthermore, grounding studies were mostly conducted o ine and primarily focused on grounding of object characteristics or spatial concepts [2,1], while conducted action grounding employed simple feature vectors, which cannot be directly translated into motor commands [5]. In this paper, we investigate the possibility of simultaneous action learning and grounding through the combination of reinforcement and cross-situational learning. More specifically, we simulate human-robot interactions during which a human tutor provides instructions and illustrations of the goal states of the corresponding actions. The robot then learns to reach the desired goals taking into account di erent manipulation behaviors for di erent object shapes and grounds the words and detected phrases of the instructions, including synonyms, through obtained percepts.
The employed grounding and action learning system consists of three parts: (
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
After about 60,000 situations the reinforcement learner required 1 episode to converge to the optimal policy, when using a continuously decreasing exploration rate that is shared across situations. In contrast, when the exploration rate was reset for each situation, the reinforcement learner required after about 9,000 situations on average 28 episodes. That the agent did not execute the optimal policy immediately in the latter case, is due to the high exploration rate at the beginning of each situation because it was reset. Thus, a continuously decreasing exploration rate that is shared across situations works best for the investigated scenario. The employed CSL algorithm is able to successfully ground all 39 words used in this study through their corresponding percepts after about 800 situations. Afterwards the number of correct mappings is constantly 39, while the number of false mappings oscillates between 0 and 2 because the algorithm allows a word to be grounded through several percepts to be able to learn homonyms. The two additional incorrect mappings are for di erent word combinations, depending on the most recently encountered situations. 4
The proposed framework allowed learning of actions through reinforcement learning as well as identification of auxiliary words and phrases, and grounding of words and phrases, including synonyms, through cross-situational learning during simulated humanrobot interactions. In future work, the framework will be extended to handle real shape, color and preposition percepts obtained with a stereo camera.