Human-Robot Teaming: Perspective on Analysis and Implementation Issues Antonio Chella1,2[0000−0002−8625−708X] , Francesco Lanza1[0000−0003−4382−6366] , Arianna Pipitone1[0000−0003−2388−5887] , and Valeria Seidita1,2[0000−0002−0601−6914] 1 Dipartimento di Ingegneria Universitá degli Studi di Palermo 2 C.N.R., Istituto di Calcolo e Reti ad Alte Prestazioni, Palermo, Italy name.surname@unipa.it Abstract. Interaction in a human-robot team in a changing environ- ment is a big challenge. Several essential aspects that deserve investiga- tion are at the base for efficient interactions. Among them the ability to produce a self-model and to apply elements from the theory of mind. This case is much more cumbersome than just implementing a system in which the various parts have to co-operate and collaborate to achieve a common goal. In the human-robot team, some factors that cannot be known before the execution phase intervene. Our goal is to investigate how a human-human team works and replicate it on the robot by defin- ing a new cognitive architecture which attempts to model all the involved issues. This means enabling the robot with the capability to understand the world around, itself and the other, human or robot as well. In this paper, we present the first step towards the creation of a multi-agent architecture to realize human-robot teaming interaction. 1 Introduction The long-term research goal of the described research focuses on the analysis and development of teams where humans and robots are mates, and they collaborate at the same level in a human-like fashion. A team member, first and foremost, knows the overall goal of the team. She knows what she wants to do. Hence, she intentionally decides which purpose to commit. She is also aware of what she can do, i.e., of her capabilities; accordingly, she selects the goals she can reach and the corresponding plan of actions. A team member is also aware of the surrounding environment: she can asso- ciate any new element in the situation with the content of her knowledge base. A team member owns a knowledge base that includes a large number of items; only a few of them are of interest for what she was doing. A team member can communicate with the other teammates to update her knowledge base on the environment. A teammate explicitly or implicitly delegates action to other mates; she asks how to do something she is not able to do. She observes what 2 A. Chella et al. other members are doing and, if the case, she anticipates actions or cooperates with other teammates. She observes what the other teammates are doing and decides what to do, and whether to do something by her emotional and stress state, on the trust level she has on the other teammates and herself. A team member understands if the operative condition for pursuing actual objectives changes. In this case, she can re-plan or create a new plan from experience. She can explain to the other team members what she is doing and why and, if the case, why she is not able to do something. Finally, she learns from experiences and stores all the information about the continuous changing world. Environment Environment Goals Actions Communication Human Robot Object/Resources ReasoningPerceptions/ Process Observation Goals Robot Actions Communication Human Object/Resources Reasoning Process Perceptions/Observation Fig. 1. Human-Robot teaming scenario in an environment composed of cognitive agents, objects and resources. During each one of the previous activities, each team member performs differ- ent processes concerning vision, speech understanding, learning, state of mind, decision making that account for a description of the world in real time. Taking into account these aspects in human-robot teaming means to analyze and implement the processes of: (i) knowledge acquisition and representation, including memory management; (ii) representation of the external environment; (iii) plans selection and creation; (iv) learning. Moreover, we take into account a simplified process of introspection to allow the robot mate to be aware of itself and its capabilities. Fig. 1 sketches this scenario. The environment the robot is plunged in is made of humans that interact/communicate with him and resources to use and to act on. The robot, so as human, can reason about perceptions and observations and about the situation he has in his mind of the world around and himself. Human-Robot Teaming: Perspective on Analysis and Implementation Issues 3 This paper focus on knowledge and we propose an agent-oriented architecture for robot knowledge acquisition and representation. 2 From the Standard Model of Mind to the BDI architecture In [9], authors propose a standard model of mind. The standard model is intended to be a reference point and a driving theoretical approach for developing and implementing cognitive architectures in several research areas. The core of the standard model is the cognitive cycle - perception, working memory, and action - that allows realizing complex behaviors. Taking inspiration from the standard model of mind we hypothesize a set of modules of a cognitive architecture for a robot teammate [6]: the knowledge and memory module, the perception module, the communication system and the rea- soner that allows the robot to choose by taking into account the retrieved data. The behavior of the robot is deliberated by the planner module which interacts with the context in which the robot is plunged. Thus, we got inspiration from the standard model to create an agent-oriented architecture for robot knowledge acquisition and representation. We employ the agent-oriented technology and the BDI paradigm for knowl- edge representation and acquisition with a robot able to generate a simplified model of introspection. Fig. 2 illustrates the agent architecture we propose; here we employ the multi-agent systems paradigm to implement each module as an agent which interacts with all the others for achieving its objectives and at the same time the overall system objective. Agents employ inputs from the environment perception and from memory for choosing which action to execute. The agent decides actions to perform af- ter a reasoning process, and it completes and continuously observes the results of its work on the environment. To integrate self-consciousness aspects in the architecture, we use specific agents for decision and memory modules. We represent knowledge by including the objects in the environment, the goals to be pursued and the motivations to execute a specific action. The knowl- edge representation allows us considering the situation as composed of objects, other cognitive agents and also the agent inner state (see Fig. 1). All these elements are parts of the agent’s self-consciousness that triggers the agent deci- sion process. Continuous observation and perception allow the agent to update knowledge during the execution phase [5]. We model and update the agent knowledge base at runtime. Knowledge is necessary for the decision process and for communicating and interacting with other agents. Besides, knowledge representation lets the agent be able to under- stand what it does not know. We employ multi-agent theoretical and technological aspects to implement the reasoning cycle at the core of the decision process. We also employ the Belief-Desire-Intention model (BDI) [11] to describe the reasoning process of each agent. We employ Jason [2] as a programming language that implements 4 A. Chella et al. BDI agents. The decision-making model underpinning BDI systems is known as practical reasoning, a reasoning process to do actions, where agents’ desires and agents’ beliefs supply the relevant factor [3]. Practical reasoning consists of the activities of deliberation and intentions and means-ends reasoning. Memory OWL 01000100 01011011 Management Ontology Framework Workspace Knowledge Knowledge Workspace Manager Perception E n Execution v Working Memory i Vision r Current Situation Workspace o n m e n t Motion Perception Planner Decision Process Workspace Sensor Robot Workspace Global Updating Internal Status Fig. 2. BDI multi-agent architecture for human-robot teaming interaction. In our approach (Fig. 2), each agent is orchestrated, regarding knowledge and memory access, by the controller agent Knowledge Manager, implementing the planning and reasoning functions. The module ensures the knowledge necessary to allow collaboration among the agents. Across the extended reasoning cycle, each agent employs its experience to perform the action and to reason on the situation by analyzing inner states and external perceptions. Once the plan selects a response, it is executed by changing the state of the environment and also the internal state of the robot. With the aim to endow the robot with self-consciousness abilities, we model tasks through sets of beliefs and intentions. The robot can identify failures in executable plans and actions and to explain and justify the incompleteness of its performances. Perception and external stimuli are modeled in the knowledge ontology of the robot. When a goal is detected, the related beliefs are generated from the ontology by allowing the robot to select the appropriate plan. Each action modifies the state of the environment and the robot inner state. Our approach involves knowledge allowing the robot to identify the motiva- tions for which a plan could fail [7][4]. We keep a separation between the rea- Human-Robot Teaming: Perspective on Analysis and Implementation Issues 5 soning component and the environmental managing tools; these two components are implemented respectively by Jason [2] and CArtAgO [12]. Jason implements the BDI agents, and it manages the interactions among them, whereas CArtAgO manages the interaction with all the resources and the objects in the environ- ment. Beyond simple actions, each plan involves context variables representing the preconditions to be satisfied to perform the steps of the plan. When one of these variables, instantiated by the perception module, does not meet the pre- requisites (i.e., it has an unexpected value, or it is false), then the plan execution fails, and the robot can infer the motivations of the failure, thus implementing a simple form of self-consciousness. The motives of failures are then sent to the other members of the team which may solve the situation by enforcing collabo- ration. 3 Conclusions In this paper we propose a cognitive architecture for human-robot teaming in- teractions in changing and partially known environment. An efficient interaction may be performed on the basis of several factors among which we stress the role of self-modeling and theory of mind that we claim to be a way for resembling human-human interactions. We also propose to use of the multi-agent system paradigm for filling the gap between design and system level. The power of BDI paradigm let us to implement an high modular framework that may be scaled by simply adding agents to the whole system. Indeed, the strength of our approach is the extension of the BDI reasoning cycle, we did not perform any change but simply added portion of Jason code for implementing the modules of the architecture. The part of the architecture we detail in this paper is related to the knowledge management and is inspired by a well established standard model of mind such as to several cognitive architectures of the literature [1][8][10]. Acknowledgment. This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0232. References 1. Anderson, J.R., Matessa, M., Lebiere, C.: ACT-R: A theory of higher level cog- nition and its relation to visual attention. Human-Computer Interaction 12(4), 439–462 (1997) 2. Bordini, R.H., Hübner, J.F., Wooldridge, M.: Programming multi-agent systems in AgentSpeak using Jason, vol. 8. John Wiley & Sons (2007) 3. Bratman, M.E.: What is intention. Intentions in communication pp. 15–32 (1990) 4. Chella, A., Lanza, F., Seidita, V.: Representing and developing knowledge using Jason, CArtAgO and OWL. vol. 2215, pp. 147–152 (2018) 5. Chella, A., Lanza, F., Pipitone, A., Seidita, V.: Knowledge acquisition through in- trospection in human-robot cooperation. Biologically Inspired Cognitive Architec- tures 25, 1 – 7 (2018). https://doi.org/https://doi.org/10.1016/j.bica.2018.07.016, http://www.sciencedirect.com/science/article/pii/S2212683X18300847 6 A. Chella et al. 6. Chella, A., Lanza, F., Seidita, V.: A cognitive architecture for human-robot team- ing interaction. In: Proceedings of the 6th International Workshop on Artificial Intelligence and Cognition. Palermo (July 2-4 2018) 7. Chella, A., Lanza, F., Seidita, V.: Human-agent interaction, the system level using Jason. In: Proceedings of the 6th International Workshop on Engineering Multi- Agent Systems (EMAS 2018). Stockholm (July, 10-15 2018) 8. Franklin, S., Madl, T., D’Mello, S., Snaider, J.: LIDA: A systems-level architecture for cognition, emotion, and learning. IEEE Transactions on Autonomous Mental Development 6(1), 19–41 (2014) 9. Laird, J.E., Lebiere, C., Rosenbloom, P.S.: A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics. AI Magazine 38(4) (2017) 10. Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: An architecture for general intel- ligence. Artificial intelligence 33(1), 1–64 (1987) 11. Rao, A.S., Georgeff, M.P., et al.: BDI agents: From theory to practice. 12. Ricci, A., Viroli, M., Omicini, A.: Cartago: A framework for prototyping artifact- based environments in mas. E4MAS 6, 67–86 (2006)