Cognitive Robotics, as described by Levesque and Reiter [1], is “the study of the knowledge representation and reasoning problems faced by an autonomous robot (or agent) in a dynamic and incompletely known world.” In general, it is concerned with endowing agents with a wide variety of higher level cognitive functions that involve reasoning, for example, about goals, perception, actions, the mental states of other agents, collaborative task execution. To achieve these objectives, cognitive robotics requires (re)integrating Artificial Intelligence (AI) and thus hybrid methods. In this talk, I will give examples from four diferent cognitive robotics applications to illustrate the need for (re)integrating spatial and temporal reasoning with robotics methods: service robotics, digital forensics, manipulation planning, and robot construction problems. Missing child scenario Consider a shopping mall where several service robots help people with their inquiries and requests. Suppose that two parents are looking for their missing son in a shopping mall and request help from a robot in a food court. The robot has received sightings of the child at the south or west of the pool, and wants to find out the potential whereabouts of the child. Once the robot receives the sightings of the child, they first check whether this information makes sense or not. This can be done, e.g., by formulating the cardinal directional relations between spatial objects in the shopping mall and the sightings of the child, as constraints in Cardinal Directional Calculus (CDC) [2, 3], and by checking its inconsistency with a Qualitative Spatial Reasoning (QSR) system that supports CDC. After the consistency of the gathered information is verified, the robot tries to find out the possible locations of the child that make sense. For that, the robot also needs to consider relevant commonsense knowledge and assumptions, like the following: “Children are generally around the ice-cream truck. The ice-cream truck is by default in the free areas that are to the north, east or northeast of the movie theater.” To represent such default relations about cardinal directions and to integrate reasoning about defaults with qualitative spatial reasoning, we extend CDC by default CDC constraints (called nCDC) and build a QSR system (called nCDC-ASP [4, 5]) on top of it, by utilizing the foundational and practical methods of Answer Set Programming [6, 7, 8] that support nonmonotonic reasoning over defaults [9]. To be able to infer the relevant missing CDC relations and then to express the possible locations of the child to the parents in an understandable way (like the following: “Your child might be to the southeast of the food court and to the east of the park. That is, to the southeast of where you are now.”), we further extend nCDC by inferred CDC constraints and introduce a method in nCDC-ASP to infer missing CDC relations. This example illustrates the need to extend QSR [10] with nonmonotonic reasoning about defaults and inference of relevant missing relations, in order to involve commonsense knowledge and reasoning. Interrogation of suspects Suppose that a robotic investigator receives some camera images of the crime scene, and the interrogation of the suspects by the police, like the following: “Suspect 2: ... I noticed a suitcase in front of the table. There was a knife on the floor, to the right of the body, and in front of the phone.” Considering the digital evidence and the relevant commonsense knowledge (e.g., the phone is generally on the table), the robot wants to check whether Suspect 2 is truthful or not. To be able to perform reasoning about cardinal directions in 3D, we extend nCDC and nCDC-ASP to 3D (called 3D-nCDC and 3D-nCDC-ASP respectively) [11] in the spirit of [12, 13]. Using the reasoner 3D-nCDC-ASP, the robot can check whether the 3D-nCDC constraint network obtained from Suspect 2's statement, digital evidence and commonsense knowledge is consistent or not [14]. Suppose that 3D-nCDC-ASP finds Suspect 2's statement inconsistent with respect to the digital evidence. The investigator asks: Why? For that, we extend the capabilities of 3D-nCDC-ASP further so that it can generate explanations for inconsistencies [11], like the following: “The knife cannot be both below and to the front of the phone, as it is to the right of the body according to the digital evidence.” This example illustrates the need to extend QSR with explanation generation, for trustworthiness.
Manipulation planning Consider, for instance, a robot manipulator (as depicted in the figure below) that aims to move an object from the right hand side of the table to the left hand side of the table, without colliding with any obstacle. Once the pick and place actions are described in a planning language, a task plan can be found by a planner as follows: first pick the object from the right hand side of the table (this action is possible since the robot’s gripper is empty), and then place it on the left hand side of the table (this action is possible since the left hand side of the table is clear). However, this plan is not feasible: if the robot proceeds with executing this plan then it will collide with the obstacles above and the plan will fail.
We introduce a hybrid method [
We present soundness and completeness results for this hybrid planning method [
This example (and the applications mentioned above) illustrate the need to integrate temporal reasoning over states and actions (like task planning), with feasibility checks (like reachability checks via motion planning), for feasible and safe robotic plan execution.
Robotic construction Given an initial configuration of blocks of diferent sizes on a table, some goal conditions (e.g., build a bridge) and an upper bound on a makespan, the robot construction problem asks for (i) a final stable configuration of blocks stacked on each other that satisfy some specified goal conditions, and (ii) a feasible stack rearrangement plan to obtain that final configuration from a specified initial configuration of the blocks. The ifgure below illustrates an example [ 24, Scenario 13], where an initial state is given and a bridge (along with a plan) is computed. Note that these problems ask for not only a feasible plan but also a feasible goal state.
M3 M2 M1
7 units Initial State
M7
M6 M5 M4
C3 C4 C5
C1 C2 C6
C3 C1 M1
C2 M2
M7 M3
M6 7 units Final State
C5 M5
C6
C4
M4
Therefore, they require not only reachability checks but also stability checks, and not only for preconditions of
pick and place actions but also for states. Also, these problems ask for plans that make sense in the spirit of [
We introduce a hybrid method to solve robotic construction problems [
These examples illustrate the need to extend temporal reasoning over states and actions to allow complex but commonsensical actions, and to embed stability checks (via simulations by physics engines) in logic-based representations of actions and change, for feasible and safe robotic construction.