The inner speech is the common mental experience the humans have when they dialogue with themselves. It is widely acknowledged that inner speech is related to awareness and self-awareness. The inner speech reproduces and expands in the mind social and physical sources of awareness. In this preliminary work, a calculus based on a rst-order modal logic to automate inner speech is presented. It attempts to make the existing inner speech theories suitable for robot. By making robot able to talk to itself, it is possible to analyze the role of inner speech in robot awareness and self-awareness, opening new interesting research scenarios not yet investigated.
The dialogue with the self plays a fundamental role in human's consciousness [
Morin [
When analyzing the robot awareness and self-awareness, it seems desirable
to provide the robot with a kind of self dialogue. Few works investigate this
scenario. The authors proposed a cognitive architecture [
We attempt to automate the Morin's causal directions by de ning a calculus
couched in rst-order modal logic. The Deontic Cognitive Event Calculus
(DCE C) [
malize thorny situations, such as akrasia [
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution
We show that our formalization may a ect the execution by robot of a simple task when a self-perception stimuli occurs and triggers inner speech. The idea we propose lays the groundwork for new investigations related to the robot's inner speech ability, and for testing its e ects on robot awareness and self-awareness.
at section 2. Then, how we think to encode inner speech and self-awareness by EC is explained at section 3. The syntax of our calculus (with sorts, functions and axioms de nitions) is detailed at section 4. Inference schemata and formal conditions, that allow to make deduction and to reason on the syntax, are discussed at section 5. The section 6 shows a simple simulation for reasoning by the proposed calculus. Future works and implications are discussed at 7. 2
The Deontic Cognitive Event Calculus (DCE C)
The common Event Calculus (EC) [
DCE C [
The DCE C's intensional processes was successfully used to automate the
falsebelief task [
Involving same kinds of states, the calculus for inner speech and self-awareness is an intensional model, and we inspired to DCE C for our work. 3
As shown, Morin's theory [
Since a uent represents a state (or fact) of the world, we can de ne inner and external uents. An inner uent represents an internal agent state, eg: an emotion, its intensity value, the status of an its physical component (on, o , functioning,...). An external uent is related to the external context, and represents a fact of the environment, that is the typical de nition of EC's uent. Thus, how a typical uent is initiated or terminated by an action which changes the environmental set (named reactive action), similarly an internal uent is initiated or terminated by an internal action (named inner action, i.e. a self-regulation action, the perception of an entity, the rehearse of inner voice,...) which changes the internal set.
We think that awareness and self-awareness of an arti cial agent could be computationally modeled as all the uents (both inner and external) the agent knows to be active at a time. A uent is active at time t when it holds at t. The active uents that are not in the knowledge of the agent are not in the set modeling awareness and self-awareness: the agent does not know them, and it is not aware of them.
Now, let suppose that the object x at the instant t1 is at the location l1. The native EC function
holds(location(x; l1); t1) models the state of the world related to that fact, that is the active state of the uent location(x; l1) at the instant t1.
The action a = move x to l2 at the instant t2 with t1 < t2, causes the end of the previous uent
terminates(a; location(x; l1); t2) and the beginning of the new
uent location(a; l2) by initiates(a; location(a; l2); t2) and for the inferential logic of EC, the creation of a new function holds related to such a new uent.
When such an action has to be performed by an autonomous agent, it may
involve inner uents and statements beyond the previously seen ones. The agent
will use inner speech as a cognitive tool [
Kendall et. al [
All these evaluations will modify the set of active uents, and hence the state of awareness and self-awareness of the agent.
The calculus we propose attempt to model such evaluations. It could be extensible by adding uents representing further internal or external facts. 4
Commonly used functions, sorts and relation symbols from EC and some ones
from DCEC are included in our formalization. The resulted calculus has a unique
syntax and a proof calculus for reasoning and theorem proving. In particular,
the natural deduction by Gentzen [
Modal Fragment The modal fragment speci es the modal operators of the calculus.
While DCEC includes the standard modal operators for common-sense knowledge C and domain knowledge Kd, in our formalization we include other two types of knowledge by adding two new modal operators, representing: { the self-knowledge Kself , that is the knowledge the robot owns about itself, (i.e. inner world ). Then, Kself (a; t; ) means that the agent a knows the proposition representing an inner fact at time t; { the contextual-knowledge Kcx, that is the knowledge about the physical environment in which the robot is plunged, (i.e. external world ). The contextualknowledge we considered is not equal to the general one K, because it may include some new objects the robot does not know (it has never seen them). Moreover, the contextual knowledge may not include some concepts of the general one because these concepts could not be in the environment at a time. Kcx(a; t; ) means that agent a knows the proposition representing an external fact at time t.
The general knowledge we refer to becomes
K = Kself _ Kd _ Kcx The standard intensional operators for belief B, desire D, intention I are included too, and have the same semantics from DCEC.
Other DCEC modals operators we include are P for perceiving a state, and S for agent-to-agent communication or public announcement. But the S operator we de ne crucially di ers from those of DCEC because we consider the possibility to have the same agent (the robot) in its agent-to-agent arguments, leading to the inner speech formalization. The single argument means public announcement as for DCEC. Formally: { S(a; b; t; ) means that the agent a sends the message related to proposition to the agent b at time t; { S(a; a; t; ) means that the agent a rehearses the message related to proposition it sends to itself at time t; { S(a; t; ) means that the agent a makes a public announcement related to proposition at time t.
Finally, we add the modal operator M for producing a message related to a speci c proposition . Then, M(a; t; ) means that the agent a produces the message related to the proposition at time t. We have to specify that M regards the action \to produce a message" which can be in turn sent to another agent or rehearsed according to S. 4.2
Sorts Speci cation
the typical Agent and Entity sorts de nitions because we consider the agent as a part of the universe, hence the Agent sort becomes a sub-type of the Entity sort. The abstract sorts are instantiated at particular times by an actors.
The following table shows all the sorts we de ned. The highlighted sorts are the new ones introduced or modi ed by our calculus.
Entity Object Agent Percept Moment Event
An entity in the universe, including agent.
A subtype of Entity, representing a physical object in the environment that is not an actor.
A subtype of Entity, representing human and arti cial actors.
A perception from the environment; it can be a concrete Entity (an Agent or an Object), or a generic concept meaning an event.
A time in the domain.
An event in the domain.
ActionType
An abstract reactive action, i.e. an action that a ects the state of the external environment.
An abstract inner action, i.e. an action that a ects SelfActionType the inner state of the agent.
Examples: keep calm, evaluate, feel.
Action SelfAction
A subtype of Event that occurs when an agent performs a concrete ActionType action.
A subtype of Event that occurs when an agent performs a SelfActionType action.
Fluent Aware Message
A state of the universe, that can be inner or external.
A subtype of Fluent, representing a uent the agent knows.
The set of active Aware uents forms the agent awareness and/or self-awareness.
A message of an agent-to-agent, agent-to-itself, agent-to-public communication. 4.3
De ning the message m EC and DCEC have not any operators or functions to de ne the message of a communication. We take a modal approach for modeling such a situation without de ning speci c axioms. We assume that a message is related to a speci c fact, represented by a proposition. Given a proposition , we de ne by the operator the set of all constants and names of not native function expressions in . For example: (happens(action(Bob; loves(M ary))); t) = fBob; loves; M aryg (holds(located(apple; table)); t) = flocated; apple; tableg A message is the set returned by the operator, that is: : ! m So, given the corresponding message m will be m = ( ).
the i-th element in m. 4.4
The Syntax The whole syntax of the calculus is formalized by the formulas in the following table, where S represents the sorts, f represents the functions, the term represents the possible variables and nally are the propositions.
S :: =
Entity jAgent v Entity j Object v Entity j Percept v Entity j Percept = Agent t Object t Action t SelfAction j Fluent j ActionType j SelfActionType j Event j SelfAction v Event j Action v Event j Boolean j Moment j Message f :: = >8action : Agent ActionType ! Action > >>initially : Fluent ! Boolean > > >>>>holds : Fluent Moment ! Boolean > >>>>happens : Event Moment ! Boolean > >>>>clipped : Moment Fluent Moment ! Boolean >>>>>initiates : Event Fluent Moment ! Boolean >><terminates : Event Fluent Moment ! Boolean >selfaction : Agent SelfActionType ! SelfAction > > >>focuses : Percept ! SelfActionType > > >>>>aware : Agent Percept ! Fluent > >>>contenti : Message ! Percept > > >>>comprehends : Message ! SelfActionType > > >>produces : Message ! SelfActionType > > :>innerspeaks : Message ! SelfActionType term :: = var : S j const : S j f (term1; term2; :::termn) :: = >8>term:Boolean j : j ^ j _ j P(a; t; ) j K(a; t; ) j >>>C(t; ) j B(a; t; ) j D(a; t; ) j I(a; t; ) > > > < >S(a; b; t; ) j S(a; a; t; ) j S(a; t; ) > > > > > > >:M(a; t; )
The functions in bold are purposely de ned for formalizing inner speech, awareness and self-awareness, while the others are native from EC.
These new functions are : { selfaction(a, at)! as: that returns the concrete self action as of type at the agent a performs; it is the self-version of native action function of EC; { focuses(p)! at: that represents the selfaction type the agent a performs when it focuses on the percept p. Intuitively, self action(a; f ocuses(p)) means the action by a of focusing on the percept p; { aware(p)! f : that states the awareness about the percept p. The activation of that uent by holds(aware(p); t) means that an agent is aware of the percepts p at t. Let's notice that the aware uents may be a sub-set of the all active uents at a time. They are not the whole robot awareness and self-awareness, but are the robot awareness about the percepts.
{ contenti(m)! pi: that returns the i-th percept pi in the message m; { comprehends3 (m)! at: that states the comprehension of the message. Intuitively, self action(a; comprehends(m)) means the action by a of comprehending the message m; { produces4(m) ! at: that states that the agent a is producing the message m.
Intuitively, self action(a; produces(m)) means the action by a of producing the message m.
and they have the standard rst-order semantics. 4.5
Axioms As for DCE C, the standard axioms of EC are considered as common-knowledge, that is: [A1] C(8 f; t : initially(f ) ^ :clipped(0; f; t) ) holds(f; t)) [A2] C(8 e; f; t1; t2 : happens(e; t1) ^ initiates(e; f; t1) ^ t1 < t2 ^ :clipped(t1; f; t2) ) holds(f; t2)) [A3] C(8 f; t1; t2 : clipped(t1; f; t2) () [ 9 e; t : happens(e; t) ^ t1 < t < t2 ^ terminates(e; f; t)]) [A4] C(8 a; d; t : happens(action(a; d); t) ) K(a; happens(action(a; d); t); t)) The axioms from [A1] to [A3], that are native of EC, state general and innate understanding human capacity about the causality of events. [A4] is native of
the event which occurs by doing such an action.
We postulate one more axiom pertains the inner speech triggering. It
expresses that when an agent rehearses a message it produces, the inner speech
happens. That is:
[A5] C(8 a; m; t1; t2 : happens(self action(a; produces(m)); t1) ^
happens(self action(a; comprehends(m)); t2) ^ t1 < t2
, happens(self action(a; innerspeaks(m)); t2))
We de ne three more axioms that, according to Morin's theory, state the three
main causal directions between inner speech and self-awareness. Considering that
inner speech is an unconscious activity [
As result:
[A6] 8 a; ; m; (8 pi 2 m); t : initiates(self action(a; innerspeaks(m));
aware(a; pi); t)
3 The term comprehends is the name of the function de ned in the Fluid Construction
Grammar engine [
Finally, [A8] states that inner speech is triggered by focus, that is a speci c inner action. Then: 1. If the agent a focuses on a percept p at time t:
happens(selfaction(a, focus(p)), t) 2. the inner speech action starts, begin m the message whose content is p: selfaction(a, innerspeaks(m)) with p 2 m The following axiom states that is common-knowledge that when an agent perceives a fact, it is focusing almost one percept which will be related to that fact:
[A9] C(8a; t; 9 ( ; p) : P(a; t; ) , happens(self action(a; f ocuses(p)))) Considering that the action to formulate a message generates the corresponding message m, we also postulate the axiom [A10]:
[A10] C(8a; t; 9 ( ; m) :M(a; t; ) ) happens(self action(a; produces(m)))). Finally, the awareness about itself, leads to the self-knowledge of the proposition triggering it, so: [A11] C(8a; t : holds(aware(a); t) , [9 : Kself (a; t; )]).
Some of the inference rules we de ne for reasoning by the calculus are: [R1] SM(a(;aa;;tt;; )) [R2] MP((aa;;tt;; )) The generation of a message M about leads to rehearse it. The perception of leads to a message about .
Kself (a;t; ) [R3] Kself (a;t; Kself (a;t; )) It captures an essential property of self-awareness: the knowledge of self about leads to the knowledge about such a knowledge. That is \I know to know". [R4] K(a;t; )
The knowledge of implies . 6
A concrete example allows us to clarify and to show the application of the
calculus we propose. The scenario we consider is inspired to those described
at [
Therefore, we describe the method for reasoning by the proposed calculus about self-re ection during task execution. The knowledge of the robot changes by self-re ection in respect to the knowledge of the typical reactive behavior, putting the robot in the conditions to make more inferences than the case without self-re ection.
At this step, our goal is to demonstrate how the robot awareness and selfawareness grow by modeling inner speech by the proposed calculus implementing self-re ection. How the questions emerge, and the answers are produced are out of the scope of this paper, and regard important future works. 6.1
Encoding self-re ection We suppose that the robot is engaged in a simple task, that is to remove an object from its locations and to put it in a di erent location. In the reactive behavior, the robot runs a set of routines which allow: to identify the object o, to move to the location l, to grasp the object o from the location l, to place the object o in the location l.
Let suppose that at a certain time during task execution, the robot perceives itself by a mirror, and it sees itself to perform one of the described actions. In particular, it sees itself to grasp the object.
It will be engaged in the following soliloquy: \What am I doing? Did I perform yet this task?".
We introduce the sort Location, and the following function symbols for reasoning about this situation:
grasp : Object ! ActionType remember : ActionType
Moment ! Boolean The set of questions will generate a set of corresponding propositions: \What am I doing?" ) \I grasp the object o" ) happens(action(a, grasp(o)),t) \Did I perform this task?" ) \Yes, I do!" ) remember(grasp(o), t)
We consider two temporal lines which allow to specify the narrative of selfre ection, that are the times before inner speech and the time after inner speech. Begin t1 the limit point, for all times before t1 (i.e. t < t1), the robot performs the task in reactive way, i.e. without self-re ection. At t = t1, the inner speech triggers. For all times after t1 (i.e. t > t1), the robot is provided with selfre ection ability by inner speech. Therefore, the robot upon t1 will know that until this moment it was not aware of the task: then it will be conscious that it has carried out the speci c action. 6.2
Reasoning Now, let suppose that an event triggers inner speech. The robot perceives itself by a mirror. 1. The starting premise is that the robot a sees itself by mirror to perform the action at time t: 2. As consequence, from A9 it focuses on itself:
P(a; t; happens(action(a; ); t)) happens(self action(a; f ocuses(a); t))
Other kinds of percepts may activate the focus, as the object to take. 3. Given A9, from A8 the inner speech starts:
happens(self action(a; innerspeaks( (happens(action(a; )))); t) 4. Hence, from A5 the following events take place:
e1 = happens(self action(a; produces( (happens(action(a; )))))) e2 = happens(self action(a; comprehends( (happens(action(a; )))))) 5. From A6, the uents of awareness start: 8pi 2
(happens(action(a; ))) ! initiates(e; aware(pi))); t) 6. Form A10 the previous uents extend the knowledge about the self because involves a: 7. From R4:
Kself (a; t; happens(action(a; )))
Kself (a; t; Kself (a; t; happens(action(a; )))) By invoking the process with
related to the rst question, that is = happens(action(a; grasp(o); t1); the message becomes = fa; grasp; og. The nal conclusion of an iteration of the reasoning process is the following: the robot talks to itself upon t1, and it becomes aware of itself, of the object o, and that it is performing the action grasp. Moreover, it knows that it knows it. All these knowledge are not considered under t1.
If in the meanwhile the agent focuses on the new proposition
(remember(grasp(o); t1)) the reasoning process iterates by starting from the new message = fremember, grasp, og and new uents extent the agent awareness and self-awareness set.
The proposed method is general-purpose, and allows to reason on inner speech and self-awareness generically: it can be reused in any context that requires reasoning by self-talking, not only for self-re ection.
In this paper, a preliminary version of a sorted-calculus to automate inner speech and its links to awareness and self-awareness is proposed. The idea is to consider the robot awareness and self-awareness as the set of uents the robot knows as active at a time. The calculus allows to reason by natural deduction, and extents the robot general knowledge by including uents representing internal and external conditions. The inner speech allows to activates such uents. A simple scenario is described for demonstrating how the calculus works. Many other aspects have to be considered: the way the inner queries emerge, and the ability to formulate answers are fundamentals. The processes underling the composition of queries and answers may be automated by the same calculus by adding further axioms, functions and sorts, or by considering models of questionanswering systems. The work opens new challenges and research scenario not yet investigated for robot awareness and self-awareness.