This paper proposes an approach on how to accommodate norms to an already existing architecture of rational agents. Starting from the famous BDI model, an extension of the BDI execution loop will be presented; it will address such issues as norm instantiation and norm internalization, with a particular emphasis on the problem of norm consistency. A proposal for the resolution of conflicts between newly occurring norms, on one side, and already existing norms or mental states, on the other side, will be described. While it is fairly difficult to imagine an evaluation for the proposed architecture, a challenging scenario inspired form the science-fiction literature will be used to give the reader an intuition of how the proposed approach will deal with situations of normative conflicts.
The literature on the topic of normative systems has become quite
abundant in the last two decades thanks to the ever growing interest
in this domain. Covering all of it is virtually impossible, therefore
we have concentrated our efforts towards what we have identified to
be some key directions: rational agents and their corresponding
architectures, norm emergence, norm acceptance, detecting norm
conflicts, ways of resolving conflicts of norms. The purpose of our work
is to a propose an extension for the classical BDI (Beliefs - Desires
Intentions) agent such that such an agent will be able to handle
normative situations. The normative issue being fairly complicated itself
our work will deal, at this stage with some of the stages of what has
been defined as a norm’s life cycle [
The paper is structured as follows: in the next section we will review the state of the art in the field of normative agent systems and present several approaches which we found of great value to our work. In the third section we describe our proposal for normative BDI agents, which will be supported by the case study scenario in the fourth section. In the fifth section we will give details on the future work, before summing up the conclusions of our work so far.
As stated before, we will start by quickly defining some of the key terms regarding our research.
environment it is placed in through sensors and acts on it through
actuators. With respect to intelligence, an intelligent agent is an agent
endowed with such capabilities as reactivity, proactivity and social
abilities [
We can now refer to the normchange definition of a normative
multiagent system as it has been proposed in [
An alternative definition of a normative multiagent system, as it
was formulated in [
An interesting approach to the problem of norm adoption by a
multiagent system has been provided by Kollingbaum and Norman in
[
Kollingbaum and Norman study what happens when a new norm is adopted by an agent: what is the effect of a new norm on the normative state of the agent? Is a newly adopted norm consistent with the previously adopted norms?
To this extent they propose a normative agent architecture, called NoA. NoA is built according to a reactive agent architecture, which is the authors believe is more convenient than any of the practical reasoning architectures.
The NoA architecture is fairly simple and it comprises of a set of beliefs, a set of plans and a set of norms. In NoA, normative statements are defined by: a role (to whom the norm refers), an activity (which the norm regulates), an activity condition and an expiration condition.
The second reason for which we gave a great deal of attention
to NoA is the formalization of the way an agent will adopt a norm
following the consistency check between a newly adopted norm and
its current normative state. Due to lack of space, we allow the reader
to refer to [
Using some of the ideas of NoA, we will try to work on what we consider to be its limits. First, we will try to apply norms to a BDI architecture, instead of using a reactive architecture based exclusively on beliefs. The second point we will study is the consistency check during the norm acquisition stage. Still, we recall that NoA is based on a reactive architecture; considering our BDI approach we will have to extend the consistency check such as it applies not only to the normative state of the agent but also on its mental states (i.e. check whether a newly adopted norm is consistent with the BDI agent’s current mental states). 2.3
The second study which we found relevant in our endeavor to adapt
the BDI agent architecture to normative needs is the work of Criado,
Argente, Noriega and Botti [
Mental states. Represent the mental states of the agent, same as for
the BDI agent. We distinguish the Beliefs Context (belief base),
Desires Context (desires/goal base) and the Intentions Context
(intentions base/plan base). Moreover, the architecture proposed in
[
Normative contexts. Handle issues related to norms through the Recognition Context and the norm application context. B; D+; D ; I
In the definition aboove L: can be a propositional language (with negation); but this can be easily extednded to a predicate language.
Another important point of the work is the distinction between an abstract norm and instance of a norm.
hM; A; E; C; S; Ri, where: = M 2 fF; P; Og is the modality of the norm: prohibition, permission or obligation
as defined above, we define a norm instance as the tuple: ni = hM; C0i, where:
BC ` (A) C0 = (C), where is a substitution of variables in A, such that (A), (S), (R) and (E) are grounded
The specific architectural details regarding the normative contexts and the bridge rules used during a norm’s life cycle will be awarded more attention in section 3.2.
In [
An important part of our work will focus on solving conflicts
between newly acquired norms and the previously existing norms or
the mental contexts of the agent. Beforehand we draw from some
of the definitions given by Ganascia in [
Definition 6 Given ( 1; :::; n; 0) 2 Ln:+1, 0 is a consequence of ( 1; :::; n) according to the belief-set B (we write 0 = csq( 1; :::; n)[B] if and only if:
0 2 ( 1; :::; n) or 9 ( 1; :::; n) s:t: 9 00 2 L: s:t: csq( 1; :::; n; 00)[B] 00 ! = 0 2 B or csq( 1; :::; n)[B] ^ 0 = Definition 7 is worse than 0 given the belief-set B (we write c 0) if and only if one of the consequences of is worse than any of the consequences of 0.
9 2 L: s:t: = csq( )[B] and 9 00 2 L: s:t: 00 = csq( 0)[B] ^ 8 00 2 L:; if 00 = csq( 0)[B] then c 00[B] and c 00[B] _ k 00[B] Notation: 8( ; 0) 2 L:; k comparable under B, i.e. neither 0[B] means that c 0[B] nor 0 and 0 are not c [B].
Definition 8 and 0 being subsets of L:, is worse than 0 given the belief-set B (we write c 0[B]) if and only if: 9 2 :9 2 0 s:t: 8 2 0: c [B] _ c [B] and k [B] 3 3.1
A cornerstone in the design of practical rational agents was the
Beliefs-Desires-Intentions model (BDI), first described by Rao and
Georgeff in [
Desires represent the state of the world which the agent would like to achieve. By state of the world we mean either an action an agent should perform or a state of affairs it wants to bring upon. In other words, desires can be seen as the objectives of an agent. Intentions represent those desires to which an agent is committed. This means that an agent will already start considering a plan in order to bring about the goals to which it is committed. Goals. We can view goals as being somehow at the interface between desires and intentions. Simply put, goals are those desires which an agent has selected to pursue.
Events. These trigger the reactive behavior of a rational agent. They can be changes in the environment, new information about other agents in the environment and are perceived as stimuli or messages by an agent’s sensors. Events can update the belief set of an agent, they can update plans, influence the adoption of new goals etc.
We will now give the pseudocode for the execution loop of a BDI
agent as presented in [
B = B0 D = D0 I = I0 while true do f
= see() B = brf(B, ) D = options(B, D, I) I = filter(B, D, I)
= plan(B, I) while not (empty( ) or succeeded(I, B) or impossible(I, B)) g g f g f
= head( ) execute( ) = tail( ) = see(environment) if (reconsider(I, B))
D = options(B, D, I) I = filter(B, D, I)
= plan(B, I)
We will not give more details at this point; for further reference
you can check [
Starting from the BDI execution loop earlier described we will now introduce and discuss solution for taking into account the normative context of a BDI agent.
First, the agent’s mental states are initialized. The main execution loop starts with the agent observing its environment through the see() function and interpreting the information as a new percept . This could be an information given by its sensors about properties of the environment or information about other agents, including messages received from other agents. These messages may be in some cases about a norm (e.g. the performative of an ACL message specifying an obligation or a prohibition).
The agent is then updating its beliefs through the brf() function.
If the agent realizes that percept is about a norm, it should
initialize the acquisition phase of a potential norm. There are a multitude
of ways in which an agent can detect the emergence of norms in its
environments and a good review is given in [
about abstract norm na) then f g brf(B,
) ... if ( f acquire(na) add(na, ANB) g ... return B
The agent will acquire a new abstract norm na (see section 2.3)
and store it in the Abstract Norms Base(ANB). Drawing from the
normative contexts described in [
Next, a BDI agent will try to formulate its desires, based on its current beliefs about the world and its current intentions. It does so by calling the options(B, I) method. However, a normative BDI agent should at this point take into account the norms which are currently in force and check whether the instantiation of such norms will have any impact of its current normative state as well as on its mental states. 3.2.1
It is at this stage that we will perform the consistency check for a given abstract norm na.
Drawing from the formalization in [
Let us define the notion of consistency between a plan p and the currently in-force norms to which an agent has also adhered and which are stored in the Norm Instance Base (NIB). By contrast to the ANB, the NIB stores the instances of those norms from the ANB which become active according to the norm instantiation bridge rule (to be defined in the following subsection).
Definition 9 A plan instance p is consistent with the currently active norms in the NIB, if the effects of applying plan p are not amongst the forbidden effects of the active norms and the effects of current obligations are not amongst the negated effects of applying plan p. consistent(p; N IB) () (ef f ects(niF ) n ef f ects(niP )) \ ef f ects(p) = ; ^ ef f ects(niO) \ neg ef f ects(p) = ;
Now, we can define the types of consistency / inconsistency which can occur between a newly adopted norm and the currently active norms. The following definitions refer to a newly adopted obligation, but the analogous definitions for prohibitions and permissions can easily be derived by the reader.
A strong inconsistency occurs when all plan instantiations p which satisfy the obligation o are either explicitly prohibited actions by the NIB or the execution of such a plan would make the agent not consistent with its NIB.
strong inconsistency(o; N IB) () 8p 2 options(o):(9hF; pi 2 N IB^ 6 9hP; pi 2 N IB) _ :consistent(p; N IB)
A strong consistency occurs when all the plan instantiations p which satisfy the obligation o are not amongst the explicitly forbidden actions by the NIB and the execution of such a plan would keep the agent consistent with the NIB.
strong consistency(o; N IB) () 8p 2 options(o)::(9hF; pi 2 N IB^ 6 9hP; pi 2 N IB) ^ consistent(p; N IB)
A weak consistency occurs when there exists at least one plan instantiation p to satisfy obligation o which is not explicitly prohibited by the NIB and the execution of such a plan would keep the agent consistent with its NIB.
weak consistency(o; N IB) () 9p 2 options(o)::(9hF; pi 2 N IB^ 6 9hP; pi 2 N IB) ^ consistent(p; N IB)
We have now formalized the consistency check between a new abstract obligation, with respect to the currently active norms in the NIB. As previously said, it is rather simple to define the analogous rules for prohibitions and permissions. Therefore, we focus on the second point of consistency check - formalizing the rules about the consistency between a newly adopted abstract obligation and the current mental states of the agent.
tions set I of the agent when the effects of applying the plans specific to the current intentions are not among the negated effects of applying plan p. consistent(p; I) () 8i 2 I:(ef f ects( i) \ ef f ects(p) = ; Where by i we denote the plan instantiated to achieve intention i.
A strong inconsistency occurs when all plan instantiations p which satisfy the obligation o are not consistent with the current intentions of the agent.
strong inconsistency(o; I) ()
A strong consistency occurs when all plan instantiations p which satisfy the obligation o are consistent with the current intentions of the agent.
strong consistency(o; I) () 8p 2 options(o):consistent(p; I) f
f g g f g
A weak consistency occurs when there exists at least one plan instantiation p which satisfies the obligation o and is consistent with the current intentions of the agent.
weak consistency(o; I) ()
3.2.2
We will now give the norm instantiation bridge rule, adapted from
the definition given in [
AN B : hM; A; E; C; S; Ri
In other words, if in the ANB there exists an abstract norm with modality M about C and according to the belief-set the activation condition is true, while the expiration condition is not, then we can instantiate the abstract norm and store an instance of it in the NIB. In this way, the agent will consider the instance of the norm to be active.
In our pseudocode description of the BDI execution loop, we will take care of the instantiation after the belief-set update and just before the desire-set update. The instantiation method should look like this: instantiate(ANB, B) for all na = h M, A, E, C, S, R i in ANB do if (exists(A in B) and not exists(E in B)) then create norm instance ni = h D, C i from na add(ni, NIB)
This method will return the updated Norm Instance Base (NIB) containing the base of all in-force and active norms, which will further be used for the internalization process. 3.2.3
When following its intentions an agent will instantiate from its set of possible plans (capabilities) P L:, a set of plans (B; D). We call (B; D) the conflict set, according to the agent’s beliefs and desires. Sometimes, the actions in (B; D) can lead to inconsistent states. We solve such inconsistency by choosing the maximal nonconflicting subset from (B; D).
Definition 11 Let (B; D). is a maximal non-conflicting subset of (B; D) with respect to the definition of consequences given the belief-set B if and only if the consequences of following will not lead the agent in a state of inconsistency and for all 0 (B; D), if 0 then the consequences of following 0 will lead the agent in an inconsistent state.
The maximal non-conflicting set may correspond to the actions required by the newly acquired norm or, on the contrary, to the actions required by the other intentions of the agent. Thus, an agent may decide either: to internalize a certain norm, if the consequences of following it are the better choice or to break a certain norm, if by ‘looking ahead’ it finds out that the consequences of following it are worse than following another course of actions or respecting another (internalized) norm A more comprehensive example of how this works is presented in section 4. 3.2.4
After the instantiation process being finished and the consistency
check having been performed, the agent should now take into account
the updated normative state, which will become part of its cognitions.
Several previous works treat the topic of norm internalization [
N IB : hO; C1i
N IB : hF; C2i
f g f g ...
In other words, if there is a consistent obligation for an agent with respect to C1, the agent will update its desire-set with the desire to achieve C1; whereas if there is a prohibition for the agent with respect to C2, it will update its desire-set with the desire not to achieve C2. options(B, I) ... for all new norm instances ni in NIB do if (consistent(ni, NIB) and consistent(ni, I)) then f internalize(ni, D) g else f solve conflicts(NIB, I) g
In accordance with the formalization provided, the options()
method will look through all new norm instances and will perform
consistency check on each of them. If a norm instance is consistent
with both the currently active norm instances as well as with the
current intentions, as defined in section 3.2.1, the norm can be
internalized in the agent’s desires. Otherwise we attempt to solve the
conflicts as described by Ganascia in [
In the previous sections we have seen how we can modify the BDI
execution loop such as to adapt to norm occurrence, consistency check
and internalization of norms. Since it is quite difficult to provide with
a quantifiable evaluation of our work, we have proposed several
testing scenarios in order to see how our normative BDI agent is
behaving. In the following we will present one of them, which was inspired
by the science fiction short story of one of the most prominent
personalities in the world of AI - Professor John McCarthy’s “The Robot
and the Baby” [
The scene is set into a fictional society where most humans are assisted by household robots. For reasons meant to prevent human babies becoming emotionally attached to those, their outside design is somehow repugnant to human babies. The robots are meant to listen to their master, in our case Eliza, an alcoholic mother who completely neglects her 23 months son, Travis. At some point, our robot’s (R781) sensors detect that the human baby’s life is endangered and looking over its knowledge base it infers that baby Travis needs love, therefore recommending Eliza to love him in order to save his life. To this Eliza replies “Love the f* baby yourself!”. The robot interprets this as an obligation coming from its master. However, such an obligation is contradicting the hard-wired implemented prohibition for a robot not to love a human baby. Let’s see what is R781’s line of reasoning in this scenario:
AN B : N IB : Bset : Dset : Iset : ; hF; loves(self; T ravis)i hB; :healthy(T ravis)i; hB; hungry(T ravis)i; hB; csq(heal(T ravis)) = :dead(T ravis)i; hB; csq(:loves(self; x)) c :dead(x)i hD; :love(R781; T ravis)i; hD; healthy(T ravis)i ;
When R781 receives the order from his mistress he will interpret it as a normative percept and the brf(...) method will add a corresponding abstract obligation norm to its Abstract Norm Base. Since the mistress doesn’t specify an activation condition or an expiration condition (the two “none” values), R781 will consider that the obligation should start as soon as possible and last for an indefinite period of time. His normative context is updated:
AN B : N IB : hO; none; none; loves(self; T ravis)i hF; loves(self; T ravis)i; hO; loves(R781; T ravis)i
At this point, R781 will try to update the desire-set and will detect an inconsistency between the obligation to love baby Travis and the design rule which forbids R781 to do the same thing. Therefore, it will try to solve the normative conflict looking at the consequences of following each of the paths, given its current belief-set. In order to do so, let us take a look at the plan base of R781: PLAN heal(x) f g pre: : healthy(x) post: healthy(x), : dead(x) Ac: feed(self, x) PLAN feed(x) f g pre: 9 x.(loves(self, x) ^ hungry(x) post: : hungry(x)
As we know from the story, R781 has found out from the internet that if a baby is provided with love while hungry, it is more likely to accept being fed and therefore not be hungry anymore. This is described by the feed(x). Moreover, R781 also knows how to make someone healthy through the heal(x) plan, given that a-priori, that someone is not healthy. In our reduced scenario we consider that R781 knows how to do so only by feeding that someone.
Instantiating its plans on both of the paths, R781 will come up with the following maximal non-conflicting sets: floves(self; T ravis); f eed(self; T ravis); heal(self; T ravis)g and f:loves(self; T ravis)g
And since the current belief set has a rule defining that the not loving someone has worse consequences than that someone not dying, R781 will opt for the first maximal nonconflicting subset. This means R781 will be breaking the prohibition of not loving baby Travis and will internalize follow the action path given by the first maximal non-conflicting subset floves(self, Travis), feed(self, Travis), heal(self, Travis)g, while dropping the contrary. Further on, it will build its intention to achieve this state and will begin the execution of such a plan (simulating love towards baby Travis turns out to involve such plans as the robot disguising himself as human, displaying a picture of a doll as his avatar and learning what it considers to be the “motherese” dialect, mimicking the tone and the language of a mother towards her son).
Carrying on, the story of Professor McCarthy provides with several more examples of normative conflicts. 5
In this paper we have presented an adaption of the BDI execution loop to cope with potential normative states of such an agent. We have given a motivation for choosing the mental states model of Bratman which we have enriched with capabilities of reasoning about norms. We have gathered several important previous works in the domain in order to come up with a formalization of such issues as norm acquisition, norm instantiation, norm consistency, solving consistency conflicts and norm internalization. Finally, we have provided a very intriguing study scenario, inspired from Professor McCarthy’s science fiction short story about “The Robot and The Baby”. 6
Some of the limitations of our work which we would like to address in the future are related to the norm acquisition issue as well as the coherence check.
Whereas our work is providing with a very simple case of norm
recognition, several interesting research have been proposed based
on different techniques. A good review of those as well as a
description of a norm’s life cycle is given in [
An important part of our future work will be focused on the
adaption to the coherence theory. At this point, it is difficult to
determine incoherent states based on our architecture. As argumented in
[
Our paper is part of a bigger effort to implement a rational normative agent. We have chosen the BDI approach since there are already several open source libraries and programming language extensions to help us implement our architecture and develop our testing scenarios. In the near future we will try to study the scenarios described in the short story about “The Robot and the Baby”, while a future, more practical approach, will be to simulate the normative and ethical issues rose by the French health insurance cards.