<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Workshop on Artificial Intelligence and Formal Verification, Logics, Automata and Synthesis (OVERLAY),
September</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>A Fixed-point Model-checker for BDI Logics over Finite-state Worlds∗</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Salvatore La Torre</string-name>
          <email>1slatorre@unisa.it</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gennaro Parlato</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Molise</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Salerno</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <volume>25</volume>
      <issue>2020</issue>
      <fpage>47</fpage>
      <lpage>52</lpage>
      <abstract>
        <p>bdi agents are among the most widely studied models of rational agents. In this architecture, systems are seen as rational agents with certain mental attitudes such as belief, desire, and intention. In this paper, we consider the model-checking problem for Ctlbdi, the branching-time logic Ctl augmented with the BDI modalities, over finite-state structures, and in particular, a proof-ofconcept tool that is based on a translation to a formula in a fixed-point logic. We give a description of this tool in some details and also discuss some preliminary evaluations.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>Λ(σ, ψ, u) := Λaσt ∨ Λσ¬ ∨ Λσ∨ ∨ Λ∀σ ∨ Λ∃σ ∨ Λ∀σU ∨ Λ∃σU ∨ ΛσB ∨ ΛσD ∨ ΛIσ where:
1. Λ0at := Atomic(ψ) ∧ ¬Label(ψ, u)
2. Λ1at := Atomic(ψ) ∧ Label(ψ, u)
3. Λσ¬ := Neg(ψ) ∧ ∃ψ0.( Ready(ψ0, u) ∧ Sub(ψ0, ψ) ∧ Λ(1 − σ, ψ0, u) )
4. Λσ∨ := Or(ψ)</p>
      <p>∧∃ψ0,ψ00. (Ready(ψ0)∧Sub(ψ0, ψ)∧Ready(ψ00)∧Sub(ψ00, ψ)∧(Λ(σ, ψ0, u) oσΛ(σ, ψ00, u)) ),
where oσ is ∨ if σ = 1 and ∧ otherwise
5. Λ0 := Existential(ψ) ∧ Next(ψ)
∃</p>
      <p>∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∀v. ( ¬SuccT (v, u) ∨ Λ(0, ψ0, v) ) )
6. Λ1 := Existential(ψ) ∧ Next(ψ)
∃</p>
      <p>∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∃v. ( SuccT (v, u) ∧ Λ(1, ψ0, v) ) )
7. Λ0 := Universal(ψ) ∧ Next(ψ)
∀</p>
      <p>∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∃v. ( SuccT (v, u) ∧ Λ(0, ψ0, v) ) )
8. Λ1 := Universal(ψ) ∧ Next(ψ)
∀</p>
      <p>∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∀v. ( ¬SuccT (v, u) ∨ Λ(1, ψ0, v) ) )
9. Λ0∃U := Existential(ψ) ∧ Until(ψ)
∧ ∃ψ0, ψ00. ( Ready(ψ0) ∧ Ready(ψ00) ∧ Sub(ψ0, ψ00, ψ) ∧ Λ(0, ψ00, u)</p>
      <p>∧ ( Λ(0, ψ0, u) ∨ ∀v. ( ¬SuccT (v, u) ∨ Λ(0, ψ, v) ) ) )
10. Λ1∃U := Existential(ψ) ∧ Until(ψ)
∧ ∃ψ0, ψ00. ( Ready(ψ0) ∧ Ready(ψ00) ∧ Sub(ψ0, ψ00, ψ)</p>
      <p>∧ ( Λ(1, ψ00, u) ∨ ( Λ(1, ψ0, u) ∧ ∃v. ( SuccT (v, u) ∧ Λ(1, ψ, v) ) ) ) )
11. Λ0 := Universal(ψ) ∧ Until(ψ)
∀U
∧ ∃ψ0, ψ00. ( Ready(ψ0) ∧ Ready(ψ00) ∧ Sub(ψ0, ψ00, ψ) ∧ Λ(0, ψ00, u)</p>
      <p>∧ ( Λ(0, ψ0, u) ∨ ∃v. ( SuccT (v, u) ∧ Λ(0, ψ, v) ) ) )
12. Λ1∀U := Universal(ψ) ∧ Until(ψ)
∧ ∃ψ0, ψ00. ( Ready(ψ0) ∧ Ready(ψ00) ∧ Sub(ψ0, ψ00, ψ)</p>
      <p>∧ ( Λ(1, ψ00, u) ∨ ( Λ(1, ψ0, u) ∧ ∀v. ( SuccT (v, u) ∧ Λ(1, ψ, v) ) ) ) )
13. Λ0 := bel(ψ) ∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∃v. ( SuccB(v, u) ∧ Λ(0, ψ0, v) ) )</p>
      <p>(siBmilarly for desire and intention formulas)
14. Λ1 := bel(ψ) ∧ ∃ψ0. ( Ready(ψ0) ∧ Sub(ψ0, ψ) ∧ ∀v.( ¬SuccB(v, u) ∨ Λ(1, ψ0, v) ) )
(siBmilarly for desire and intention formulas)</p>
      <p>Syntax. Ctlbdi formulas are inductively defined starting from atomic propositions by applying the
logical connectives, the path quantifiers coupled with a temporal operator such as next ( ) and until (U),
and the belief (bel), desire (des), and intention (int) operators.</p>
      <p>Semantics. The meaning of the formulas from Ctlbdi is defined according to a possible world semantics
where each possible world is not an instantaneous state but a transition system. All the worlds are
synchronized over a shared branching-time structure whose time points (nodes) represent the instantaneous
states. The meaning of the belief-desire-intention (bdi) operators is then given through accessibility
relations that relate the possible worlds at each time point and thus can possibly vary over time, while
the meaning of temporal operators is related to the temporal accessibility relation defined by the the
branching-time structure.</p>
      <p>A finite-state structure constrains a general Ctlbdi structure such that:
• the shared branching-time structure is a full k-ary tree for an integer k &gt; 0 (i.e., an infinite tree
where each node has exactly k children),
• each of the worlds is defined by a Kripke structure whose tree unrolling from its unique initial state
is contained into the shared branching time structure (i.e., each transition is mapped to a child from
1 through k and from each state there is at most an outgoing transition corresponding to a given
child i),
• each bdi relation is defined by a finite automaton that takes as input sequences over the alphabet
{1, . . . , k}, i.e., an automaton that maps paths of the tree-structure to the worlds modeling the bdi
attitude.</p>
      <p>For a structure M, a world w and a Ctlbdi formula ϕ we denote that ϕ is fulfilled on M starting
from w as M, w |= ϕ.</p>
      <p>
        Model-checking. The Ctlbdi model-checking problem over finite-state structures asks whether M, w |=
ϕ holds for given finite-state structure M, world w and Ctlbdi formula ϕ. In [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], we show that this
problem is Pspace-complete and give a fixed-point algorithm to decide it. This algorithm relies on
the construction of a finite graph GM that captures the semantics of M as the cross product of its
Kripke structures and automata synchronized over the sequences from alphabet {1, . . . , k}. The graph
has different kinds of edges depending on whether they capture the successor in a Kripke structure or I
accessibility. Then, starting from the atomic propositions, the algorithm labels the nodes of GM similarly
to the standard labeling algorithm to decide Ctl model-checking (see [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]).
      </p>
      <p>Figure 1 gives the relation that captures the labeling of GM nodes with the fulfilled sub-formulas. This
definition uses predicates to denote the successors in the graph GM, to relate formulas to sub-formulas,
and to denote whether a sub-formula is an atomic proposition, the negation/disjunction of formulas,
universally/existentially quantified, a next/until/belief/desire/intention formula. Additionally, Ready(ψ, u)
denotes the formula ∃σ.Λ(σ, ψ, u).</p>
      <p>
        In particular, the following theorem holds:
Theorem 1. [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] Given a Ctlbdi formula ϕ, a finite-state structure M and a world w,
      </p>
      <p>M, w |= ϕ iff Λ(1, ϕ, u) holds true
where u is the initial state of GM corresponding to w.
3</p>
    </sec>
    <sec id="sec-2">
      <title>Proof-of-concept tool</title>
      <p>
        In this section we give an overview of a prototype tool called bdi-checker for solving the model-checking
problem of Ctlbdi. The underling search space engine uses Binary Decision Diagrams to implement the
algorithm’s operations as well as to represent the set of reachable states during the analysis. To illustrate
our translation we consider a case study taken from [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. We finally elaborate a benchmark starting from
this case study to evaluate the tool.
      </p>
      <p>Case study. We consider a simple scenario where a robot can essentially perform two tasks: getting
a beer from the refrigerator and opening the door. The only uncertainties in the environment are the
presence or not of a beer can in the refrigerator and of a person at the door house. We can model these as
beliefs and thus have four different Kripke structures one for each of the possible beliefs.</p>
      <p>In the beginning, all the beliefs are possible and thus all the worlds are belief-accessible. As soon
as the robot realizes that no beer is in the refrigerator only the two worlds matching this belief become
accessible. Also, if the doorbell rings, the robot changes its beliefs about the presence of a person at the</p>
      <sec id="sec-2-1">
        <title>TRANSLATION INTO PREDICATES</title>
        <p>• Transition systems nodes
• AP labeling
• temporal, belief, desire,
intention successors
• Subformula representation
• Subformula relation
• Subformula types</p>
      </sec>
      <sec id="sec-2-2">
        <title>COMPOSE</title>
      </sec>
      <sec id="sec-2-3">
        <title>FIXED-POINT</title>
      </sec>
      <sec id="sec-2-4">
        <title>FORMULA</title>
        <p>Predicates
Fixed-point
algorithm</p>
      </sec>
      <sec id="sec-2-5">
        <title>MUCKE</title>
        <p>YES/NO
counterexample
house door. After opening the door, the robot becomes again agnostic on whether there is a person at
the door. After sensing that no beer is in the refrigerator instead its beliefs about the content of the
refrigerator will not change forever (this might be changed by adding a further event that a delivery man
brings some beer cans).</p>
        <p>Robot desires and intentions can be modeled by adding more worlds “refining” the belief related worlds.
To keep the description simple, in this version of the paper we omit a detailed discussion of such aspects.</p>
        <p>We consider two Ctlbdi formulas: ϕ1 = ∀ ( bel br → ∃ bb), i.e., “whenever the robot believes that
a beer can is in the refrigerator, she can possibly bring it back”, and ϕ2 = ∀ ( bel br → ∃ bb ), i.e.,
“whenever the robot believes that a beer can is in the refrigerator, she can always bring it back”. The first
one is fulfilled on our structure, while the second one is not.</p>
        <p>Architecture. The tool architecture is shown in Figure 2. bdi-checker takes as input a finite-state
structure M and a Ctlbdi formula ϕ, and returns as result “YES”, if the model M meets the specification
ϕ, and a counter-example witnessing that M does not satisfy ϕ, otherwise.</p>
        <p>The first internal module of bdi-checker, called TRANSLATION INTO PREDICATES, takes as input
M and ϕ and transforms them into a series of definitions of Boolean predicates. In particular, we define
the predicates used in the definition of formula Λ given in Figure 1: we encode the nodes of graph GM
from the Kripke structures and the automata of M, and capture the successor relations using predicates;
and from ϕ we encode the subformulas used in the labeling algorithm and capture their syntactic type
again by using predicates.</p>
        <p>For our case study, we encode the nodes of the Kripke structures and the automata of M as integers,
and thus the nodes of GM as tuples of integers. The generated predicates use this encoding. As an
example, the predicate encoding the labeling of GM vertices with formulas is defined as follows:
bool Label(Formula f, GVertex u) (</p>
        <p>(u.w=0 &amp; Lab1(f,u.s0) )
| (u.w=1 &amp; Lab2(f,u.s1) )
| (u.w=2 &amp; Lab3(f,u.s2) )
| (u.w=3 &amp; Lab4(f,u.s3) )
)
where Lab1, Lab2, Lab3, and Lab4 are the predicates capturing the labeling of the four Kripke structures
of M , w is the u component denoting the current Kripke structure (i.e, the current world), and s0, s1,
s2, and s3 denote the u components corresponding to the states of the respective Kripke structures.</p>
        <p>
          We then encode the seven subformulas of ϕ1 as integers in the interval [
          <xref ref-type="bibr" rid="ref6">0, 6</xref>
          ] (according to the syntax
tree of ϕ1 starting with 0 for ϕ1 itself, and then proceeding top-down and left-to-right). With such an
encoding, we define the predicates used in Figure 1. For example, we have the following definitions:
bool Atomic( Formula f )
bool Universal( Formula f )
bool Until( Formula f )
( f=5 | f=6 )
( f=0 )
( false )
        </p>
        <p>Once, all the predicates are defined, the original model-checking question, that is, determining whether
M, w |= ϕ, can be translated in the following clause:</p>
        <p>( exists GVertex u. Lambda(true,0,u) ),
which, in our case study, simply asks the backend solver to verify whether a vertex of GM exists where the
input formula ϕ1 holds true. Thus, in the second module, called COMPOSE FIXED-POINT FORMULA,
we just put together the predicates computed by the first module with the fixed-point algorithm given in
Figure 1 and the above clause.</p>
        <p>The last module just invokes the backend solver Mucke on the built formula and returns its outcome.
Evaluation. To evaluate our prototype tool we have elaborated a benchmark by extending the given
case study with the layout of the environment. A layout is given as a bi-dimensional grid where the
refrigerator, the door, the lounge (the base position of the robot) and some obstacles are positioned. This
benchmark is parameterized on the size of the grid, the number of instances and the positioning of the
objects (refrigerator, door, lounge, and obstacles). This will allow us to evaluate the scalability of the
approach by varying the parameters. However, to date, we have only exercised our tool on a simple
scenario (2 × 2 grid) against the two different Ctlbdi formulas given above. The tool has performed quite
well: it gives the right answer in a fraction of a second with BBD maximum size of 1134. We were unable
to perform a systematic evaluation since most of the translations in our tool are still performed manually
which prevents us to manage large benchmarks. As future work, we plan to complete the automatization
of the modules of our tool and then perform a full evaluation over the described benchmark.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>M.</given-names>
            <surname>Benerecetti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Giunchiglia</surname>
          </string-name>
          , and
          <string-name>
            <given-names>L.</given-names>
            <surname>Serafini</surname>
          </string-name>
          .
          <article-title>Model checking multiagent systems</article-title>
          .
          <source>J. Log. Comput.</source>
          ,
          <volume>8</volume>
          (
          <issue>3</issue>
          ):
          <fpage>401</fpage>
          -
          <lpage>423</lpage>
          ,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>A.</given-names>
            <surname>Biere</surname>
          </string-name>
          . μcke
          <article-title>- efficient μ-calculus model checking</article-title>
          . In O. Grumberg, editor, Computer Aided Verification, 9th International Conference, CAV '97,
          <string-name>
            <surname>Haifa</surname>
          </string-name>
          , Israel, June 22-25,
          <year>1997</year>
          , Proceedings, volume
          <volume>1254</volume>
          of Lecture Notes in Computer Science, pages
          <fpage>468</fpage>
          -
          <lpage>471</lpage>
          . Springer,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.</given-names>
            <surname>Blee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Billington</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Governatori, and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Sattar</surname>
          </string-name>
          .
          <article-title>Levels of modality for BDI logic</article-title>
          .
          <source>J. Applied Logic</source>
          ,
          <volume>9</volume>
          (
          <issue>4</issue>
          ):
          <fpage>250</fpage>
          -
          <lpage>273</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>R. H.</given-names>
            <surname>Bordini</surname>
          </string-name>
          , M. Fisher, W. Visser, and
          <string-name>
            <given-names>M. J.</given-names>
            <surname>Wooldridge</surname>
          </string-name>
          .
          <article-title>Verifying multi-agent programs by model checking</article-title>
          .
          <source>Autonomous Agents and Multi-Agent Systems</source>
          ,
          <volume>12</volume>
          (
          <issue>2</issue>
          ):
          <fpage>239</fpage>
          -
          <lpage>256</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Emerson</surname>
          </string-name>
          .
          <article-title>Temporal and modal logic</article-title>
          .
          <source>In Handbook of Theoretical Computer Science</source>
          , Volume B:
          <article-title>Formal Models</article-title>
          and
          <string-name>
            <surname>Sematics</surname>
          </string-name>
          (B), pages
          <fpage>995</fpage>
          -
          <lpage>1072</lpage>
          . The MIT Press/Elsevier,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>E. A.</given-names>
            <surname>Emerson</surname>
          </string-name>
          and
          <string-name>
            <given-names>E. M.</given-names>
            <surname>Clarke</surname>
          </string-name>
          .
          <article-title>Using branching time temporal logic to synthesize synchronization skeletons</article-title>
          .
          <source>Sci. Comput. Program.</source>
          ,
          <volume>2</volume>
          (
          <issue>3</issue>
          ):
          <fpage>241</fpage>
          -
          <lpage>266</lpage>
          ,
          <year>1982</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>La Torre</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Madhusudan</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Parlato</surname>
          </string-name>
          .
          <article-title>Analyzing recursive programs using a fixed-point calculus</article-title>
          . In M.
          <article-title>Hind and A</article-title>
          . Diwan, editors,
          <source>Proceedings of the 2009 ACM SIGPLAN Conference on Programming Language Design and Implementation</source>
          ,
          <string-name>
            <surname>PLDI</surname>
          </string-name>
          <year>2009</year>
          , Dublin, Ireland, June 15-21,
          <year>2009</year>
          , pages
          <fpage>211</fpage>
          -
          <lpage>222</lpage>
          . ACM,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>S.</given-names>
            <surname>La Torre</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Madhusudan</surname>
          </string-name>
          , and
          <string-name>
            <given-names>G.</given-names>
            <surname>Parlato</surname>
          </string-name>
          .
          <article-title>Model-checking parameterized concurrent programs using linear interfaces</article-title>
          . In T. Touili,
          <string-name>
            <given-names>B.</given-names>
            <surname>Cook</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P. B.</given-names>
            <surname>Jackson</surname>
          </string-name>
          , editors,
          <source>Computer Aided Verification, 22nd International Conference, CAV</source>
          <year>2010</year>
          ,
          <article-title>Edinburgh</article-title>
          ,
          <string-name>
            <surname>UK</surname>
          </string-name>
          ,
          <source>July 15-19</source>
          ,
          <year>2010</year>
          . Proceedings, volume
          <volume>6174</volume>
          of Lecture Notes in Computer Science, pages
          <fpage>629</fpage>
          -
          <lpage>644</lpage>
          . Springer,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>S. La</given-names>
            <surname>Torre</surname>
          </string-name>
          and
          <string-name>
            <given-names>G.</given-names>
            <surname>Parlato</surname>
          </string-name>
          .
          <article-title>Model checking bdi logics over finite-state worlds</article-title>
          .
          <source>In Proceedings of the 1st Workshop on Artificial Intelligence and Formal Verification</source>
          , Logic, Automata, and
          <article-title>Synthesis co-located with the 18th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2019), Rende (CS), Italy</article-title>
          ,
          <source>November 19-20</source>
          ,
          <year>2019</year>
          , pages
          <fpage>11</fpage>
          -
          <lpage>16</lpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S. La</given-names>
            <surname>Torre</surname>
          </string-name>
          and
          <string-name>
            <given-names>G.</given-names>
            <surname>Parlato</surname>
          </string-name>
          .
          <article-title>On the model-checking of branching-time temporal logic with BDI modalities</article-title>
          . In
          <string-name>
            <given-names>A. E. F.</given-names>
            <surname>Seghrouchni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Sukthankar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>An</surname>
          </string-name>
          , and N. Yorke-Smith, editors,
          <source>Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems</source>
          , AAMAS '20,
          <string-name>
            <surname>Auckland</surname>
          </string-name>
          , New Zealand, May 9-
          <issue>13</issue>
          ,
          <year>2020</year>
          , pages
          <fpage>681</fpage>
          -
          <lpage>689</lpage>
          . International Foundation for Autonomous Agents and
          <string-name>
            <given-names>Multiagent</given-names>
            <surname>Systems</surname>
          </string-name>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>J.-J. C. Meyer</surname>
          </string-name>
          , J. Broersen,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Herzig</surname>
          </string-name>
          .
          <article-title>Bdi logics</article-title>
          .
          <source>In Handbook of Epistemic Logic</source>
          , pages
          <fpage>453</fpage>
          -
          <lpage>498</lpage>
          . College Publications,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>A. S.</given-names>
            <surname>Rao</surname>
          </string-name>
          and
          <string-name>
            <given-names>M. P.</given-names>
            <surname>Georgeff</surname>
          </string-name>
          .
          <article-title>Modeling rational agents within a bdi-architecture</article-title>
          . In J. F. Allen,
          <string-name>
            <given-names>R.</given-names>
            <surname>Fikes</surname>
          </string-name>
          , and E. Sandewall, editors,
          <source>Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning (KR'91)</source>
          . Cambridge, MA, USA, April
          <volume>22</volume>
          -
          <issue>25</issue>
          ,
          <year>1991</year>
          ., pages
          <fpage>473</fpage>
          -
          <lpage>484</lpage>
          . Morgan Kaufmann,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A. S.</given-names>
            <surname>Rao</surname>
          </string-name>
          and
          <string-name>
            <given-names>M. P.</given-names>
            <surname>Georgeff</surname>
          </string-name>
          .
          <article-title>A model-theoretic approach to the verification of situated reasoning systems</article-title>
          . In R. Bajcsy, editor,
          <source>Proceedings of the 13th International Joint Conference on Artificial Intelligence. Chambéry</source>
          , France,
          <source>August 28 - September 3</source>
          ,
          <year>1993</year>
          , pages
          <fpage>318</fpage>
          -
          <lpage>324</lpage>
          . Morgan Kaufmann,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>