<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Group Intention is Social Choice with Commitment</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Guido Boella</string-name>
          <email>guido@di.unito.it</email>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gabriella Pigozzi</string-name>
          <email>gabriella.pigozzi@uni.lu</email>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marija Slavkovik</string-name>
          <email>marija.slavkovik@uni.lu</email>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Leendert van der Torre</string-name>
          <email>leon.vandertorre@uni.lu</email>
        </contrib>
      </contrib-group>
      <fpage>115</fpage>
      <lpage>133</lpage>
      <abstract>
        <p>A collaborative group is commonly defined as a set of agents, which share information and coordinate activities while working towards a common goal. How do groups decide which are their common goals and what to intend? According to the much cited theory of Cohen and Levesque, an agent intends g if it has chosen to pursue goal g and it has committed itself to making g happen. Following the same line of reasoning, a group intention should be a collectively chosen goal with commitment. The literature often considers a collective goal to be one of those individual goals that are shared by all members. This approach presumes that a group goal is also an individual one and that the agents can act as a group if they share the beliefs relevant to this goal. This is not necessarily the case. We construct an abstract framework for groups in which common goals are determined by social choice. Our framework uses judgment aggregation to choose a group goal and a multi-modal multi-agent logic to define commitment and revision strategies for the group intentions.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        1 Introduction
An agent acts according to its beliefs and intentions. According to what does a group
act? We would expect that in order for groups to act, jointly or by coordinating their
activities, they need to establish what to believe i.e., form epistemic attitudes, and what
to aim for, i.e., to form motivational attitudes. In existing frameworks for collaborative
activities [
        <xref ref-type="bibr" rid="ref12 ref16 ref9">9, 12, 16</xref>
        ] and group decision-making protocols [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ], the formation of group
attitudes is defined only for a specific type of groups. These groups consist of agents
that engage in pursuing a group goal only when the members have the same beliefs
regarding this goal, or when they are successful in reaching an agreement on a given set
of beliefs.
      </p>
      <p>Consider a group of robots in charge of office building maintenance. One candidate
group goal for them is to clean the meeting room. The decision rule to adopt the goal is
that the room needs to be cleaned when the following conditions (reasons) are met: the
floors are dirty or the garbage bin is full, and people do not occupy the room. To decide
whether to pursue this goal, the robots need to decide if the reasons to adopt the goal
are true. The robots cannot check whether the room is occupied or what state it is in.
Hence, to estimate the state of affairs, the robots need to rely on their individual beliefs,
which may diverge.</p>
      <p>
        Assume there are three robots in the group. One robot believes that the room is
occupied and thus, according to it, the group should not adopt the goal. According to the
other two robots, the group should adopt the goal. One robot believes that the garbage
bin is full and the floors clean and the other that the floors are dirty. The question is how
to should one aggregate the beliefs of the robots in this case? The majority of the robots
would estimate that the goal should be adopted. However the group is not univocal
and as a result, the goal would not be chosen for a group goal when the robots reason
according to [
        <xref ref-type="bibr" rid="ref12 ref16 ref9">9, 12, 16</xref>
        ]. A general method for forming group attitudes needs to specify
how group attitudes are formed also when agents have disagreeing beliefs or limited
persuasive abilities. A framework that allows for such general methods is still lacking.
      </p>
      <p>Consider now that the group adopts the goal, but before pursuing it, the robots learn
that there is a seminar scheduled in the meeting room. The group has to de-commit from
their intention to clean the room. However, to be able to do so, the group has to have a
commitment strategy that allows de-committing upon change in the reasons for the goal
adoption. Furthermore, the group needs to be able to reconsider its reasons for goals,
and goals, after de-committing. If they simply drop the goal, without “remembering”
why, they would not be able to re-deliberate and commit again once the seminar is over.</p>
      <p>
        An intelligent agent reacts to the changes in its environment and a group should be
capable of doing the same. When new information becomes available the group faces
a choice: to remain committed to its group attitudes or to reconsider them. The most
sensitive commitment strategy proposed by Rao and Georgeff [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] allows for a group
reaction only when the goal is accomplished or impossible. However, the existence of a
goal is intertwined with the existence of some beliefs [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Consequently, there is a need
for a commitment strategy that reacts to new information regarding those beliefs on
which the goal hinges. Furthermore, groups need to know not only when to de-commit
from their goals but also how to reconsider their goal-related beliefs.
      </p>
      <p>The research question we address in this paper is:</p>
    </sec>
    <sec id="sec-2">
      <title>How can groups choose and reconsider their goals?</title>
      <p>
        A good methodology for collectively choosing and reconsidering goals is one that
can be used by any group of agents regardless of the homogeneity of the individual
beliefs of its members and their persuasion abilities. The relation between individual goals
and beliefs can be specified and analyzed in modal agent logics like BDILT L [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ].
The challenge in group goal generation is to incorporate the aggregation of individual
attitudes into collective ones, as studied in merging, judgment aggregation and social
choice [
        <xref ref-type="bibr" rid="ref11 ref13 ref14 ref17 ref3">3, 11, 13, 14, 17</xref>
        ]. However, using this approach is not straightforward. The main
difficulty lies in the inability to use judgment aggregation directly in a BDILT L
framework. Properties of judgment aggregation and modal agent logic are of different kinds,
as they were initially developed for different purposes.
      </p>
      <p>Our research question thus breaks down into the following sub-questions:
1. How to aggregate individual (epistemic and motivational) attitudes?
2. What are the desirable properties for this aggregation?</p>
      <p>A good methodology for collectively choosing and reconsidering goals is also a
methodology that is dynamic enough to allow for the group to change its epistemic
and motivational attitudes in light of new information. Having such a methodology
increases the autonomy and reactivity of groups. Hence we need to answer the following
sub-question as well:
3. Which commitment and reconsideration strategies should be available for groups?</p>
      <p>We thus focus on finding the following solutions:
Formal framework. We extend a multi-agent modal language to be able to represent
judgment aggregation in it.</p>
      <p>Choice of aggregation. Judgment aggregation is an abstract framework that allows for
various desirable properties to be specified for the aggregation procedure. The task is
to determine which aggregation properties are necessary and desirable for aggregating
individual beliefs and goals.</p>
      <p>Commitment and reconsideration strategies. Within our formal framework, we define
when to de-commit from intentions and how to change them.</p>
      <p>Multiple goals. Since a group can have more than one goal, we need to model the effect
that the commitment to (and reconsideration of) one goal has over the commitment to
(and reconsideration of) other goals.</p>
      <p>We make the following assumptions. The group has a fixed membership – agents
do not join or depart from the group. The group has a set of candidate group goals
and an order of priory over these goals. The decision rules for each candidate goal are
available to the group. How the decision rules are learned is outside of the scope of this
paper. Here we do not consider how plans are generated, executed and revised once the
group goals are selected or reconsidered, nor we consider whether the group goals are
executed via individual or joint actions.</p>
      <p>The layout of the paper is as follows. In Section 2 we introduce judgment
aggregation. In Section 3 we extend a multi-modal agent formalism to capture the aggregation
of individual attitudes. Sections 4 and 5 respectively study the commitment and
reconsideration strategies. Related work, conclusions and outlines for future work are in
Section 6.
2</p>
      <p>From individual attitudes to group goals
Let us consider again the example of the robot cleaning crew from Section 1.
Example 1. Let C w 1, w2, w3 be a crew of cleaning ro bots. We denote the group
goal to clean the meeting room with g1, and the reasons to adopt this goal with: there
are no people in the room (p1), the room is dirty (p2), the garbage bin in it is full (p3).
The individual beliefs of the robots on whether g1 should be the group goal are justified
by individual beliefs on p1, p2, p3 using the decision rule p 1 p 2 p 3 g 1.</p>
      <p>
        A group of agents could collectively decide to adopt or reject a goal by voting.
However, the goals of the agents are not independent from their beliefs [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], which we
express using decision rules. When the decision is whether to adopt a goal or not, we
also need to explain why this goal should (not) be adopted. Having reasons for (not)
adopting a goal enables the agents to re-considerer this goal in light of new information.
Judgment aggregation deals with the problem of reaching decisions for a set of logically
dependent issues, by aggregating individual opinions on these issues.
2.1
      </p>
    </sec>
    <sec id="sec-3">
      <title>Judgment aggregation preliminaries A general overview of judgment aggregation is given in [17]. Here we present the terminology and definitions of judgment aggregation we use in our framework.</title>
      <p>Consider a logic L with entailment operator . In a judgment aggregati on
framework, an agenda A L is the pre-defined set of issues, on which every agent casts
her judgments. E.g., the agenda of Example 1 is A p 1, p2, p3, g1. Consider a
valuation function v such that a judgment “yes” on issue a is a valuation va 1, while
a judgment “no” on the same issue is a valuation va 0. A profile is the set of a ll
judgments assigned, on the agenda issues, by the decision-making agents.
Definition 1 (Profile). Let N 1, 2, . . . be a set of agent names, A L an agenda
and A A a a A. A profile π is the set of judgments fo r the agenda items,
submitted by the agents in the group: π N A. We define two operators:
The judgment set for agent i is π i a i, a π.</p>
      <p>The set of all the agents who judged “yes” to a A is π a i i, a π.</p>
      <p>
        Definition 1 extends the definition of a profile given [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] with the operators
and . We introduce these oper ators to ease the explanation of the various aggregation
properties we present later on. To get a better intuitive grasp on these operators, the
reader should envision the profile as a two-dimensional object with the agenda items
identifying the columns and the agents identifying the rows:
π
π is a possible profile for Example 1. We identify π w 2 as the row labeled w2 and
π p 2 as the 1 entries in the column labeled p2, which identify the agents who casted
judgement “yes” on p2.
      </p>
      <p>In judgment aggregation, the judgments over A, both individual and aggregates,
are constrained by decision rules R L. The set R contains only formulas r such
that all the non-logical symbols of A occur in r R. For instance, in Exampl e 1, the
decision rule is: p 1 p 2 p 3 g 1 and all agents respect it. In general, each agent
could follow a different decision rule and yet another decision rule can be imposed
for the group. In judgment aggregation, the agents are allowed to submit only those
judgment sets which are consistent with r R. Often, the agents are als o required to
cast judgments on all the agenda issues. We construct Definition 2 to formalize what it
means for a judgment set to be admissible.</p>
      <p>Definition 2 (Admissible profiles). A judgment set π i is complete if and only if for
every a A, either i, a π or i, a π. A judgment set π i is consistent with
r R if and only if r π i . A judgment set π i is admissible if it is
consistent with the given r R and complete for A. The set of all admissible judgment
sets for a given A and r is denoted by W.</p>
      <p>A profile π is admissible if π i is admissible for all i N . The set of all admissibl e
profiles for agents N is denoted by Π.</p>
      <p>We denote conr, ϕ 1 if ϕ is consistent with r and conr, ϕ 0 otherwise. For
the profile in Figure 1, conp 1 p 2 p 3 g 1, π i 1 for every i C.</p>
      <p>
        We now present the definition of a judgment aggregation function, as given in [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
Definition 3 (Judgment aggregation function). Given an agenda A and agents N , a
judgment aggregation (JA) function is f : 2N A 2 A.
      </p>
      <p>We refer to f π as the collective judgmen t set for N . We denote the result of f π with
when the JA function pr oduces an inadmissible judgment set. If f π then we
call f π a decision and denote it b y Dπ. Figure 1 illustrates an example of a judgment
aggregation function and profile, for which the collective judgment set is because
conp 1 p 2 p 3 g 1, p 1, p2, p3, g 1 0.</p>
      <p>The JA function is a useful abstraction, because many properties of judgment
aggregation can be defined in terms of it. It then can be studied which properties can be
accepted together (avoiding impossibility results). Given a JA function f , we describe
the most common properties in the JA literature.</p>
      <p>Universal domain. f satisfies universal domain if and only if its domain is Π.
Anonymity. Given a profile π Π, letπ π 1, . . . , π n, be the multiset of
all the individual judgment sets in π. Two profiles π, π Π are permutations of each
other if and only ifπ π . f satisfies anonymity if and only if f π f π for all
permutation π and π .</p>
      <p>Unanimity on a A. f satisfies unanimity on a A if and only if for every profile
π Π it holds: if for all i N , i, a π, then a f π.</p>
      <p>Collective rationality. f satisfies collective rationality if and only if for all π Π,
conr, f π 1 for a given r R, and either a f π or a f π for every a A.
Constant. f is constant when there exists ϕ 2 A such that for every π Π, f π ϕ.
Independence. Let Φ π a a A for every π Π. Let f 1, . . . , fm be
functions defined as fj : A Φ 0, 1. The JA function f satisfies independence if and
only if for all π Π, there exist functions f i such that for all ϕ f π it holds that
ϕ a a A, f j a, π a 1 a a A, f j a, π a 0.</p>
      <p>The best known example of fj : A Φ 0, 1 is the simple majority v oting
fm which counts how many agents expressed judgment “yes” on agenda item a. The
function fm returns a if that number of agents is greater than n2 and a otherwise.</p>
    </sec>
    <sec id="sec-4">
      <title>The issue-wise majority function fmaj is defined as</title>
      <p>fmaj
ϕ f mπ a a A if ϕ is complete and consisten
otherwise
t</p>
      <p>
        The JA function fmaj satisfies universal domain, anonymity, unanimity,
completeness, and independence but it does not satisfy collective rationality, as it can be seen on
Figure 1. All judgement aggregation functions which satisfy universal domain, anonymity,
independence and collective rationality are constant [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. The most debated [
        <xref ref-type="bibr" rid="ref17 ref3">3, 17</xref>
        ] is
independence. The reason why it is convenient to have independence is because it is a
necessary condition to guarantee the non-manipulability of f [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. An aggregation
function is non-manipulable, if no agent can obtain his sincere judgment set ϕ selected as
the collective judgment set by submitting another judgment set ϕ .
      </p>
      <p>
        Two aggregation procedures that violate independence but guarantee universal
domain, anonymity and collective rationality have been proposed in the literature:
premisebased and conclusion-based procedures. A distinguishing feature of judgment
aggregation with respect to social choice theory is the distinction between premises and
conclusions. The agenda is a union of two disjoint sets: the premise set (Ap), and the
acnodncAlucsiong set2(.AWc)e. AgiveAa gpenAeracl,dAefipnAitiocn o.nInwEhxeanmaplJeA1,fuAnctionp isp pre1m,pis2e,-p3or
conclusion-based in Definition 4. The literature on judgment aggregation [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] defines
premise- and conclusion-based procedures only in terms of issue-wise majority.
Definition 4. Let πp i, a i, a π, a A p, π c i, a i, a π, a A c
and let the premise and conclusion aggregation functions be defined as
f p : Π 2 Ap and f c : Π 2 Ac correspondingly. The JA function f is
premise-based if and only if there exists a f p such that f pπ p f π and
conclusionbased if and only if there exists a f c such that f cπ c f π, for all π Π.
      </p>
      <p>An example of premise and conclusion-based aggregations is given in Figure 1.
Intuitively, a JA function is premise-based when the collective judgment set on the
premises is obtained as a result of aggregating only the premise profile πp. The
decisions f π are those ϕ W for which f pπ p ϕ (or alternatively f cπ c ϕ for
conclusion-based aggregations). However, it is possible that, depending on the decision
rule, there are more than one ϕ W that satisfy the conditio n f pπ p ϕ (or
alternatively f cπ c ϕ). This is what happens fo r the conclusion-based procedure we
illustrate in Figure 1.
2.2</p>
    </sec>
    <sec id="sec-5">
      <title>Judgment aggregation for BDI agents</title>
      <p>In this section first we set the problem of finding collective decisions for group goals in
the context of judgment aggregation. We then argue that the aggregation function used
for this problem should satisfy: collective rationality, universal domain, unanimity and
select a unique ϕ W. For a democratic group , in which all agents have equal say on
what the group attitudes should be, anonymity should be satisfied. For a group in which
the agents have different levels of expertise, anonymity can be omitted.</p>
      <p>We use Gg to denote that “g is a group goal”. A set of the candidate group goals is
the set G Gg g L which contains all the g oals which the group considers to
adopt. For each goal Gg G, the group has at its dispo sal decision rules Rg R and an
agenda Ag composed of the goal g (conclusion), and all the reasons (premises) relevant
for g, which were given by the decision rule. Each agent submits her judgments on the
agenda thus generating a profile πg, such that conR g, πg i 1 for all i N .</p>
      <p>The decision rules contain all the constraints which the individual and collective
judgment sets should satisfy. These constraints contain three types of information:
just), rules
rules describing how the goal depends on the reasons (justification rules Rg
describing the constraints of the world inhabited by the agents (domain knowledge
RgDK ) and rules that describe how g interacts with other candidate goals of the group
coord). Hence, the decision rule for a group goal g is Rg
(coordination rules Rg
Rjgust R gDK R cgoord.
just
Example 2 (Example 1 revisited). Consider the cleaning crew from Example 1. Rg1
is p 1 p 2 p 3 Gg 1 and Ag1 p 1, p2, p3, Gg1. Suppose that the crew h as
the following candidate group goals as well: place the furniture in its designated
location (g2) and collect recyclables from garbage bin (g3). The agendas are Ag2
p 4, p5, p6, p7, Gg2, A g3 p 3, p8, p9, Gg3. The justification rules are Rjgu2st
p 4 p 5 p 6 p 7 Gg 2 and Rjgu3st p 8 p 9 p 3 Gg 3. The formulas
p4 p 9 are: the furniture is out of place (p4), the designated location for the furniture
is empty (p5), the furniture has wheels (p6) , the furniture has handles (p7), the agents
can get revenue for recyclables (p8), there is a container for the recyclables (p9).
An example of a domain knowledge could be RgD2K p 4 p 5, since it can not
happen that the designated location for the furniture is empty while the furniture is not
out of place. Group goal Gg3 can be pursued at the same time as Gg1, however, Gg2
can only be pursued alone. Thus the coordination rule for all three goals is
Rcgo1ord R cgo2ord R cgo3ord Gg 2 Gg 1 Gg 3 Gg 2.</p>
      <p>We want the justifications for a goal to explain when a goal should be adopted/refuted.
Having collective justifications for a goal enables the agents to, when the world changes,
consider adopting a goal that has been rejected previously. To this end, we take into
consideration justification rules which are of form Gg Γ , where Γ L such that all the
non-logical symbols of Γ occur in Agp and Gg A cg.</p>
      <p>We now continue to discuss the desirable properties for the aggregation of
individual beliefs and goals. We need a JA function that satisfies universal domain to be able
to aggregate all admissible profiles. We can use only JA functions that satisfy
collective rationality. If the collective set is not complete, for example, if it contains only a
collective judgment on the goal, then we do not know why the goal was (not) adopted
and consequently we would not know when to revise it. For example, the cleaning crew
decides for the goal g3 (to collect recyclables), without having the reasons like p9 (a
container where to put them). If the world changes and p 9 holds, the robots will
continue to collect recyclables.</p>
      <p>The aggregation of all admissible profiles should produce a set consistent with the
decision rule because otherwise we would not be generating the group goals and
justifications for them. For the same reason we need an aggregation method that selects a
unique ϕ W.</p>
      <p>To guarantee that conR g, f π g 1, we need to choose betw een
conclusionbased and premise-based procedures. At first glance, the premise-based procedure seems
an obvious choice since it will produce complete collective judgment sets under our
decision rules. However, upon closer inspection, this procedure has notable drawbacks.</p>
      <p>As it is observable from the profile in Figure 1; a premise-based procedure may lead
the group to adopt a conclusion that the majority (or even the unanimity) of the group
does not endorse. In our case, the conclusion is the goal and a premise-based
aggregation may establish a group goal which none of the agents is interested in pursuing. To
avoid this we need to aggregate using a conclusion-based procedure. In particular we
want a conclusion-based procedure that has the property of unanimity on Gg.</p>
      <p>
        Given that our decision rules are of the form g Γ , there exist profiles for w hich
a conclusion-based procedure will not produce complete collective set of judgments.
However, the conclusion-based aggregation can be supplemented with an additional
procedure that completes the produced set of judgments when necessary. Such
aggregation procedure is the complete conclusion-based procedure (CCBP) developed in [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ].
This CCBP satisfies universal domain and is collectively rational. However, it does
not always produce a unique decision. CCBP produces a unique collective judgment
for the goal, but it can generate more than one set of justifications for it. This is an
undesirable, but not an unmanageable property. To deal with ties, as it is the practice
with voting procedures, the group can either determine a default set of justifications for
adopting/rejecting each candidate goal, or it can appoint one member of the group as
tiebreaker. Tie-breaking problems in judgment aggregation are the focus of our ongoing
research.
      </p>
      <p>
        The CCBP from [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] also satisfies anonimity. Whether this is a desirable property
for a group of artificial agents depends entirely on whether the group is democratic or
the opinions of some agents are more important. CCBP can be adjusted to take into
consideration different weights on different agents’ judgment sets.
3
      </p>
      <p>Formal framework for modeling group attitudes
In this section we introduce the language of modal multi-agent logic in which we
represent individual and collective mental attitudes. We then combine the methodology of
judgment aggregation with this representation language and show how collective
attitudes are generated. To model commitment to a group goal and reconsideration of a
group goal we use temporal logic.
3.1</p>
    </sec>
    <sec id="sec-6">
      <title>Modal multi-agent logic</title>
      <p>Just like modal agent logic is concerned mainly with the relation between the individual
goals and beliefs over time, modal multi-agent logic is concerned with the relation
between group goals and beliefs over time. We use modal multi-agent logic to represent:
the agenda, individual judgments, collective judgments and new information that may
prompt goal revision. In line with judgment aggregation proper, we do not use the
formal language to represent the judgment aggregation function, but only the arguments
of this function and the results from it. We assume that there is a service, available to
the agents, that elicits the individual judgments, performs the aggregation and makes
the aggregation results available, to the members and for plan-generation.</p>
      <p>Agenda issues in judgment aggregation are usually represented by propositional
formulas. This is not a viable option in our case. First, we want to represent the
difference between a goal and the supporting reasons by means of representing the
distinction between conclusions and premises explicitly in the logic. Second, the logic should
represent the distinction between individual and collective judgments. We distinguish
conclusions from premises by using a single K modal operator Gg for representing that
“g is a group goal”.</p>
      <p>
        The obvious choice for modeling the judgment “true” on agenda issue a, of an agent
i, is the modal operator belief Bia (correspondingly Bia for a judgment “false” o n
a). However, we find that beliefs are ill suited for modeling collective judgments of
agents. While a belief Bia is an exclusively private mental state, judgments are public
and contributed for the decision-making purposes of the group. A judgment is thus
closer to a public commitment than to a private belief. Hence, we model judgments
by using the modal operator of acceptance AS a [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. AS a denotes: agents in S accept
a. The operator AS a allows us to represent both individual judgments, S i, for
i N and collective judgment s with S N . We define the group ac ceptance A a
to be the result of applying judgment aggregation to the individual acceptances. NWe
present the formal definitions of profile and judgment aggregation function in our logic
in Definition 6. The modal operator AS we use is inspired by the modal operator of the
acceptance logic of [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. The details on the relation between acceptance logic and our
acceptance operator are given in the Appendix.
      </p>
      <p>We represent the new information that becomes available to the agents with a normal
modal K operator E. Ea denotes: “it is observed that a is true”.</p>
      <p>Lastly, to model how the group attitudes evolve with reconsideration we need a
temporal logic. By using LT L we do not need to distinguish between path formulas
and state formulas, but we are able to quantify over traces. Just as in BDILT L, where
Ba denotes that a is believed to be necess ary, we use Ea to denote that a is
observed to be impossible.</p>
      <p>We now give the syntax of our modal multi-agent logic AGELT L.</p>
      <p>Definition 5 (Syntax). Let Agt be a non-empty set of agents, with S Agt, and L P be
a set of atomic propositions. The admissible formulae of AGELT Lare formulae ψ0, ψ1
and ψ2 of languages Lprop, LG and LAELT L correspondingly:
ψ0 :: p ψ 0 ψ 0 ψ 0
ψ1 :: ψ 0 Gψ 0
ψ2 :: ψ 0 A S ψ1 Eψ 2 Xψ2 ψ 2Uψ2
where p ranges over LP and S over 2Agt. Moreover, ♦φ Uφ, φ ♦φ, and
φRφ φUφ .</p>
      <p>We can now adjust the definition for a judgment aggregation function. We represent
individual judgments with Ai a with intuitive reading “agent i judges a as true” and
Ai a with reading “agent i judges a as false”.</p>
      <p>Definition 6 (JA in AGELT L ). Let N 1, 2, . . . be a set of agent name s and
G L GL prop a set of candidate goals.</p>
      <p>An agenda Ag L G for goal Gg G is a set of formulas such that Ag Agp Acg,
with Agp A gp a a A gp, A gp L prop and Acg Gg, Gg.</p>
      <p>A profile of judgments is the set π A i a i N , a A g.
π i a A i a π is the judgment set of age nt i.
π a i A i a π is the set of all the agent s that accepted a.</p>
      <p>Given a set of decision rules Rg L prop, a profile is acceptable if and only if, for all
i N and all a A, conR g, π i 1 and either A i a or Ai a. The set of all
acceptable profiles for N and Ag, given Rg is Π.</p>
      <p>The decision for a profile π, Dπ A a a f a π, f a : Π 2</p>
      <p>
        AGELT Lhas Kripke semantics. As in Schild [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ], a Kripke structure is defined as a
tuple M W, R , G , E , A, L. W is a set of possible situat ions, and R is the temporal
relation over situations R W W . G is the goal relation over s ituations G W W ,
while E is the observation relation over situations E W W . Let Δ 2 N Inst.
A : Δ W W maps every S Δ to a relation A S between possible situations. L
is a truth assignment to the primitive propositions of LP for each situation w W , i.e.,
Lw : P rop true, f alse.
      </p>
      <p>
        Temporal formulas are validated in the standard manner [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. Normal modal
formulas Gψ and Eψ have standard semantics, see for example [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Acceptance formulas
AS ψ are validated according to the semantics of acceptance logic presented in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. The
axiomatization of AGELT Lis given in the Appendix. Note that AGELT Lis a fusion of
the decidable logics: LT L , acceptance logics and two K modal logics.
      </p>
    </sec>
    <sec id="sec-7">
      <title>3.2 Generation of group goals</title>
      <p>The mental state of the group is determined by the mental states of the members and the
choice of judgment aggregation function. We represent the mental state of the group by
a set χ of AGELT L formulas. The set χ contains: the set of all candidate goals for the
group G L GL prop and, for each Gg G, the corresponding decis ion rules Rg, as
well as the individual and collective acceptances made in the group regarding agenda
Ag. The set χ is common knowledge for the group members. An agent uses χ when it
acts as a group member and its own beliefs and goals when it acts as an individual.</p>
      <p>To deal with multiple, possibly mutually inconsistent goals, the group has a priority
order x over the group goals G χ. To avoid overburdening the language with a x
operator, we incorporate the priority order within the decision rules Rjguist Γ i Gg i.
We want the decision rules to capture that if Ggi is not consistent (according to the
coordination rules) with some higher priority goals Gg1, . . . , Ggm, then the group can
accept Ggi if and only if none of Gg1, . . . , Ggm is accepted. Hence, we replace the
jGusgtjifiGca,tGiogn rjulexRGjguigsitandχ GwgitihGRg gpjijuRst cgΓoiordi . jmA N Gg j Gg i, where
Example 3. Consider the goals and rules of the robot crew C from Example 2.
Assume the crew has been given the priority order Gg1 χ Gg2 χ Gg3. χ contains:
G Gg 1, Gg2, Gg3, one background knowle dge rule, one coordination rule, three
justification rules, out of which two are new priority modified rules:
G, p 4 p 5, Gg 2 Gg 1 Gg 3 Gg 2, Gg1 p 1 p 2 p 3,
Gg2 p 4 p 5 p 6 p 7 A C Gg 1, Gg 3 p 8 p 9 p 3 A C Gg 2.</p>
      <p>The agents give their judgments on one agenda after another starting with the agenda
for the highest priority candidate goal. Once the profile π and the decision Dπ for a
goal g are obtained, they are added to χ. To avoid the situation in which the group casts
judgments on an issue that has already been decided, we need to remove decided issues
from Ag before eliciting the profile for this agenda.</p>
      <p>The group goals are generated by executing GenerateGoals(χ, N ).
function GenerateGoals(χ, S):
for {eaBch: GagAi G s.tN.[aGgχ aj AG: (Gg j Gag χi)A(A N Ggig;j χ or A C:χGg j χ)]
comment // B is the set of already cNollectively accpted issues from Agi
elicit requests the agents to submit complete judgment sets for πgi χ. We require
that elicit is such that for all returned π it holds conχ, π i 1 for all i N and
that conχ, f a π 1. When a higher priority goal Ggi is accepted by the group, a
lower priority incompatible goal Ggj cannot be adopted regardless of the judgments on
the issues in Agj . Nevertheless, Although elicit will provide individual judgments for
the agenda Agj . If the acceptance of Ggi is reconsidered, we can obtain a new decision
on Ggj because the profile for Ggj is available.</p>
      <p>Ag1 , Ag1
p 8, p9, Gg3.</p>
      <p>Example 4. Consider the χ for robots given in Example 3. The following calls to elicit
are made in the given order. First, πg1 elicitN , A g1 , χ with the GenerateGoalsχ
χ χ πg1 f a πg1 . Second, πg2 elicitN , A g2 , χ , with GenerateGoalsχ
χ χ πg2 f a πg 2. Last, πg3 elicitN , A g3 , χ , with GenerateGoalsχ
χ χ πg3 f a πg3 . Since there is no overla pping between agendas Ag2 and</p>
      <p>A g1 and Ag2 A g2 . However, since Ag2 A g3 p 3, then Ag3
Proposition 1. GenerateGoals terminates if and only if elicit terminates and does
not violate the candidate goal preference order.</p>
    </sec>
    <sec id="sec-8">
      <title>The proof is straightforward.</title>
      <p>4</p>
      <p>
        Commitment strategies
The group can choose to reconsider the decision (acceptance or rejection) on a group
goal in presence of new information. Whether the group chooses to reconsider depends
on how committed it is to bring the goal about. According to Cohen and Levesque
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], an agent intends g if it has chosen to pursue goal g and it committed itself to
making g happen. Following the same line of reasoning, we define group intention
to be I g A Gg and read it as “the agents N intend g”. We defined collective
      </p>
      <p>N N
acceptance as resulting from judgment aggregation, which is a social choice method.
Thus, in our framework, group intention is social choice with commitment. The level of
commitment of a group to a collective acceptance depends on the choice of commitment
strategy.</p>
      <p>
        These are the three main commitment strategies (introduced by Rao and Georgeff [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]):
Blind commitment: Iig I igUBig
Single-minded commitment: Iig I igUB ig B ig
Open-minded commitment: Iig I igUB ig G ig
      </p>
      <p>These commitment strategies only consider the relation between beliefs regarding
g and Gg. Instead, a commitment to a goal can now be reconsidered upon new
information on either one of the agenda issues in Ag and also upon new information on a
higher priority goal.</p>
      <p>The strength of our framework is exhibited in its ability to describe the groups’
commitment not only to its decision to adopt a goal, but also to its decision to reject
aangdoaAl. Ngamje.lyC, oifmtmheitamgeennttstoderecjiedcetdgI NallgoiwasnfdoArgNtgo bje arreecocnosmidmeritetdedantod beovtehntIuNalglyi
adopteNd if the state of the world changes.</p>
      <p>Let N be a set of agents with a set of candidate goals G. Let Ggi, Ggj G have
agendas Agi , Agj . We use p A gpi and qi A cgi , qj A cgj . The profiles and decisions
are πgi and f a πgi ; Gg j Gg i, and Ggj cannot be pursued at the same time as Ggi.</p>
      <p>We use the formulas α 1 α 7 to refine the blind, single- minded and open-minded
commitment. Instead of the “until”, we use the temporal operator release: ψ R φ
ψ U φ, meaning that φ has to be true until and in cluding the point where ψ first
becomes true; if ψ never becomes true, φ must remain true forever. Unlike the until
operator, the release operator does not guarantee that right hand-side formula will ever
become true, which in our case translates to the fact that an agent could be forever
committed to a goal.
(α1) Egi R I g
(α2) R A NGgi i
(α3) Eg
(α4) A
(α5) A</p>
      <p>Ni Eg i R A N qi</p>
      <p>NN qp EjpRRAAN qi N qi
Blind commitment: α1 α 2.</p>
      <p>Only the observation that the goal is achieved (Egi) can release the intention to achieve
the goal I gi. If the goal is never achieved, the group will always be committed to it.</p>
      <p>N
If a goal is not accepted, then the agents will not reconsider accepting it.</p>
      <p>Single-minded commitment: α3.</p>
      <p>Only new information on the goal (either that the goal is achieved or had become
impossible) can release the decision of the group to adopt /reject the goal. Hence, new
information is only regarded if it concerns the conclusion, while information on the
remaining agenda items is ignored.</p>
      <p>Extended single-minded commitment: α3 α 4.</p>
      <p>Not only new information on gi, but also the collective acceptance to adopt a more
important incompatible goal gj can release the intention of the group to achieve gi.
Similarly, if gi is not accepted, the non-acceptance can be revised, not only if gj is
observed to be impossible or achieved, but also when the commitment to pursue gj is
dropped (for whatever reason).
Open-minded commitment: α3 α 5.</p>
      <p>A group will maintain its collective acceptances to adopt/reject a goal as long as the new
information regarding all collectively accepted agenda items is consistent with f a πgi .
Extended open-minded commitment: α3 α 4 α 5.</p>
      <p>Extending on the single-minded commitment, a change in intention to pursue a higher
priority goal Ggj can also release the acceptance of the group on Ggi.</p>
      <p>Once an intention is dropped, a group may need to reconsider its collective
acceptances. This may cause for the dropped goal to be re-affirmed, but a reconsideration
process will be invoked nevertheless.
5</p>
      <p>Reconsideration strategies
In Section 3.2 we defined the mental state of the group χ. We can now define what it
means for a group to be coherent.</p>
      <p>N</p>
      <p>N
Definition 7 (Group coherence). Given a Kripke structure M and situations s W ,
a group of N agents is coherent if the following conditions are met:
(ρ1): M A S a A S a for any S N and any a A g.
(ρ2): If M, s χ then χ .
(ρ3): M, s G g for all Gg G.
(ρ4): Let Gg G and G GGg, then M G Eg XGg.
(ρ5): Let p A gp and q Gg, Gg. Ep Ep R A q XA p
The first condition ensures that no contradictory judgments are given. The second
condition ensures that the mental state of the group is logically consistent in all situations.
The third and fourth conditions ensure that impossible goals cannot be part of the set of
candidate goals and if g is observed to be impossible in situation s, then it will be
removed from G in the next situation. ρ5 enforces the acceptance of the new information
on the group level, when the commitment strategy so allows – after a is observed and
that lead the group to de-commit from g, the group necessarily accepts a.</p>
      <p>A coherent group accepts the observed new information on a premise. This may
cause the collective acceptances to be inconsistent with the justification rules.
Consequently, the decisions and/or the profiles in χ need to be changed in to ensure that ρ1
and ρ2 are satisfied. If, however g or g is observed, the group r econsiders χ by
removing Gg from G. In this case, the decisions and profiles are not changed.</p>
      <p>For simplicity, at present we work with a world in which the agents’ knowledge can
only increase, namely the observed information is not a fluent. A few more conditions
need to be added to the definition of group coherence, for our model to be able to be
applicable to fluents. E.g., we need to define which observation is accepted when two
subsequent contradictory observations happen.</p>
      <p>For the group to be coherent at all situations, the acceptances regarding the group
goals need to be reconsidered after de-commitment. Let Dg χ contain the group
acceptances for a goal g, while πg χ contain the profile for g. There are two basic
ways in which a collective judgment set can be reconsidered. The first way is to elicit a
new profile for g and apply judgment aggregation to it to obtain the reconsidered Dg .
The second is to reconsider only Dg without re-eliciting individual judgments. The first
approach requires communication among agents. The second approach can be done
by each agent reconsidering χ by herself. We identify three reconsideration strategies
available to the agents. The strategies are ordered from the least to the most demanding
in terms of agent communication.</p>
      <p>Decision reconsideration (D-r). Assume that Ep, p A gp, q Gg, Gg and the
group de-commited from A q. The reconsidered decision Dg is such that p is
acrcuelpetse,dn, aim.ee.,lyAcNonpR Dgpjusgt,, DanNgd tNhe e1n.tiIrfethdeecDis-irosnpeiscicfioenssaisnteunntiqwuiethDthe jugst,ififocrataionyn
observed information and any Dg, then χ can be reconsidered without any
communication among the agents. Given the form of Rgpjust (see Section 3.2), this will always be
the case.</p>
      <p>However D-r is not always an option when the de-commitment occurred due to a
change in collective acceptance of a higher priority goal g . Let q Gg , Gg .
Let the new acceptance be AN q . D-r is possible if and only if Dg D g and
caoccneRptagpnjcuesto,fDqg oAr q Nisq neve1r.inRethcealdl etchiastioAn for Nπgq. was not in Ag and as such the
Partial reconsideration of the profile (Partial π-r). Assume that Ea, a A g, Gg
G. Not only the group, but also the individual agents need to accept a. The Partial
π-r asks for new individual judgments be elicited. This is done to ensure the logical
consistency of the individual judgment sets with the observations. New judgments are
only elicited from the agents i which Ai a.</p>
      <p>Let W N be the subset of agents i s.t. A i a χ. Agents i are s.t. A i a χ
when the observation is Ea. Let πgW πg be the set of all acceptances made by
agents in W . We construct χ χ πgW . The new profile and decision are obtained by
executing GenerateGoals (χ , W ).</p>
      <p>Example 5. Consider Example 4. For πg1 , πg2 and πg3 of the robot crew C, the
decisions Dg1 A C p1, AC p 2, AC p3, AC Gg1, D g2 A C p4, AC p5, AC p6, AC p7, AC Gg 2
and Dg3 A C p8, AC p9, AC Gg3 are made. Assume the gro up de-commits on Gg1because
of Ep 2. If the group is committed to Gg3, the commitment on Gg3 will not allow for
A p3 to be modified when reconsidering Gg1. Since A p3 exists in χ , p3 will be</p>
      <p>N N
excluded from the (new) agenda for g1, although it was originally in it. elicit calls only
on the agents in W to complete πg1 χ with their judgment sets.</p>
      <p>Full profile reconsideration (π-r). The full profile reconsideration is the same with
the partial reconsiderations in all respects except one – now W N . Namely, within
the full profile revision strategy, each agent is asked to revise his judgment set by
accepting the new information, regardless whether he had already accepted the information or
not.</p>
    </sec>
    <sec id="sec-9">
      <title>5.1 Combining revision and commitment strategies</title>
      <p>
        Unlike the Rao and Georgeff commitment strategies [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], in our framework the
commitment strategies are not axioms of the logic. We require that the commitment strategy
is valid in all the models of the group and not in all the models of AGELT L. This allows
the group to define different commitment strategies and different revision strategies for
different goals. It might even choose to revise differently depending on which
information triggered the revision. Choosing different revision strategies for each goal, or each
type of new information, should not undermine the coherence of the group record χ.
The conditions of group coherence of the group ensures that after every reconsideration
χ must remain consistent. However, some combinations of commitment strategies can
lead to incoherence of χ.
      </p>
      <p>Example 6. Consider the profiles and decisions in Example 5 . Assume that initially the
group chose open-minded commitment for IC g1 and blind commitment for IC g3, with
goal open-minded commitment for AC Gg 2. If Eg1 and thus IC g1 is dropped, then
the extended open-minded commitment would allow AC Gg 2 to be reconsidered and
eventually IC g2 established. However, since the group is blindly committed to IC g3,
this change will not cause reconsideration and as a result both IC g2 and IC g3 will be in
χ thus making χ incoherent.</p>
      <p>Problems arise when subR gpijust subR gpjjust , where subR gpjust denotes
the set of atomic sub-formulas of g (Ggi, Ggj G). Proposition 2 summari zes under
which conditions these problems are avoided.</p>
      <p>Proposition 2. Let α and α be the commitment strategies selected for gi and gj
correspondingly. χ α α (in all situations):
a) if φ subR gpijust subR gpjjust and p A gi A gj , then α5 is either in both α
and α or in none;
b) if Ggi is more important than Ggj and Gj and Gi cannot be accepted at the same
time, then α4 α .</p>
      <p>Proof. The proof is straightforward. Namely, if the change on acceptance of Ggi causes
the decision on Ggj to induce group incoherence, we are able to de-comit from Ggj .
If we were not able to de-comit on Ggj group coherence is blocked. If the change on
collective acceptance on Ggi is caused by an observation on a premise p A gi A gj
then condition a) ensures that the commitment to the collective acceptance regarding
Ggj does not block group coherence. If the change on collective acceptance on Ggj is
caused by a change in commitment to a higher priority goal the condition b) ensures
that a commitment regarding Ggj does not block group coherence. Condition b) allows
only “goal sensitive” commitments to be selected for lower level goals.
6</p>
      <p>
        Conclusions
We constructed a group decision-making framework by combining judgment
aggregation and multi-agent modal logic. We identified the desirable judgment
aggregation properties for aggregation in collaborative groups. Our multi-agent modal logic
AGELT L is an extension of BDILT L with modal operators for representing individual
and collective acceptances and observations of new information. We extend the
commitment strategies of Rao and Georgeff [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] to increase the reactivity of the group to
new information. Having a group goal Gg in our framework does not imply that the
members individually have the goal Gg and groups can have different levels of
commitment to different goals.
      </p>
      <p>
        Our framework is intended for groups that engage in joint activity. Our framework
is applicable when it is not possible to assume that the agents persuade each other on
a single position and goal, but it is necessary anyway that the group presents itself as
a single whole from the point of view of beliefs and goals, and above all as a rational
entity that has goals justified by the beliefs it holds, and it is able to revise these goals
under the light of new information. This requirement was held by Tuomela [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] and
adopted in agent theory by Boella and van der Torre [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] and Lorini [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. There are
many situations where the proposal of the paper can be applied. For example in an
opensource project, where several people have to discuss online to agree on which is
their position on issues (e.g. which algorithm is better for a certain task) and which is
their goal (e.g. delivering a new realize on which date).
      </p>
      <p>
        Work on collaborative group activities [
        <xref ref-type="bibr" rid="ref12 ref16 ref9">9, 12, 16</xref>
        ] and group decision-making
protocols [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] focus on how to define collective intentionality and how to distribute the
collective intentions over the agents. We define group intention to be the collective
acceptance of a group goal and focus on defining commitment strategies for the collective
acceptances. An advantage of our framework is its ability to allow groups to commit
to a decision to reject a goal, thus having the option to reconsider rejected goals.
Furthermore, we do not only show when to reconsider, but also how, by defining
reconsideration strategies. Table 1 summarizes our commitment and reconsideration strategies.
      </p>
      <p>
        Icard et al. [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] consider the joint revision of individual attitudes, with the revision
of beliefs triggering intention revision. However, they do not allow for the revision of
intentions to cause a revision of beliefs. We focus on joint reconsideration of group
attitudes and we allow for both the change in epistemic and in motivational attitudes can
be a cause for reconsideration.
      </p>
      <p>
        In our framework, the new information is simultaneously available to the entire
group. In the future we intend to explore the case when only some members of the group
observe the new information. The only assumptions we make regarding the connectivity
of the members is that they are able to communicate their acceptances and receive the
aggregation result. The problem of elicitation and communication complexity in voting
is a nontrivial one [
        <xref ref-type="bibr" rid="ref6 ref7">6, 7</xref>
        ] and we intend to study these properties within our framework.
      </p>
      <p>
        In the work we presented, we do not consider how individual acceptances are formed.
We can take that Biφ A i φ, but this need not be the case. We can define dishonest
agents as those for which Biφ A i φ does not hold. In this case, the agent might
declare Ai φ while it does not believe φ. The question is whether there are scenarios in
which incentives for doing so arise. Furthermore, given that the some reconsideration
strategies call for re-elicitation of judgments, can an agent have the incentive to
deliberately give judgments that would lead to sooner re-elicitation? We intend to devote more
attention to answering these questions as well as studying the manipulability properties
of our decision-making framework.
Appendix – Relations between AGELT L and acceptance logic and
axiomatization of AGELT L
Here we elaborate in more detail on the fusion logic AGELT L we use. The modal
operator AS φ we use is equivalent to the modal operator AS:xφ of the acceptance logic
of [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] with one syntactic and one semantic exception.
      </p>
      <p>The operator AS:xφ uses x ranging over a set of labels to describe the context under
which the acceptance is made. In our case the context is that of the group and since we
deal with only one group, we have no use of these labels. The context labels play no
role in the semantics of the acceptance logic formulas.</p>
      <p>On the semantic level, the axioms for AS φ are all the axioms of AS:xφ except two:
the axiom inclusion (Inc.) and the axiom unanimity (U n.). Dropping (U n.) and (Inc.)
does not affect the decidability of the logic of acceptance. (U n.) (not to be confused
with unanimity in judgment aggregation) states that if A N , A i:x φ.
In our case, it is the aggregation of individual acceptances Nth:axtφd,ettheremniines the collective
acceptance. Since we use the acceptances to model judgments, we do not want an axiom
that states that the individual judgments mirror the collective judgments. The agents use
the collective acceptance when functioning as a group and their private beliefs when
acting as individuals. In our framework we do not model the private mental states, but
only individual acceptances which are “declared” to all the agents in the group.</p>
      <p>(Inc.) states that if a the group C accepts ϕ, so will any subgroup B C. In our
case, the judgment aggregation over the profile containing only the judgment sets of
B can produce a different collective judgment set than the profile containing all the
judgment sets of C.</p>
      <p>The axiomatization of the AGELT Llogic is thus:
(ProTau) All principles of propositional calculus
(LTLTau) All axioms and derivation rules of LTL
(K-G) Gφ ψ Gφ Gψ
(K-E) Eφ ψ Eφ Eψ
(K-A) AS φ ψ A S φ A S ψ
(PAccess) AS φ A M AS φ if M S
(NAccess) A S φ A M A S φ if M S
(Mon) A S A M if M S
(MP) From φ and φ ψ infer ψ
(Nec-A) From φ infer A S φ
(Nec-G) From φ infer Gφ
(Nec-E) From φ infer Eφ</p>
      <p>Given M W, R , G , E , A, L and s W , the truth conditions for t
AGELT L(in a situation s) are:
he formulas of</p>
      <p>N
– M, s ;
– M, s p if and only if p Lp;
– M, s φ if and only if M, s φ;
– M, s φ ψ if and only if M, s φ and M, s ψ;
– M, s A φ if and only if M, s φ for all s, s</p>
      <p>A;</p>
      <p>A formula φ is true in an AGELT Lmodel M if and only if M, s φ for every
situation s W . The formula φ is valid (noted AGELT L ) if and only if φ is true in all
AGELT Lmodels. The formula φ is AGELT L-satisfiable if and only if the formula ϕ
is not AGELT Lvalid.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>G.</given-names>
            <surname>Boella</surname>
          </string-name>
          and
          <string-name>
            <surname>L. van der Torre.</surname>
          </string-name>
          <article-title>The ontological properties of social roles in multi-agent systems: Definitional dependence, powers and roles playing roles</article-title>
          .
          <source>Artificial Intelligence and Law Journal (AILaw)</source>
          ,
          <volume>15</volume>
          (
          <issue>3</issue>
          ):
          <fpage>201</fpage>
          -
          <lpage>221</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>C.</given-names>
            <surname>Castelfranchi</surname>
          </string-name>
          and
          <string-name>
            <given-names>F.</given-names>
            <surname>Paglieri</surname>
          </string-name>
          .
          <article-title>The role of beliefs in goal dynamics: Prolegomena to a constructive theory of intentions</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>155</volume>
          :
          <fpage>237</fpage>
          -
          <lpage>263</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>B.</given-names>
            <surname>Chapman</surname>
          </string-name>
          .
          <article-title>Rational aggregation</article-title>
          .
          <source>Politics, Philosophy and Economics</source>
          ,
          <volume>1</volume>
          (
          <issue>3</issue>
          ):
          <fpage>337</fpage>
          -
          <lpage>354</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>B. F.</given-names>
            <surname>Chellas</surname>
          </string-name>
          .
          <source>Modal Logic: An Introduction</source>
          . Cambridge University Press, Cambridge,
          <year>1980</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>P. R.</given-names>
            <surname>Cohen</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Levesque</surname>
          </string-name>
          .
          <article-title>Intention is choice with commitment</article-title>
          .
          <source>Artificial Intelligence</source>
          ,
          <volume>42</volume>
          (
          <issue>2-3</issue>
          ):
          <fpage>213</fpage>
          -
          <lpage>261</lpage>
          ,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>V.</given-names>
            <surname>Conitzer</surname>
          </string-name>
          and
          <string-name>
            <given-names>T.</given-names>
            <surname>Sandholm</surname>
          </string-name>
          .
          <article-title>Vote elicitation: Complexity and strategy-proofness</article-title>
          .
          <source>In AAAI/IAAI</source>
          , pages
          <fpage>392</fpage>
          -
          <lpage>397</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>V.</given-names>
            <surname>Conitzer</surname>
          </string-name>
          and
          <string-name>
            <given-names>T.</given-names>
            <surname>Sandholm</surname>
          </string-name>
          .
          <article-title>Communication complexity of common voting rules</article-title>
          .
          <source>In ACM Conference on Electronic Commerce</source>
          , pages
          <fpage>78</fpage>
          -
          <lpage>87</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>F.</given-names>
            <surname>Dietrich</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>List</surname>
          </string-name>
          .
          <article-title>Strategy-proof judgment aggregation</article-title>
          .
          <source>STICERD - Political Economy and Public Policy Paper Series</source>
          <volume>09</volume>
          ,
          <article-title>Suntory and Toyota International Centres for Economics and Related Disciplines</article-title>
          ,
          <string-name>
            <surname>LSE</surname>
          </string-name>
          ,
          <year>Aug 2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>B.</given-names>
            <surname>Dunin-Keplicz</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Verbrugge</surname>
          </string-name>
          .
          <article-title>Collective intentions</article-title>
          . Fundam. Inf.,
          <volume>51</volume>
          (
          <issue>3</issue>
          ):
          <fpage>271</fpage>
          -
          <lpage>295</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <given-names>B.</given-names>
            <surname>Grosz</surname>
          </string-name>
          and
          <string-name>
            <surname>L. Hunsberger.</surname>
          </string-name>
          <article-title>The dynamics of intention in collaborative activity</article-title>
          .
          <source>Cognitive Systems Research</source>
          ,
          <volume>7</volume>
          (
          <issue>2</issue>
          -3):
          <fpage>259</fpage>
          -
          <lpage>272</lpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11. S. Hartmann, G. Pigozzi, and
          <string-name>
            <given-names>J.</given-names>
            <surname>Sprenger</surname>
          </string-name>
          .
          <article-title>Reliable methods of judgment aggregation</article-title>
          .
          <source>Journal of Logic and Computation</source>
          , forthcoming.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <given-names>N. R.</given-names>
            <surname>Jennings</surname>
          </string-name>
          .
          <article-title>Controlling cooperative problem solving in industrial multi-agent systems using joint intentions</article-title>
          .
          <source>Artif</source>
          . Intell.,
          <volume>75</volume>
          (
          <issue>2</issue>
          ):
          <fpage>195</fpage>
          -
          <lpage>240</lpage>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <given-names>S.</given-names>
            <surname>Konieczny</surname>
          </string-name>
          and
          <string-name>
            <surname>R.</surname>
          </string-name>
          <article-title>Pino-Pe´rez. Merging with integrity constraints</article-title>
          .
          <source>Lecture Notes in Computer Science</source>
          ,
          <volume>1638</volume>
          /
          <year>1999</year>
          :
          <fpage>233</fpage>
          -
          <lpage>244</lpage>
          ,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <given-names>L.</given-names>
            <surname>Kornhauser</surname>
          </string-name>
          and
          <string-name>
            <surname>L. Sager.</surname>
          </string-name>
          <article-title>The one and the many: Adjudication in collegial courts</article-title>
          .
          <source>California Law Review</source>
          ,
          <volume>81</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>51</lpage>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <given-names>F.</given-names>
            <surname>Kro</surname>
          </string-name>
          <article-title>¨ger. Temporal Logic of programs</article-title>
          . Springer, Berlin,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>H. J. Levesque</surname>
            ,
            <given-names>P. R.</given-names>
          </string-name>
          <string-name>
            <surname>Cohen</surname>
            , and
            <given-names>J. H. T.</given-names>
          </string-name>
          <string-name>
            <surname>Nunes</surname>
          </string-name>
          .
          <article-title>On acting together</article-title>
          .
          <source>In Proceedings of the Eighth National Conference on Artificial Intelligence</source>
          , pages
          <fpage>94</fpage>
          -
          <lpage>99</lpage>
          ,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <given-names>C.</given-names>
            <surname>List</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Puppe</surname>
          </string-name>
          .
          <article-title>Judgment aggregation: A survey</article-title>
          . In P. Anand,
          <string-name>
            <given-names>C.</given-names>
            <surname>Puppe</surname>
          </string-name>
          , and P. Pattanaik, editors,
          <source>Oxford Handbook of Rational and Social Choice</source>
          . Oxford,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <given-names>E.</given-names>
            <surname>Lorini</surname>
          </string-name>
          and
          <string-name>
            <given-names>D.</given-names>
            <surname>Longin</surname>
          </string-name>
          .
          <article-title>A logical account of institutions: From acceptances to norms via legislators</article-title>
          .
          <source>In KR</source>
          , pages
          <fpage>38</fpage>
          -
          <lpage>48</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19. E.
          <string-name>
            <surname>Lorini</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <string-name>
            <surname>Longin</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Gaudou</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <surname>A. Herzig.</surname>
          </string-name>
          <article-title>The logic of acceptance</article-title>
          .
          <source>Journal of Logic and Computation</source>
          ,
          <volume>19</volume>
          (
          <issue>6</issue>
          ):
          <fpage>901</fpage>
          -
          <lpage>940</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20. G. Pigozzi,
          <string-name>
            <given-names>M.</given-names>
            <surname>Slavkovik</surname>
          </string-name>
          , and
          <string-name>
            <surname>L. van der Torre.</surname>
          </string-name>
          <article-title>A complete conclusion-based procedure for judgment aggregation</article-title>
          .
          <source>In First International Conference on Algorithmic Decision Theory Proceedings</source>
          , pages
          <fpage>1</fpage>
          -
          <lpage>13</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <given-names>A. S.</given-names>
            <surname>Rao</surname>
          </string-name>
          and
          <string-name>
            <given-names>M. P.</given-names>
            <surname>Georgeff</surname>
          </string-name>
          .
          <article-title>Intentions and rational commitment</article-title>
          .
          <source>In In First Pacific Rim Conference on Artificial Intelligence (PRICAI-90) Proceedings</source>
          , pages
          <fpage>94</fpage>
          -
          <lpage>99</lpage>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <given-names>K.</given-names>
            <surname>Schild</surname>
          </string-name>
          .
          <article-title>On the relationship between bdi logics and standard logics of concurrency</article-title>
          .
          <source>Autonomous Agents and Multi-Agent Systems</source>
          ,
          <volume>3</volume>
          (
          <issue>3</issue>
          ):
          <fpage>259</fpage>
          -
          <lpage>283</lpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <given-names>T.</given-names>
            <surname>Icard</surname>
          </string-name>
          , E.Pacuit, and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Shoham</surname>
          </string-name>
          .
          <article-title>A dynamic logic of belief and intention</article-title>
          .
          <source>In Principles of Knowledge Representation and Reasoning: Proceedings of the Twelfth International Conference (KR-10)</source>
          , page forthcoming,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <given-names>R.</given-names>
            <surname>Tuomela</surname>
          </string-name>
          and
          <string-name>
            <given-names>K.</given-names>
            <surname>Miller</surname>
          </string-name>
          .
          <article-title>Groups beliefs</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>91</volume>
          :
          <fpage>285</fpage>
          -
          <lpage>318</lpage>
          ,
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <article-title>- M, s Gφ if and only if M, s φ for all s, s G ; - M, s Eφ if and only if M, s φ for all s, s E ; - M, s Xφ if and only if M, s φ for the s , s, s R - M, s φUψ if and only if M, s φ ; M, s i φ for all s i, i 1, 2</article-title>
          , . . . ,
          <article-title>k such that s, s 1, s 1, s2, . . . s k1 , sk R and for s k1 such that s k, sk1 R it holds M, sk1 φ and M, s k1 ψ</article-title>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>