=Paper=
{{Paper
|id=Vol-1283/paper26
|storemode=property
|title=
Not all Judgment Aggregation Should be Neutral
|pdfUrl=https://ceur-ws.org/Vol-1283/paper_26.pdf
|volume=Vol-1283
|dblpUrl=https://dblp.org/rec/conf/ecsi/Slavkovik14
}}
==
Not all Judgment Aggregation Should be Neutral==
https://ceur-ws.org/Vol-1283/paper_26.pdf
Not all judgment aggregation should be neutral
Marija Slavkovik
University of Bergen, Bergen, Norway
marija.slavkovik@infomedia.uib.no
Abstract. Judgment aggregation is concerned with the problem of ag-
gregating individual views on logically related issues. It offers a general
framework in which several different types of aggregation problems can
be represented and studied. Furthermore, judgment aggregation can be
applied to problems that consider together issues that are typically con-
sidered as separate aggregation problems. E.g., judgment aggregation
can be used to determine group goals and collectively advanced support-
ing beliefs. While goals can be seen as preferences, typically studied by
preference aggregation, beliefs cannot. Neutrality of a judgment aggre-
gation rule is the property of aggregating the views on all issues in the
same manner and it is uniformly considered to be a desirable property.
This paper defends the position that multi issue type aggregation prob-
lems may require non-neutral judgment aggregation rules, and proposes
a method for developing such rules.
Keywords: Judgment aggregation, neutrality, preference aggregation,
merging beliefs and goals
1 Introduction
A committee of investors is considering whether or not to buy the stock of
company X (proposition φ). It is known that company X has launched a new
product. Some consider that a market success of X (proposition ϕ1 ) and sufficient
available committee funds (proposition ϕ2 ) are reasons to purchase X shares. The
committee thus needs to assign truth values to φ, ϕ1 , and (ϕ1 ∧ ϕ2 ) → φ. The
committee is making these decisions around lunch time and they are feeling a bit
peckish. Typically they order pizza, however some are contemplating ordering
burgers instead. They have to yet find out where to order the burgers from, but
there are really only two options for pizza in town: Napoli and Peppe’s. If people
are not worried about spending too much money for lunch (proposition sP v),
then Napoli is far better than Peppe’s, at least according to some members’
opinion. After some discussion they realise they that what they have to decide
is: whether pizza is better than burger (proposition pP b), whether Napoli is
indeed better than Peppe’s (proposition nP e) and whether (pP b ∧ sP v) → nP e.
Judgment aggregation is a theory concerned with finding a combination of
consistent collective opinions on logically related issues by aggregating individual
views over those issues. It offers a general framework in which several different
types of aggregation problems can be represented and studied. Judgment ag-
gregation can be used to solve both the lunch and the investment problems of
the committee. Furthermore, both committee problems have the same underly-
ing structure. The same judgment aggregation rule can be applied to determine
both whether to invest in X, which is a belief aggregation problem, and whether
to order pizza from Napoli, which is a preference aggregation problem1 .
That same framework abstractness that allows for such versatility of problems
representable in judgment aggregation also allows for more complex problems
to be represented in it as well, problems that consider together issues that are
typically considered as separate aggregation problems. Such framework flexibility
can be useful, particularly in multi-agent system’s settings.
Self governing cooperating agents would encounter the need to agree on which
beliefs to uphold as a group and which goals to collectively pursue. Beliefs and
goals influence one another. A rational agent is one that chooses her goals in
accordance to the believes she holds about the world, but also that adjusts
her goals as her beliefs change. If the same consideration is to be extended to
groups of agents, a choice on whether to pursue a goal cannot be made separate
from the decisions on what to believe as a group. Agreements can be reached
by aggregating, or merging, the individual opinions of the agents; in the case
of determining collectively held beliefs and goals, by aggregating the individual
beliefs and goal choices of the agents [15, 3]. Judgment aggregation can be used
to concurrently reach agreements regarding goals and beliefs.
Judgment aggregation was originally concerned with collective reasoning in
collegiate courts of law [16, 17]. It was later shown that the same theory can
be used to represent preference aggregation problems [24]. Further, judgment
aggregation has been considered for the purpose of: identifying collective goals
and collectively supported beliefs [3, 28], finding collective argument labelings
in argumentation theory [4, 2], and most recently also to collective annotate
linguistic resources [11].
A judgment aggregation problem is specified by an agenda and possibly a set
of constraints. An agenda is the set of, possibly logically related, issues for which
individual judgments are collected and collective judgments are requested. The
issues are typically binary questions to which a positive (yes, true, agree, etc.)
or a negative answer (no, false, disagree, etc) can be given. A judgment is “an
answer” to an issue. The constraints are a set of formulas expressing additional
relations among the agenda issues that must be observed when “answering” the
issues. In the logical framework of judgment aggregation, in line with [6], an
agenda issue is represented in an abstract manner with a pair of propositions
ϕ, ¬ϕ. A positive “answer” or judgment is represented with ϕ, and a negative
one with ¬ϕ.
The abstract representation of the judgments and agenda issues induces the
versatility of the judgment aggregation framework. It also allows us to focus
1
Remark that a preference aggregation problem would typically be represented as the
problem of selecting one from the list of options: order pizza from Peppe’s, order
pizza from Napoli, order a burger, as specified in Section 4.
on the relations between individual and collective judgments and to study the
properties of aggregation functions in an abstract way, with findings of such
studies extending to all the domains of application. The abstract representation
of the agenda issues is beyond reproach when all the agenda issues have the same
meaning, i.e., are of the same type, for example when they are all beliefs, or all
preferences. However when the agenda contains a mixture of different types of
issues, as is the case when judgment aggregation is used to reach an agreement
on a common goal, the “flat” representation is no longer adequate.
In [6] it was shown that the representation language of the agenda, in general,
cannot be exploited to circumvent the impossibility results regarding judgment
aggregation rules [24]. This result appears to have ended the interest in the
representation language in judgement aggregation. However, the representation
of the agenda issues is rather the lesser problem, more crucial is that different
types of issues would require different treatment by the aggregation function,
which is not the case at present.
The property of aggregation functions which stipulates that all issues should
be aggregated in the same fashion is called neutrality. At present, neutrality is
considered desirable to that extent, that non-neutral rules virtually have not been
designed and a precise formulation of the neutrality property has not been given.
When the agenda has the same type of issues, neutrality seems natural to require.
However there are agendas where this is not the case. Consequently, neutrality
should not be considered a de facto desirable condition in judgment aggregation
and non-neutral aggregation methods must be explored. Furthermore, the type of
issues being aggregated must be taken into consideration and desirable properties
of aggregation should be associated with the distinct type.
This work is structured as follows. In Section 2 we elaborate on different types
of issues that can be found in the agendas of various judgment aggregation prob-
lems. In Section 3 we give the definitions of the main concepts in the judgment
aggregation framework. In Section 4 we discuss the property of neutrality and
other assumed-desirable properties. In Sections 5 one method of non-neutral ag-
gregation is proposed. In Section 6 we make a summary and outline directions
for future work.
2 Types of issues and their aggregation
The representation of issues in the logic framework abstracts from the meaning
of those issues. An issue ϕ, ¬ϕ that arrises from a preference has the intuitive
reading of “is it the case that you prefer option x to option y”. Often special
predicates xP y, ¬xP y are used, e.g., in [8, 20], for representing issues that are
preferences, or as in the lunch committee decision example. An issue ϕ, ¬ϕ that
arises from beliefs has the intuitive reading of “is it the case that you believe
ϕ is true”, whereas if that same issue arises from possible agent goals it can be
interpreted both as “should goal ϕ be pursued” and “do you want to pursue goal
ϕ”. An issue ϕ, ¬ϕ may represent an, at present, unknown or unobservable fact
about the world, e.g., will the project of company X be a market success, thus
having the intuitive reading of “are you of the conviction that ϕ is true”.
In a preference aggregation problem all issues in the corresponding judgment
aggregation agenda are of the same type – preferences. When a jury considers
evidence, all the issues are of the same type – past-observable facts of the world.
However, there is nothing preventing anyone from using judgment aggregation
on problems with agendas that concurrently have different types of issues.
In fact, the original judgment aggregation problem, the “doctrinal para-
dox”[18], has an agenda in which two types of issues occur: premises and a
conclusion. The agenda in the “doctrinal paradox” consists of three issues. The
premises are: ‘Is there a contract?’, and ‘Is there an estoppel?’. The conclu-
sion is the decision on the case – ‘Does the judge(s) rule for the plaintiff ?’ (or
against). The judges must rule in favour of the plaintiff when there was both a
contract and an estoppel. The truth of both the issue of the existence of contract
and estoppel are facts of the world (which might not be observable), while the
decision on the case is the (informed) opinion of the judge(s).
It is straightforward to observe that preferences differ from presently unob-
servable facts about the world. One agent can prefer pizza over burger (pP b),
and another can prefer burger over pizza (¬pP b) but both of these positions can
be held at the same time by different agents because one does not invalidate the
other; preferences are matters of subjective views and tastes. One agent might
be convinced that the project of company X will be a market success (φ), while
another might be convinced that the project will be failure (¬φ). However, only
one of these judgments is correct, and some time after the project’s release it
would be known which one. Preferences give rise to subjectively valued propo-
sitions, while unknown facts give rise to objectively valued propositions in the
agenda. The truth-value of subjectively valued propositions exists based solely
on the bias, opinion, emotions, convictions, etc. of the entity that assigns this
value. In contrast, the truth-value of an objectively valued proposition exists
independently of the bias of the entity that assigns the value.
Regardless of whether the agenda issue is a subjectively or objectively valued
proposition, there may exist only one entity, agent, or source, that is in the unique
position to assign the judgment for this issue. For example, I am the only person
in the position to decide whether for me Napoli pizza is affordable lunch-food.
My affordability preference however might be included in an agenda with other
issues that are related but over whose truth-value I do not have full control.
Objectively valued propositions yield even better examples of full control issues.
Consider the owner of the funds that the committee will invest. Whether there
are available funds for investing (if ϕ2 is true or false) can only be decided by
the owner, as he or she fully controls this resource. An issue can also be fully
controlled by a group of agents. For example, Mary and John can together decide
to marry each other, thus determining the judgment on the issue of “Mary and
John are married”. Instances of objectively valued fully controlled judgments
can be expected to occur when one agent coordinates with a group of agents, or
a group of agents coordinates with another group.
The beliefs and goals give rise to different types of propositions as well. The
beliefs of the agents represent the current information that the agent has about
the world. The proposition G x denotes the decision of the agent to have his
future state be one in which x holds. We can view the agent’s goals as giving
rise to subjective valued propositions, because the truth value of the associated
proposition is the prerogative of the agent. The agent however, may not be able
to guarantee a future in which x holds, as much as a person can not always
guarantee that pizza will be served to them instead of a burger, as per their
preference. Beliefs, on the other hand can be viewed as giving rise to objectively
valued propositions. Although an agent chooses which beliefs to adopt, the aim
of beliefs is to as faithfully as possible represent the state of the world.
The type of the issues that represent labels of arguments in an argumentation
context and annotations of linguistic resources would depend on the specific
context in which these issues are created.
It is not our aim at this section to give an ultimate taxonomy of issue types
that can occur in judgment aggregation but to emphasise that different types
do exist and that there is a need for them to be aggregated accordingly. For the
purpose of this article it is sufficient to distinguish between objectively valued
and subjectively valued issues, and orthogonally, between fully controlled and
partially controlled issues.
3 Preliminaries
We begin by introducing the definitions of the basic concepts that can be found
in a logical framework of judgment aggregation before discussion the property
of neutrality in greater detail.
An agenda A is a set of propositions, not necessarily atomic, from a set of
well formed propositions L. This set L is often taken to be the set of well formed
formulas of the (classical) propositional logic. For each ϕ ∈ A it holds that
¬ϕ ∈ A, and ϕ is neither tautology nor a contradiction. An agenda issue is a
pair {ϕ, ¬ϕ} ⊆ A and a judgment is thus either ϕ or ¬ϕ, corresponding to taking
the position that the issue in question is true or false respectively2 . For example,
the agenda of the investment problem is {φ, ¬φ, ϕ1 , ¬ϕ1 , (ϕ1 ∧ ϕ2 ) → ψ, ¬((ϕ1 ∧
ϕ2 ) → ψ)}, while the agenda of the lunch problem is {pP b, ¬pP b, nP e, ¬nP e, (pP b∧
sP v) → nP e, ¬((pP b ∧ sP v) → nP e)}. In addition to the agenda, one can spec-
ify a set of constraints Γ ⊂ L which constrain further3 the allowed combination
of judgments that can be given for the specified agenda.
A judgment set J is a subset of the agenda that is complete when for each issue
{ϕ, ¬ϕ} ⊂ A either ϕ ∈ J or ¬ϕ ∈ J. For example, the opinions of a committee
member that has no monetary concerns, thinks that pizza is preferable to a
burger, and that pizza is better ordered from Peppe’s are represented as the
2
A proposition preceded by an odd number of consecutive negations is considered
to be a negated proposition, while a proposition preceded by an even number of
consecutive negations is considered to be a non negated proposition.
3
The agenda issues are typically logically related among each other.
set of judgments {pP b, ¬((pP b ∧ sP v) → nP e), ¬nP e}. The judgment set J is
consistent if it is a consistent set and Γ -consistent if J ∪ Γ is a consistent set.
In the rest of this article we use consistent to denote both consistent and Γ -
consistent judgment sets. The set of all non-empty consistent judgment sets for
a specified A and Γ is the set D(A, Γ ), while the set of all consistent judgments
sets that are also complete is D(A, Γ ). Clearly D ⊂ D.
A profile of judgments P ∈ D(A, Γ )n is a collection of n judgment sets,
each associated with an agent, or source, 1 ≤ i ≤ n. For convenience, we
use Ji ∈ P to denote that Ji is the i’th element of P . The number of all
judgment sets, or agents, in P that contain a particular ϕ ∈ A is NP (ϕ),
NP (ϕ) = |{i | ϕ ∈ Ji , and Ji is the ith element of P }|.
A judgment aggregation rule R, also called an irresolute rule, is a function
of type R : D(A, Γ )n → 2D(A,Γ ) . A judgment aggregation rule maps a profile
of judgments to a set of complete and consistent judgment sets. A judgment
aggregation function F , also called a resolute rule, is defined as a function of
type F : D(A, Γ )n → D(A, Γ ). A resolute rule maps a profile of judgments to a
single complete and consistent judgment set. The judgment sets in the set R(P )
are called collective judgment sets for P . The elements of a collective judgment
set are called collective judgments.
Typically the impossibility characterisation results, namely studies showing
which properties of aggregation cannot be satisfied at the same time, have been
done for the resolute rules, see for example [24] for an overview. Almost all
specific proposed judgment aggregation operators are irresolute rules [25, 19, 26,
5, 10, 7, 12]. The possibly best known judgment aggregation rule is issue-majority
defined as m(P ) = {ϕ | ϕ ∈ A, NP (ϕ) > 21 }. A profile P ∈ D(A, Γ )n is majority-
consistent if m(P ) is a consistent set, i.e., m(P ) ∈ D(A, Γ ).
The following properties have been defined for irresolute rules, however the
definitions extend in a straightforward way to resolute rules as well.
A judgment aggregation rule R is anonymous if for every P ∈ D(A, Γ )n , and
every A and Γ , for every permutation σ on the order of the judgment sets in P
it holds that R(P ) = R(σ(P )).
A judgment aggregation rule R is majority-preserving if for every majority-
consistent profile P ∈ D(A, Γ )n , and every A and Γ , it holds that
R(P ) = {J | m(P ) ∈ D(A, Γ ) and m(P ) ⊆ J}. Intuitively, whenever a profile P
is majority-consistent and m(P ) is a complete judgment set, then a majority-
preserving function R would select only the m(P ) as a collective judgment set.
For those majority-consistent profiles for which m(P ) is not a complete judg-
ment set, only all the complete and consistent judgment sets that are supersets
of m(P ) are selected as collective judgment sets.
Systematicity is one of the first properties studied, together with anonymity
[23, 24], and defined for resolute judgment aggregation rules. It is composed of
two properties: neutrality and independence of irrelevant alternatives (IIA). A
judgment aggregation function F satisfies IIA if for all all ϕ, φ ∈ A, and all A,
Γ , for every P, P 0 ∈ D(A, Γ )n , where P = hJ1 , . . . Jn i and P 0 = hJ10 , . . . Jn0 i, if [
ϕ ∈ Ji iff φ ∈ Ji0 ], then [ϕ ∈ F (P ) iff φ ∈ F (P 0 )]. Intuitively, a function satisfies
IIA if the collective judgment on each issue depends only on the individual
judgments for that issue in the profile4 .
The neutrality property of a judgment aggregation rule R is informally the
property: the collective judgment on an issue depends only on the number of
positive and negative judgments given for that issue in the profile.
A version of neutrality called issue-neutrality, has been formally defined for
resolute rules in [13] in the following way. A judgment aggregation function F
is issue-neutral if for all ϕ, φ ∈ A, and all A, Γ , P ∈ D(A, Γ )n , where P =
hJ1 , . . . Jn i, if [for all Ji ∈ P , ϕ ∈ Ji iff φ ∈ Ji ], then [ϕ ∈ F (P ) iff φ ∈ F (P )].
It is not straightforward to extend issue-neutrality to irresolute rules be-
cause there are several ways to do this, each varying in strength. The strongest
corresponding property can be expressed as in Definition 1.
Definition 1. An irresolute judgment aggregation rule R is strongly issue-neutral
when for all ϕ, φ ∈ A, and all A, Γ , P ∈ D(A, Γ )n , where P = hJ1 , . . . Jn i, if
[ϕ ∈ Ji iff φ ∈ Ji , for 1 ≤ i ≤ n], then [for all J ∈ R(P ), ϕ ∈ J iff φ ∈ J].
4 Desirable properties in judgment aggregation
The initial inspiration for a minimal set of desirable properties in judgment
aggregation was the desirable properties for preference aggregation according to
Arrow [1]. A preference aggregation problem is specified as follows. Given is a set
of options (or candidates, or alternatives) O a preference profile is a collection
of n preference orders %i , 1 ≤ i ≤ n, over O. A preference aggregation function
maps a preference profile to a preference order over O. A preference aggregation
function is neutral if the options are treated the same, regardless of what the
option is. It is reasonable to expect that all alternatives are treated the same by
the aggregation function.
Non-neutral aggregation functions that satisfy IIA are still dictatorial. A res-
olute judgment aggregation function F is dictatorial when for every profile P
the same collective judgment J is selected. Note that in the original formulation
of the framework and the definition of the judgment aggregation function, func-
tions were defined as F (J1 , . . . Jn ) = J, where J1 , . . . , Jn , J are subsets of the
agenda but not required to be complete nor consistent. Two additional proper-
ties for F have been additionally explicitly defined: universal domain stipulates
that function F has to be defined for each possible profile; collective rationality
stipulates that the collective judgment set J is a consistent and complete set.
The premise based rule [9], that satisfies IIA , and is anonymous and neutral,
does not satisfy universal domain.
Judgment aggregation irresolute rules that do not satisfy IIA have been con-
sidered [25, 19, 26, 5, 10, 7, 12], but all of them consider only the support for a
judgment on an issue and not the issue itself, i.e., they are neutral in the infor-
mal sense. Their neutrality is not explicitly considered, testifying to the universal
understanding of neutrality as a desirable property for rules beyond discussion.
4
Observe that the co-domain of F(P) is the set of consistent and complete judgment
sets, thus φ 6∈ F (P 0 ) implies ¬φ ∈ F (P 0 )
A property that also appears to be universally accepted as desirable, possi-
bly also as a legacy from the close connections between judgment aggregation
and preference aggregation, is majority-preservation. A rule that is majority-
preserving, when applied to a preference aggregation problem represented as a
judgment aggregation problem, produces collective judgment sets that contain
the Condorcet winner when such an alternative exists.
The motivation for desiring to maximise the number of collective judgments
supported by a majority in the profile, which is the typical method for construct-
ing majority-preserving rules, is clear when the agenda issues are preferences.
Since every agent is the ultimate authority on what they want, and since not
everyone can get what they individually want, the aim of the aggregation is to
maximise the satisfaction of as many agents as possible. However, if the issues
are objectively valued, it does not matter how many agents are “happy” with
the collective judgments. What matters is to increase the likelihood that the
issues’ objective truth-values coincide with the assigned collective judgments.
The Condorcet jury theorem [14] states that if the agents are more likely than
not to individually find the correct valuation for a proposition, than the valuation
supported by the majority is most likely to be correct. However the propositions
in question are independent, whereas the issues in an agenda are logically related
[22]. When the conditions of the Condorcet jury theorem do not apply, it is no
longer clear why a judgment aggregation rule should be majority-preserving.
Furthermore, in the case of an agenda with objectively valued issues, rational
agents can be expected to be more “satisfied”, or have an increase in utility,
with the increase of the likelihood of truth of the collective judgments, not with
the increase of their likelihood to their own opinions. In conclusion, majority-
preservation can be clearly seen as desirable only in the case of subjectively
valued propositions.
Lastly let us briefly concern ourself with the desirability of the anonymity
property. Clearly, when there is no way of establishing the relevance or reliability
of a judgment’s source, it is prudent to seek anonymous judgment aggregation
rules. However, when the value of an agenda issue is fully controlled by some,
this agent, or these agents, should have veto over the issues they control. To
aggregate an agenda that consists both of fully controlled issues and partially
controlled issues, one needs a rule that is non-neutral and non-anonymous with
respect to the fully controlled issues.
5 Sequential judgment aggregation
Having argued that a judgment aggregation rule should only be neutral if the
agenda issues are type-homogenous, we need to consider methods for construct-
ing non-neutral judgment aggregation rules. A judgment aggregation rule that
satisfies IIA can be easily made non-neutral: since each issue is aggregated sepa-
rately, one simply needs to use different, type-adequate, rules, to aggregate every
issue. However, since IIA rules will necessarily violate either universal domain
or anonymity, using a IIA rule may not be possible.
One option to treat issues differently is to apply, possibly different, judg-
ment aggregation rules sequentially to the profile. This method is what is used
when the premise based rule [9] is applied, and also in the complete conclusion
based procedure of [27]. The method of [27], applicable to agendas that are split
into premises and conclusions, is to use a resolute judgement aggregation rule
to one issue of the agenda, the conclusion issue, and add the resulting collective
judgment to the constraints before aggregating the full agenda with an irresolute
rule. An extreme example of a sequential rule is the one considered in [21], where
each issue is aggregated one by one. A similar approach can be used to aggre-
gate profiles for mixed issue type agendas: to aggregate the agenda sequentially
in several steps, at least one for each different type of issue. Thus, all of the
completely controlled issues would form one issue type, that can possibly be fur-
ther divided to a subset of objectively valued issues and a subset of subjectively
valued issues.
In [27], issue-majority over one issue and in the context of an odd number
of agents, which is a resolute, is used as the first step. In general a resolute
rule might not be available, meaning that several collective judgment sets might
be the result of a first aggregation step. Since the collective judgments from
different collective judgment sets can be mutually inconsistent, they cannot all
be included in the constraints of the next step. There are several options on how
to select the constraints. For example, one can: a) add the disjunction of the
collective judgment sets; b) add the intersection of the collective judgment set;
c) select one of the collective judgment sets using a tie-breaking mechanism; etc.
The definitions of the logic framework now need to be extended.
Definition 2.
Partial agendas.An agenda A is partitioned into k partitions when
A = Ap1 ∪ Ap2 ∪ · · · ∪ Apk , and all the partitions Ap1 , Ap2 , · · · , Apk are
mutually disjoint. Given an aggregation order, namely some permutation of
the partitions of the agenda σ(hAp1 , . . . , Apk i), the step agendas are obtained
as A1 = σ(Ap1 ), A2 = σ(Ap2 ) ∪ A1 , . . . , Ak = σ(Apk ) ∪ Ak−1 = A.
Partial profile. A partial profile Pi , 1 ≤ i ≤ k for profile P =< J1 , . . . , Jn >
of the agenda Ai is defined as Pi = hJ1 ∩ Ai , . . . , Jn ∩ Ai i.
Collective selector. A collective selector function ς : 2D(A,Γ ) → 2L maps a
set of collective judgment sets into a consistent set of formulas. It holds that
ς(R(P )) = S, S ⊂ 2L , S 2 ⊥ and if R(P ) = J, then ς(R(P )) = J.
Partial aggregator. A partial aggregator function Ri,Γi : D(Ai , Γ )n →
2D(Ai ,Γi ) , where Γ, Γi ∈ L, 1 ≤ i ≤ k are consistent and mutually consistent
constraints, is an irresolute judgement aggregation rule that aggregates a
profile of judgment sets consistent with respect to a set of constraints Γ into
a set of judgment sets consistent with respect to the set of constraints Γi .
Partial constraints. An aggregation constraint at step i, 1 ≤ i ≤ k, is
Γi ⊇ Γ , such that Γ1 = Γ , and Γi = ς(Ri−1,Γi−1 (Pi−1 )) ∪ Γi−1 .
Sequential judgment aggregation rule. Given a k-partitioned agenda A,
and an aggregation order σ, a judgment aggregation rule R is sequential when
R(P ) = Rk,Γk (P ), and Ri,Γi (Pi ), for 1 ≤ i ≤ k, is recursively defined as:
Γ1 = Γ , and Γi = ς(Ri−1 , Γi−1 (Pi−1 )) ∪ Γi−1 .
Some clarifying observations are due before an illustrative example. The se-
quential judgment aggregation rule is not neutral even when all partial aggrega-
tors used are the same functions, because the issues at each step are considered
under different constraints.
The collective selector selects R(P ) iff R(P ) is a singleton and employs some
selection method when R(P ) is not a singleton, such as the disjunction of all the
judgment sets produced by R(P ), their intersection, etc.
At the first step of aggregation, all the judgment sets in the partial profile are
consistent with the aggregation constraint Γ . At each subsequent step, Γ is made
stronger, namely more selective, by the outcome of the previous aggregation
step. The partial judgment sets in the subsequent partial profiles may not be
consistent with the new constraints Γt , but the result of the aggregation must
be. Since at each step the constraints are obtained by adding the results of the
aggregation in the previous constraints, it is not possible, at any point for Γt to
be inconsistent with Γt+1 . In the last step, the full profile A is aggregated with
the strongest constraint Γk .
The judgment aggregation functions so far defined are not from the desired
type as partial aggregator function Rt since they require that the same constraint
Γ be imposed both on the judgment sets in the profile and on the collective
judgment sets obtained by aggregation. However, the aggregation rules from [19,
10, 7], can be extended in a straightforward fashion, into partial aggregators.
Consider for example the M CSAΓ from[20], also known as the Slater rule for
judgments, defined as M CSAΓ (P ) = MaxCardCons(m(P ), Γ ), where
MaxCardCons(m(P ), Γ )) denotes the maximum cardinality set of Γ -consistent
subsets of m(P ). The M CSAi,Γi partial aggregator can be defined as
M CSAi,Γi (Pi ) = MaxCardCons(m(Pi ), Γi ). Another example is the M W AΓ
from [20], also known as the median
P rule, defined as
M W AΓ (P ) = argmax WP (J), where WP (J) = NP ϕ. The M W Ai,Γi par-
J∈D(A,Γ ) ϕ∈J
tial aggregator can be defined as M W Ai,Γi (Pi ) = argmax WP (J), where
J∈D(Ai ,Γi )
P
WP (J) = ϕ∈J NPi ϕ.
Example 1. Let A = Ap1 ∪ Ap2 ∪ Ap3 , with Ap1 = {p, ¬p, q¬q, } , Ap2 = {r, ¬r},
and Ap3 = {s, ¬s}. Let Γ = {(p ∧ q) ↔ r, r → s}. Consider the profile given in
Table 1. Assume we aggregate first A1 = Ap2 using the issue-majority rule, then
A2 = Ap1 ∪ A1 using M CSA2,Γ2 W and last A3 = A using the M W A3,Γ3 . Let the
selector function be ς(Ri (Pi )) = Ri (Pi ), 1 ≤ i ≤ 3. The collective judgments
of, and constraints applied in, each aggregation step are also given in in Table 1.
The agenda can be split into as many partitions as there are issue types,
with the issues in one partition being all of the same type. The question is then
which type of issues should be given precedence in the order of aggregation. It
is questionable whether a uniform rule for type order in sequential aggregation
can be established in this regard.
Agenda Constraints
Ap1 Ap2 Ap3
{p, ¬p, q, ¬q r, ¬r, s, ¬s} Γ = {(p ∧ q) ↔ r, r → s}
J1 = {p q r s}
J2 = {p ¬q ¬r s}
J3 = {¬p q ¬r ¬s}
m(P1 ) = {¬r} Γ1 = Γ
M CSA2,Γ2 (P2 ) = {{ p, ¬r}, Γ2 = Γ1 ∪ {¬r}
{ q, ¬r}} Γ2 = Γ1 ∪ {¬r}
M W A3,Γ3 (P ) = {{p, ¬q, ¬r, s}, Γ3 = Γ2 ∪ {(¬p ∧ q ∧ ¬r) ∨ (p ∧ ¬q ∧ ¬r)}
{¬p, q, ¬r, s}} Γ3 = Γ2 ∪ {(¬p ∧ q ∧ ¬r) ∨ (p ∧ ¬q ∧ ¬r)}
Table 1. An example of sequential aggregation.
It can be consistently assumed that the assigned values of the fully controlled
issues, by their respective controllers, are mutually consistent. If these judgments
are not mutually consistent, then either they are not really fully controlled,
or there is a problem with setting the agenda. This assumption also imposes
that the first aggregation step is allowing each of the controllers to establish
the judgments for these issues, regardless of whether they are objectively, or
subjectively valued, and adding these judgments to the aggregation constraints,
forming the constraints for the next step. The partially controlled judgments can
be considered in two separate agenda partitions with respect to whether they
are subjectively or objectively valued. It is less clear how to aggregate the latter.
On one hand, a rational group of agents would be expected to base its deci-
sions on what to desire based on what it determines is most likely to be the true
state of the world. Thus the decisions on which goals should be pursued, should
be made after making the decisions on what beliefs to support as a collective.
This would imply that the subjectively valued issues should be aggregate after
the objectively valued ones. However, an agent might choose what to adopt as
the true state of the world based on what goals it has chosen to proceed, partic-
ularly if the nature of the objectively valued issues is such that their truthfulness
cannot be expected to be established in the forceable future. Consider a family
considering to join a religious denomination. It is unforeseeable when it would
be established whether god X exists or not, however this can still be considered
an objectively valued issue. The members of the congregation of believers in
X are offered free housing. The family might first decide to pursue the goal of
obtaining free housing and then decide that god X exits.
In conclusion, the decision of order of aggregation should be further specified
by a more detailed consideration of the context of the aggregation problem.
6 Summary and Future work
The judgment aggregation framework is sufficiently abstract to allow a variety of
aggregation problems to be represented as judgment aggregation problem. Thus
studying the properties of judgment aggregation we obtain results that extend
to all related aggregation problems. However, what the literature has failed to
observe, or at least visibly emphasise, is that the abstractness of the framework
also allows for more complex aggregation problems to be represented as well.
The goal of this paper is to put a spotlight on this issue and advocate for more
research attention to be directed towards aggregating for agendas that contain
more than one type of issues.
A judgment aggregation rule is neutral if all the agenda issues are given
equal treatment when aggregated. Neutrality has been uniformly accepted as a
desirable property for rules, however this desirability is unfounded in the case of
complex agendas. There are several issues that need to be pursued when develop-
ing non-neutral rules. Primarily, there exists no formal definition for neutrality
for irresolute rules, although these are the only type of rules that exists which
are anonymous and satisfy universal domain. One definition was offered here,
however there are several approaches that can be taken and the best one, if it
exists, should be established. Although some discussion is included here regard-
ing the type of issues that can be considered in an agenda, the presented issue
taxonomy is neither exhaustive nor precise, and needs to be improved in these
regards. Further, adequate aggregation properties need to be associated for each
issue types. For instance, majority-preserving rules are desirable for subjectively
valued, but maybe not for objectively valued issues.
The here-defined sequential judgment aggregation rules and framework allow
for neutral rules to be used in combination that yields non-neutral aggregation.
However, the approach of sequential judgment aggregation opens its own set of
unexplored problems. One is the issue of the selector function that determines the
formation of constraints for each aggregation step. Three examples of selectors
are given, but this is not an exhaustive list. The advantages and disadvantages
of each need to be explored. Another big open issue is the question of property
inheritance, for example, if all the partial aggregators satisfy property x is it
the case that the resulting sequential aggregation rule will also satisfy x; which
properties are inherited and which are not, etc. Lastly there possibly are other
methods for non-neutral judgment aggregation beyond the sequential judgment
aggregation rules.
Acknowledgments. I would like to acknowledge, and express gratitude for, the
insightful discussions with Marc van Zee and Leon van der Torre from the Uni-
versity of Luxembourg.
References
1. K. Arrow. Social choice and individual values. Cowles Foundation Monograph
Series, 1963.
2. E. Awad, R. Booth, F. Tohmé, and I. Rahwan. Judgment aggregation in multi-
agent argumentation. CoRR, abs/1405.6509, 2014.
3. G. Boella, G. Pigozzi, M. Slavkovik, and L. van der Torre. Group intention is
social choice with commitment. In Marina De Vos, Nicoletta Fornara, Jeremy
Pitt, and George Vouros, editors, Coordination, Organizations, Institutions, and
Norms in Agent Systems VI, volume 6541 of Lecture Notes in Computer Science,
pages 152–171. Springer Berlin / Heidelberg, 2011.
4. R. Booth, E. Awad, and I. Rahwan. Interval methods for judgment aggregation in
argumentation. In KR, 2014.
5. R. Camps, X. Mora, and L. Saumell. A general method for deciding about logically
constrained issues. Annals of Mathematics and Artificial Intelligence, 64(1):39–72,
2012.
6. F. Dietrich. A generalised model of judgment aggregation. Social Choice and
Welfare, 28(4):529–565, June 2007.
7. F. Dietrich. Scoring rules for judgment aggregation. Social Choice and Welfare,
pages 1–39, 2013.
8. F. Dietrich and C. List. Arrow’s theorem in judgment aggregation. Social Choice
and Welfare, 29(1):19–33, July 2007.
9. F. Dietrich and P. Mongin. The premisse-based approach to judgment aggregation.
Journal of Economic Theory, 145(2):562–582, 2010.
10. C. Duddy and A. Piggins. A measure of distance between judgment sets. Social
Choice and Welfare, 39(4):855–867, 2012.
11. U. Endriss and R. Fernández. Collective annotation of linguistic resources:
Basic principles and a formal model. In Proceedings of the 51st Annual
Meeting of the Association for Computational Linguistics (ACL-2013), August
2013. A short version was presented at BNAIC-2013 and is available at
http://www.illc.uva.nl/∼ulle/pubs/files/EndrissFernandezBNAIC2013.pdf.
12. P. Everaere, S. Konieczny, and P. Marquis. Counting votes for aggregating judg-
ments. In Thirteenth International Conference on Autonomous Agents and Multi-
agent Systems (AAMAS’14), 2014.
13. U. Grandi and U. Endriss. Lifting integrity constraints in binary aggregation.
Artificial Intelligence, 199–200:45–66, 2013.
14. B. Grofman, G. Owen, and S.L. Feld. Thirteen theorems in search of the truth.
Theory and Decision, 15(3):261–278, 1983.
15. S. Konieczny and R. Pino Pérez. Propositional belief base merging or how to merge
beliefs/goals coming from several sources and some links with social choice theory.
European Journal of Operational Research, 160(3):785–802, 2005.
16. L.A. Kornhauser and L.G. Sager. Unpacking the court. Yale Law Journal, 96:82–
117, 1986.
17. L.A. Kornhauser and L.G. Sager. The one and the many: Adjudication in collegial
courts. California Law Review, 81:1–51, 1993.
18. Lewis A. Kornhauser. Modeling collegial courts. ii. legal doctrine. Journal of Law,
Economics, & Organization, 8(3):pp. 441–470, 1992.
19. J. Lang, G. Pigozzi, M. Slavkovik, and L. van der Torre. Judgment aggregation
rules based on minimization. In Theoretical Aspects of Rationality and Knowledge,
TARK’11, pages 238–246, 2011.
20. J. Lang and M. Slavkovik. Judgment aggregation rules and voting rules. In
Proceedings of the 3rd International Conference on Algorithmic Decision Theory
(ADT 2013), volume 8176 of Lecture Notes in Artificial Intelligence, pages 230–244.
Springer-Verlag, 2013.
21. C. List. A model of path-dependence in decisions over multiple propositions. Amer-
ican Political Science Review, 3(98):495 – 513, 2004.
22. C. List. Collective wisdom: Principles and mechanisms. In H. Landemore and
J. Elster, editors, Collective wisdom: a judgment aggregation perspective, chapter 9,
pages 203–229. Cambridge University Press, 2012.
23. C. List and P. Pettit. Aggregating sets of judgments: An impossibility result.
Economics and Philosophy, 18(01):89–110, 2002.
24. L. List and B. Polak. Introduction to judgment aggregation. Journal of Economic
Theory, 145(2):441 – 466, 2010.
25. M.K. Miller and D. Osherson. Methods for distance-based judgment aggregation.
Social Choice and Welfare, 32(4):575–601, 2009.
26. K. Nehring, M. Pivato, and C. Puppe. Condorcet admissibility: Indetermi-
nacy and path-dependence under majority voting on interconnected decisions.
http://mpra.ub.uni-muenchen.de/32434/, July 2011.
27. G. Pigozzi, M. Slavkovik, and L. van der Torre. A complete conclusion-based pro-
cedure for judgment aggregation. In F. Rossi and A. Tsoukiàs, editors, Algorithmic
Decision Theory, 1rst International Conference, ADT 2009, Venice, Italy, Octo-
ber 20-23, 2009. Proceedings, volume 5783 of Lecture Notes in Computer Science,
pages 1–13. Springer, 2009.
28. M. Slavkovik and G. Boella. Recognition-primed group decisions via judgment
aggregation. Synthese, 2012.