In situations where expert committees face significant uncertainty while assessing the likelihood of events, it is crucial to appropriately represent and aggregate their probabilistic opinions. In this work, we adopt the epistemological concept of imprecise probabilities to capture an expert's belief and employ the Condorcet Jury Theorem (CJT) for aggregating these beliefs through voting. To suit our requirements, we utilize a generalized version of the CJT and integrate imprecise probabilistic beliefs using a framework based on supervaluationism, a philosophical theory that addresses vagueness. Drawing inspiration from the field of expert elicitation research, we provide an interpretation of the alternatives in the voting process: Experts express their preferences by voting for probability intervals, referred to as bins. In this setting, each bin corresponds to a subinterval of the unit interval such that exactly one bin contains the objective probability for the event under evaluation to occur. Finally, we establish a bound on the maximum number of alternatives allowed in an election, which directly corresponds to the achievable precision in the aggregation process.
Consider the following four events. There will occur major changes in (i) the Atlantic meridional
overturning circulation; (ii) the Greenland ice sheet; (iii) the West Antarctic ice sheet; (iv) the
Amazon rainforest and El Nino/Southern Oscillation [
Assessing such probabilities is not only crucial for climate policy but also common across various disciplines when determining the likelihood of events under severe uncertainty. Answering questions like the one above often requires aggregating and evaluating the probabilistic beliefs of multiple experts. This latter process is the focal point of a discipline referred to as expert elicitation. On a theoretical level, when evaluating events characterized by severe uncertainty, two fundamental questions must be addressed: (1) How can we appropriately represent the probabilistic beliefs of experts? (2) What constitutes a reasonable method for aggregation?
In this work, we tackle the first question by modeling experts’ beliefs using imprecise probabilities. To address the second question, we demonstrate how imprecise probabilistic beliefs can be aggregated through voting, leveraging the Condorcet Jury Theorem (CJT). To accomplish this, we initially introduce the original version of the CJT, followed by the generalized variant that is central to our aggregation scenario. Subsequently, we present the imprecise model of belief and illustrate how it can be applied to voting by embedding it into a specific theory of vagueness known as supervaluationism. Next, we incorporate imprecise beliefs into the CJT framework, establishing a direct relationship between the number of alternatives experts can vote on and the precision achievable in the aggregation process. Finally, we provide an upper bound on the maximal permissible precision, which depends on the size and competence of the expert committee.
The CJT is a theoretical cornerstone from voting theory. In a voting setting, a set of agents may vote on a (possibly restricted) set of alternatives. A fundamental assumption of the CJT is that exactly one of those assumptions is correct in that it represents an underlying ground truth. The CJT then provides probabilistic guarantees for the capacity of the agents to identify the presumed ground truth through voting. In order to derive these probabilistic guarantees, a few central assumptions are made about the voting setting. In its original variant, it states that for a group of independent, equally competent agents with a better-than-chance probability to identify the true of two alternatives, majority voting best tracks this true state. Theorem 1 (Marquis de Condorcet 1785). For odd-numbered homogenous groups of independent and reliable agents in a dichotomic voting setting, the probability that majority voting identifies the correct alternative (1) increases monotonically with the number of agents and (2) converges to 1 as the number of agents goes to infinity (non-asymptotic part) (asymptotic part).
Given the inherent limitations and unrealistic assumptions associated with real-world
applications of the Condorcet Jury Theorem (CJT), significant research has been devoted to finding
generalizations that relax these assumptions while still preserving the core principles. While the
asymptotic part of the theorem has been subject to generalizations, it has been demonstrated
that the non-asymptotic part fails for small numbers of agents when the assumption of equal
voter competence is weakened [
Our Contribution. In this work, we address a sometimes overlooked, implicit assumption
underlying the CJT. Namely, that the agents are assumed to hold binary beliefs where an agent
either believes in a proposition and, presumably, votes for it, or does not. However, developing
a jury theorem that can aggregate non-binary beliefs, such as probabilistic opinions, remains
an open problem [
After providing an intuitive introduction to the CJT and the central generalization in our context, we proceed to present the formal framework of the Karge and Rudolph 2022 generalization in more technical detail. We first elaborate on the approval voting setting, followed by an exposition of the formal probabilistic framework.
Approval Voting. In the approval voting setting, we assume a finite set = {1, . . . , } of items referred to as alternatives as well as a finite set = {1, . . . , } of agents. Note that among the alternatives there is exactly one alternative that we consider to be correct. This correct alternative is denoted by * . alternative ∈ is defined as the overall number of votes that receives, i.e.,
Given and , a single election based on approval voting is given by the relation ⊆ × where (, ) ∈ means that agent approves choice . Moreover, the score # of some # = |{ ∈ | (, ) ∈ }|. (that is: strictly more votes) than any other alternative, that is,
Finally, an alternative ∈ wins the approval vote if it receives strictly higher scores # >
max ′∈∖{} # ′. expression * =1 ∧
⋀︀ and 0 otherwise. 0 otherwise, whereas Probabilistic Framework. As usual, the described scenario is modeled as a random process. This process generates * as well as and is governed by a joint probability distribution P over the Bernoulli (i.e., {0, 1}-valued) random variables *1 , ..., * as well as 1 , ..., for all agents 1, ..., , ..., such that the values taken by these random variables represent the outcome of a voting event as follows: * is 1 if is the actual world state (i.e., = * ), and is 1 if the th agent voted for the th world state (i.e., (, ) ∈ )
In a next step, we state the fundamental assumptions necessary for the generalization of the CJT considered here. In the presentation of these assumptions, we let [* = ] denote the ∈∖{} *=0. The first assumption that needs to be imposed is that the agents do not influence each other in their decision whether or not to approve some choice.
A joint distribution satisfies agent approval independence if, conditioned on the actual world state, the decision to approve any given is made independently across all agents, i.e., for any , ∈ and any sequence 1, ..., of values from {0, 1} the following holds: P︁( ⋀︁ =1 =1 = | [* =] = ∏︁ P(︁ ︁)
︁) = | [* =] .
The second central assumption that we need to impose is that the agents are on average more likely to identify the correct alternative than any competing one. =1 =1 the following holds:
A joint probability distribution satisfies Δ-group reliability for some Δ > 0, if the probability to approve the true world state, averaged across all agents, is at least by Δ higher than the averaged probability for approving any other state, i.e., for every , ′ ∈ with ̸= ′ 1 ∑︁P(︁ =1 | [* =] ≥
︁)
For simplicity, we refer to a voting setting satisfying those two assumptions as I&R (for independent and reliable). As final definition for the probabilistic setting, we characterize the chance for the correct alternative to be identified through approval voting. choices, the approval vote worst-case success probability w,cs is defined by
Given a family of joint probability distributions for agents and a set of min P P∈ ∈ ︁( ⋀︁ ∑︀=1 > ∑︀
=1 † | [* = ] .
︁) †∈∖{}
With that, we can more formally restate the generalization of the CJT that is central to this work: wc,→s−−− →∞
Theorem 2 (Generalized CJT). In any I&R setting with fixed ≥ 2 and Δ > 0 holds
In words, for any independent and reliable setting with at least two alternatives and a positive Δ-value the worst-case probability for the correct alternative to be identified converges to 1 as the number of agents goes to infinity.
Beyond the convergence behavior in the infinite, it is possible to derive concrete guarantees in the finite. Exploiting Hoefding’s inequality [
[
Theorem 3. In a Δ-group reliable setting with choices, the worst case approval vote success probability is at least min whenever the number of agents is equal or higher than min ︁( Δ22 ln , 1 + (︀ 1 Δ2 − 1︀) )︁ , (1) where = 2 − 1 admissible error probability.
1 − min is the twofold ratio between the number of incorrect alternatives and the This bound on the required number of agents will play a crucial role in a later part of this paper, where it will be utilized to estimate the maximal permissible precision when aggregating probabilistic opinions through voting.
In essence, the CJT is a mathematical theorem that provides asymptotic guarantees for efectively identifying an underlying truth through voting. The theorem has been generalized to accommodate more realistic assumptions and enable practical estimates of the number of agents required to track the truth. In the subsequent sections, we establish the groundwork for the generalization that facilitates the aggregation of probabilistic beliefs by introducing a formal model for representing those beliefs.
In this section, we first introduce the imprecise probability model of an agent’s belief that will
be used in our setting. Second, we illustrate how imprecise probabilistic beliefs can be used
in decision-making. For this latter objective, we embed the underlying belief model into a
philosophical account for vagueness that is referred to as supervaluationism. The presentation
of both frameworks builds upon and extends the one in Karge 2023.
3.1. Probabilistic Beliefs
When it comes to representing probabilistic beliefs, the classical approach is to use a single
probability function to capture an agent’s confidence in various propositions. This function
assigns a real number between 0 and 1 to each proposition, reflecting the agent’s degree of
belief in that proposition. This approach can be defined as follows:
Definition 4 (Probability Function). A probability function P is a function P : 2Ω → R, satisfying
the probability axioms [
Here, Ω represents a set of possible worlds or states of afairs, and propositions are subsets
of Ω. However, when evaluating propositions with severe uncertainty, such as the example of
proposition A, which will serve as our running example, it becomes implausible to represent
belief states with a single probability function [
Example 1 (Proposition A). Global sea level will rise at least 1,5 meters until the year 2100 above the level of 2000.
The question arises: What precise probability should the agent assign to proposition A? The
traditional approach of representing an agent’s belief with a single probability function requires
a specific value to be given [
We refer to a specific set of probability functions as the agent’s representor, denoted by [
When experts evaluate statements such as proposition A, imprecise probabilities aim at
reflecting the unspecific nature of the evidence that experts typically receive. Intuitively, this is
achieved by considering all values that are not excluded by the evidence [
Convexity implies that for any two points in a set, the line segment connecting them is also
within the set. Under this assumption, an agent’s imprecise degree of belief can be represented
as intervals [
To summarize, we model an agent’s probabilistic beliefs using imprecise probabilities, where
the agent assigns an imprecise degree of belief (a sub-interval of the unit interval) to each
proposition, reflecting their confidence. Unlike precise probabilistic beliefs, this imprecise
model allows for a more natural representation of scenarios involving severe uncertainty. After
formalizing the model of an agent’s beliefs based on imprecise probabilities, we incorporate
imprecise probabilistic beliefs into supervaluationism, which provides a convenient framework
for handling various decision-making scenarios, including voting, based on this belief model.
3.2. Supervaluationism
Consider a vague predicate such as tall. Vague predicates can be made more precise by
introducing cutof points. For example, one person may consider being tall to mean a height of at
least 180cm, while another person may set the cutof at 185cm. Each cutof point represents
a precisification of the predicate [
In standard supervaluationism, a proposition is deemed determinately true if it is true
according to all admissible precisifications. Conversely, a proposition is considered determinately false
if it is false according to all admissible precisifications. Additionally, supervaluationism allows
for propositions to have no semantic value, meaning that if a proposition is true according to
some, but not all, admissible precisifications, it is considered indeterminate whether it is true
[
In an extension of supervaluationism, referred to as modified supervaluationism (MSV), the
notion of propositions being determinately true is replaced by propositions being predominantly
true:
Definition 8 (Predominantly True). A proposition is predominantly true if it is true according to
a relative majority of admissible precisifications [
Both in the standard and modified supervaluationistic frameworks, imprecise probabilities
can be efectively captured by appropriately defining admissible precisifications [
To establish a coherent model of belief, the combination of supervaluationism and imprecise probabilities must provide a sound concept for comparing the confidence an agent has in diferent propositions.
Definition 9 (Predominant Confidence) . Given two propositions, A and B, an agent is considered to be predominantly more confident in proposition A than in proposition B if a greater proportion of elements within the agent’s imprecise degree of belief satisfy the condition () > ().
This definition allows us to compare the relative confidence an agent has in diferent propositions based on their respective probability values within the agent’s imprecise degree of belief.
Observe that determining whether an agent is predominantly more confident in a proposition
than in another involves measuring a potentially uncountable number of probability values,
which are the precisifications derived from the agent’s imprecise degree of belief. In order to
achieve this, we note that in our setting, admissible precisifications are confined to the unit
interval [
Example 3. Consider proposition A and its complement B, i.e. global sea level will not rise at least 1,5 meters until the year 2100 above the level of 2000. Suppose we have () = [0.4, 1] as our agent’s imprecise degree of belief. For those elements represented by (0.5, 1] it holds true that () > (). For those represented by [0.4, 0.5) we have () > (). Taking their Lebesque measure, we obtain () = 0.5 as well as () = 0.1. Thus, the agent is predominantly more confident in proposition A.
With the MSV interpretation of imprecise probabilities, our next objective is to incorporate this model into voting scenarios. Having defined the voting setting and established how imprecise probabilities can be integrated into modified supervaluationism, we now informally describe how agents vote based on their imprecise beliefs.
Definition 10. Given a set of alternatives = {1, . . . , } and set of agents = {1, . . . , }, agent approves alternative if the agent is predominantly more confident in that alternative than in its competitors.
However, to provide a more formal definition, we need to specify what constitutes an alternative in our setting. This will be addressed in the next section, where we incorporate voting based on imprecise beliefs into the CJT framework.
Upon establishing an agent’s imprecise beliefs and presenting an approach to voting based on those beliefs through modified supervaluationism, as well as delineating the CJT and its underlying voting setting with its probabilistic assumptions, we now integrate these concepts to incorporate imprecise probabilistic beliefs into the CJT voting setting. 4.1. The Alternatives To recap our main goal, we aim to determine the likelihood of a statement, such as proposition A, in our framework. We have a finite set of agents, each holding an imprecise probabilistic belief in that proposition. These beliefs will be aggregated through voting, where each agent votes for the alternative they are predominantly confident in. In the final step, the alternative that receives the most votes wins the election and represents the aggregated probabilistic assessment of the event, such as proposition A. Following this general aggregation procedure, each alternative represents a probabilistic assessment of the event by itself.
As discussed earlier a central assumption underlying the CJT is that one of the given alternatives is correct. What exactly does that mean in our setting where we aim at aggregating probabilistic opinions? A first idea is: an alternative represents a precise probability for an event to occur assuming an event has an underlying, objective probability. Then, given a set of alternatives where each represents a single probabilistic value, and given the imprecise degrees of beliefs of the agents, these agents vote for one or multiple of the single, precise values based on their beliefs.
However, in real-world applications, agents typically vote for a limited and finite set of alternatives. This introduces a problem with respect to an agent’s degree of belief, as probabilities can be arbitrarily precise. If we were to model alternatives as representations of single probabilistic values, the assumption that the correct probability for the event is among the set of alternatives chosen for an election would appear highly unrealistic. To address this issue, we model the alternatives themselves as imprecise probabilities, where each alternative represents an interval of probability values within which the correct probability for an event lies.
The formal modeling of alternatives is detailed in the next subsection and is drawing inspiration from the field of expert elicitation research.
Expert Elicitation. In typical expert elicitation frameworks, an expert’s belief in a proposition
is represented as a precise, subjective probability that characterizes their belief. However, recent
advancements have incorporated imprecise probabilities in the form of intervals of subjective
probabilities into the analysis of an expert’s competence [
One method for eliciting an agent’s imprecise probabilistic beliefs is known as the Linear Programming Imprecise Probabilities Model (LPIPM). Given some unknown real-valued continuous random variable , applying LPIPM, aims at eliciting an expert’s prior distribution of [20]. For this purpose, the expert is asked to internally evaluate and then announce minimum and maximum values for that she considers plausible. These values define an interval of the form [, ] within which the expert believes the true value of the variable lies, with the understanding that the expert assigns a probability of zero to the true value of lying outside this interval [20].
In the original LPIPM approach, the interval [min, max] is further divided into subintervals. The expert then evaluates these subintervals through pairwise comparisons, aiming to obtain a more precise estimate of the agent’s imprecise degree of belief by eliminating subintervals that the expert deems less likely compared to others. This iterative process allows for a refinement of the expert’s belief, resulting in a more nuanced understanding of their imprecise probabilities.
In our setting, we adopt the idea of partitioning an interval into subintervals, but we apply it
to the level of alternatives that experts can vote for in an election. More precisely, in an election
where experts have to evaluate the likelihood of an event such as proposition A, we regard as
simplest set of alternatives the (theoretically excluded, but conceptually noteworthy) case where
we have = 1 with 1 = [
The division of the unit interval into subintervals is reminiscent of a common practice in the evaluation of probabilistic forecasts known as the binned probability ensemble (BPE) technique. In BPE, a set of forecasts is used to divide the real line into several bins, assuming that the true forecast probability falls within one of the bins [21]. In our case, we apply a similar concept to the unit interval, considering each subinterval as an alternative to vote on, referred to as a bin. Voting for Bins. To define the alternatives in the approval vote as bins, representing subintervals of the unit interval as probability values, we proceed to formally outline how agents vote based on their imprecise degrees of belief and MSV.
In the original MSV framework, the notion of predominant confidence determined the proposition in which an agent was most confident among multiple propositions, considering their imprecise degrees of belief in each proposition.
However, in our voting setting, we focus on an agent’s imprecise degree of belief for a single proposition. The agent’s confidence must then be evaluated with respect to multiple bins by comparing them to her imprecise degree of belief. These bins, in turn, represent probabilistic assessments for the same proposition as the imprecise degree of belief. As the imprecise degree of belief provides an estimate of the likelihood of a proposition being true but not for another probabilistic estimate for the same proposition, we extend the concept of predominant confidence to this scenario. Intuitively, we model predominant confidence in probabilistic estimates for the same proposition as the imprecise degree of belief as the largest agreement between a bin and the imprecise degree of belief. Since both the bin and the belief are probability intervals, we quantify agreement as the largest Lebesgue measure between the intersection of the degree of belief and each bin.
Definition 11 (Predominant Confidence - Bins) . Let A be a proposition, () = [, ] be an agent’s imprecise degree of belief in A, and let [− 1, ), = 1, 2, ..., 2 be 2 bins defined on the unit interval reflecting probability values for A to occur. Given two bins 1 and 2, we say that an agent is predominantly more confident in 1 if the intersection of () and 1 is of greater Lebesque measure than the one of () and 2. That is, (() ∩ 1) ≥ (() ∩ 2).
Consider the following example: Example 4. Suppose there are only two bins for proposition A with 1 = [0, 0.5) and 2 = [0.5, 1] and let () = [0.3, 0.9]. We then have ()∩1 = [0.3, 0.5) and ()∩2 = [0.5, 0.9]. This results in (() ∩ 1) = 0.2 as well as (() ∩ 2) = 0.4. Thus, the agent is predominantly more confident in the second bin.
Finally, we can define how agents vote based on their imprecise degrees of belief: Definition 12. Let = 1, ..., be a set of alternatives where each represents a bin of the form [− 1, ). Moreover, let 1, ..., be a set of agents and let represent a single election. We say that an agent votes for an alternative if she is predominantly confident in that alternative. That is, if ((() ∩ ) ≥ (() ∩ )) for all ̸= then (, ) ∈ .
Note that, if the agent is predominantly confident in multiple alternatives (i.e., their intersections are of equal Lebesque measure), she votes for all of them. In particular, if an agent reports the whole unit interval as her imprecise degree of belief, she votes for all alternatives in the approval vote. 4.2. The Probabilistic Assumptions Having formally defined how agents vote based on their imprecise degree of belief as well as having specified what constitutes an alternative in our setting, we can now combine the imprecise model of belief with the CJT.
To recap, we revisit the previously described joint probability distribution: *1 , ..., * as well as 1 , ..., for all agents 1, ..., , ..., . While the distribution remains the same, our interpretation of how the random variables take their values changes. Recall that 1, ..., represent subintervals of the unit interval and let * be the true objective probability of an event to occur such that * ∈ for exactly one . The values taken by these random variables represent the outcome of a voting event as follows: * is 1 if is the bin containing the actual world state (i.e., * ∈ ) and 0 otherwise, whereas is 1 if the th agent voted for the th bin (i.e., ((() ∩ ) ≥ (() ∩ )) for all ̸= ) and 0 otherwise.
With this interpretation of the random variables in the joint probability distribution, as given by Definition 12, we can directly utilize the probabilistic assumptions we mentioned earlier: agent approval independence and Δ-group reliability. Let’s provide an intuitive rephrasing of these notions in the language of our setting. First, agent approval independence translates to the idea that for any bin and any agent, the fact that one agent, , is predominantly more confident in a particular bin, , than in all other bins, does not influence whether a diferent agent, , is also predominantly more confident in . In other words, the confidence of one agent in a bin does not afect the confidence of another agent in the same bin. Second, according to Δ-group reliability, on average, the agents are by the value of Δ more likely to be predominantly more confident in a particular bin than in any other bin. This notion captures the idea that the agents exhibit a certain level of consistency in their confidence assessments, favoring the bin containing the correct probability value over others by a small margin of Δ. Reasons for Approval Voting. The interpretation of the voting setting and probabilistic framework outlined above highlights why approval voting is a suitable choice.
Let’s consider a simpler voting rule like majority voting, as in the original CJT. In majority voting, the number of alternatives is limited to two, and agents can vote for only one of them. This voting rule is already unsuitable for our purposes due to its first restriction. Since each alternative corresponds to a bin, which is a subinterval of equal Lebesgue measure on the unit interval, using majority voting would result in a coarse-grained probabilistic aggregation method that only considers two bins, namely [0, 0.5) and [0.5, 1].
Now, let’s explore a setting that allows for any finite number of alternatives, but with the restriction that agents can only vote for one alternative, known as plurality voting. Recall that an agent votes for an alternative if the Lebesgue measure of the intersection between that bin and the agent’s imprecise degree of belief is the largest. Suppose we aim for reasonable high precision, such as 5%, which translates to 20 bins: 1 = [0, 0.05), 2 = [0.05, 0.1), ..., 20 = [0.95, 1]. Let’s assume that the correct probability value falls within 2, meaning * ∈ [0.05, 0.1). If we restrict the voting setting to allow agents to vote for only one bin while simultaneously requiring a reasonably high Δ value, we inadvertently impose a limitation on the agents’ imprecise degree of belief. In order for an agent to vote for the correct alternative in the plurality setting (i.e., 2 in this case), their imprecise degree of belief must have less overlap with the other bins compared to 2, ensuring that the Lebesgue measure of their intersection is not exceeded by the other bins. This restriction might exclude agents who abstain from making any judgment and report the whole unit interval as their imprecise degree of belief because they would be predominantly confident in all bins. To avoid this artificial restriction, we allow agents to vote for any subset of the finite set of alternatives, which aligns with the concept of approval voting.
Additionally, the embedding presented here ofers a small advantage in addressing a conceptual counterargument sometimes raised against the CJT framework. Critics argue that it is unrealistic to assume that the correct alternative is always among the options provided to the agents [22]. However, in our setting, this assumption is naturally satisfied. The correct probability for an event to occur must be an element of the unit interval, which is covered by the partitions into bins. Therefore, the embedding ensures that the correct alternative is always within the set of alternatives considered by the agents.
Having embedded imprecise probabilistic beliefs into the CJT framework, we may moreover invoke the aforementioned estimate on the number of agents required to guarantee a prescribed minimal success probability for the correct alternative to win. Furthermore, given the established correspondence between the number of subintervals of the unit interval and the number of alternatives in the approval vote, which, in turn, reflects the desired precision, we subsequently illustrate how to derive an estimate on the maximal permitted precision. 4.3. Estimating the maximal permitted precision.
In typical scenarios involving the combination of probabilistic expert opinions, the number
of voters on the expert board is often known beforehand. Examples of such studies include
Kriegler et al. 2009, who investigated the beliefs of 42 climate scientists, and Recchia et al.
2021, who analyzed the forecasting capacities of 140 experts, including epidemiologists and
statisticians, regarding COVID-19 outbreaks in the UK. Since the size of expert committees
in real-world applications is often fixed, it is not only important to estimate the minimum
number of agents necessary to ensure a prescribed minimal success probability, as previously
discussed [
Observe that excluding Δ = 1 for the Chebychev-Cantelli bound may come as a surprise, especially considering that this bound performed extremely well in the original setting of Karge and Rudolph 2022 where Δ = 1. Specifically, when estimating the minimal number of agents for any probability of success, it gave the intuitive result that only one agent is needed in the approval vote. However, in our setting, we are providing an upper bound on the number of alternatives. When Δ = 1, we expect any precision to be allowed since every agent exclusively votes for the correct bin. Thus, in this scenario, there is no finite upper bound as we permit an arbitrary number of bins, which means the number of alternatives can be infinite. Interestingly, when Δ = 1 and we allow for an unlimited number of bins, we obtain a special case where agents vote not for bins but for precise probability values.
Given this, both bounds provide an estimate for the maximum value of . In contrast to Theorem 3, we now take the maximum of both bounds to determine this maximum value.
Theorem 4. In a Δ-group reliable setting where Δ ∈ (0, 1) with agents, the worst case approval vote success probability is at least min whenever the number of alternatives is equal or lower than ( (1− min)
1 (2− 2 Δ2 ) + 1, (1− min)(1+(− 1)Δ2) + 1).
2(1− Δ2) (2)
The result obtained from the bounds yields the maximum number of alternatives that guarantee a prescribed minimal success probability min for a specific number of agents and a given average reliability assessment represented by Δ (excluding Δ = 1). This directly translates into the maximal allowed precision in percentage, denoted by . We define as the proportion of the unit interval covered by a subinterval, given by = 100 . Therefore, the closer is to 100, the more imprecise the result of the election will be, while a value closer to 0 indicates higher precision. For example, when = 20, we have a precision of = 5%, and when = 2, we have = 50%. It is important to note that when we mention that precision increases, it means that the result of the aggregation procedure will be more precise, even though the precision value in percentage decreases.
Furthermore, it is worth observing that neither of the two bounds dominates the other for all values. This is illustrated in Figure 1, where we consider a fixed average group reliability and number of agents and plot the maximum number of permitted bins against the prescribed minimal success probability. Specifically, for a small number of agents but extremely high Δ-values, the Cantelli bound provides a good estimate, while the Hoefding bound performs better for small and moderate Δ-values.
To gain a more nuanced understanding of the maximal allowed precision, we refer to Figure 2. This graph illustrates the number of bins as a function of moderate to relatively high Δ-values (0.2-0.5) and small to moderate-sized expert groups (10-100) for a fixed probability of success of min = 0.9. Several observations can be made from the graph: First, small expert groups require relatively high average competence levels to achieve meaningful precision values for aggregation. For instance, in order to achieve a precision below 10%, more than 100 agents are needed for a moderate competence margin of Δ = 0.3. Second, there is a range of values, including = 50 and Δ = 0.3, where the maximum permitted number of bins is less than two. These values indicate that the average competence margin and number of agents are not suficient to realize the prescribed success probability. In such cases, we state that the calculated precision is not available since the CJT requires at least two alternatives. Third, the permitted precision grows extremely rapidly once the number of experts and their respective competencies are suficient to achieve the prescribed success probability. For instance, a group of 150 experts with a competence margin of 0.4 is already suficient to realize a precision of 0.01%. Increasing the number of experts to 200 for the same Δ value results in a precision of 0.0002%.
In this work, we embedded imprecise probabilistic beliefs into a generalization of the Condorcet Jury Theorem that allows for approval voting and heterogeneous competence levels. For this purpose, we combined the epistemological account of imprecise degrees of belief as well as their interpretation in the supervaluationinstic theory of vagueness with a voting setting where each alternative represents an interval of probability assessments (a bin) for the same proposition as the imprecise degree of belief. Furthermore, we established a direct correspondence between the number of bins in the voting process and the maximal permitted precision during aggregation. By solving existing bounds for the number of alternatives, we were able to provide an estimate for the allowed precision in the aggregation procedure. Moving forward, it would be valuable to compare the performance of voting-based aggregation of imprecise probabilistic beliefs with traditional methods of probabilistic opinion pooling. The latter typically involves combining expert opinions through weighted averaging of probabilities to approximate the objective probability of an event. In contrast, our approach assumes the objective probability lies within one of the bins, providing guarantees for hitting that specific bin with a certain probability.
Jonas Karge was supported by the Bundesministerium fur Bildung und Forschung (BMBF, Federal Ministry of Education and Research) in the Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI). [20] A. A. Silva, F. C. de Souza, A protocol for the elicitation of imprecise probabilities., in:
ISIPTA, volume 1, 2005, pp. 315–321. [21] J. L. Anderson, A method for producing and evaluating probabilistic forecasts from ensemble model integrations, Journal of climate 9 (1996) 1518–1530. [22] M. Pivato, Epistemic democracy with correlated voters, Journal of Mathematical Economics 72 (2017) 51–69. [23] G. Recchia, A. L. Freeman, D. Spiegelhalter, How well did experts and laypeople forecast the size of the covid-19 pandemic?, PloS one 16 (2021).