In this paper we present a new method (EBBN) that aims at reducing the need to elicit formidable amounts of probabilities for Bayesian belief networks, by reducing the number of probabilities that need to be specified in the quantification phase. This method enables the derivation of a variable's conditional probability table (CPT) in the general case that the states of the variable are ordered and the states of each of its parent nodes can be ordered with respect to the influence they exercise. EBBN requires only a limited amount of probability assessments from experts to determine a variable's full CPT and uses piecewise linear interpolation. The number of probabilities to be assessed in this method is linear in the number of conditioning variables. EBBN's performance was compared with the results achieved by applying both the normal copula vine approach from Hanea & Kurowicka (2007), and by using a simple uniform distribution.
In this paper we consider the case of deriving a
discrete conditional probability distribution for a node of
a Bayesian belief network based on expert judgement.
There are many issues to consider when deriving a
conditional probability distribution via expert judgement
elicitation. The expert assessors will for example use
simplifying heuristics when assessing probabilities to
avoid too complex mental reasoning. These heuristics
might lead to biased assessments. In addition experts
might also be subject to various types of motivational
biases. There is the problem of how to select the
appropriate experts for the elicitation task and how
to properly prepare them for formulating the
assessments (e.g. motivating and training them). There is
the choice of which method to use to elicit the
probabilities: e.g. a probability-scale, probability-wheel,
gamble-like or adverb-probability matching method?
The number of probabilities that need to be specified
for a node can grow large very easily. For a node with
three states that has a parent node with also three
states, 6 probabilities need to be specified to
determine its conditional probability table (CPT). An
additional second and third parent node with three states
would consequently require a table of 18 and 54
probabilities, and so on. Apart from the huge amounts of
time it would take to assess all the probabilities for
large CPTs, it can also be questioned to what extent
assessors can be expected to coherently provide the
probabilities at the level of detail required (see e.g.
There are two ways in which the elicitation task for
discrete BBNs can be relieved. The first is to make it
easier for the assessors to provide the probabilistic
assessments required. Van der Gaag, Renooij, Witteman,
Aleman & Taal (1999) aim to achieve this by
transcribing the conditional probabilities and using a scale
containing both numerical and verbal anchors. But the
effort needed to assess a full CPT using this method,
though reduced, is still exponential in the number of
conditioning variables. The second option for relieving
the elicitation burden is to reduce the number of
probabilistic assessments to be made. This can of course be
achieved by reducing the number of conditions
(parent nodes) or the number of states of the variables, but
such reductions will often be undesirable (e.g. leading
to loss of detail needed to inform a decision).
The Noisy-OR model, originally introduced by
In this paper we develop and evaluate a methodology,
EBBN, for deriving a node’s CPT in the general case
that the states of the node are ordered and the states
of each of its parent nodes can be ordered with
respect to the influence these parent nodes have on this
node of interest. In this method only a (small) part
of the CPT- describing the joint influence of the
parents in contrast with the marginal influence elicited in
the Noisy-MAX model - is elicited. The conditional
probabilities that are not directly elicited are derived
using an interpolation method based on the ranks of
parent node states. The number of probabilities to be
assessed is linear in the number of parent nodes. Since
the method approximates the probabilities that are
not directly assessed, it will contain inaccuracies. Like
While testing this method three relevant alternatives
were presented.
Very closely related to our method is the method
presented by
In the next section we will introduce our alternative elicitation method for BBNs, EBBN, which is aimed at reducing the elicitation burden. In Section 3 we discuss when we can regard an approximation of a CPT to be ‘good’, providing the means to assess the performance of the proposed method and compare it with the copula vine approach (Section 4). In the final section we present our conclusions and suggestions for future work. 2
We regard the problem of expert assessment of the
probability distribution of a discrete variable Xc (a
node in a BBN), conditional on a set of two or more
discrete variables, which we will denote with pa(Xc)
(the set of parent nodes). We require (1) the values of
Xc to be ordered, and (2) that the values of each of
the elements of pa(Xc) can be ordered such that each
of these variables have either a positive or a negative
influence on Xc. By stating that Xk ∈ pa(Xc) has a
positive influence on Xc, denoted by S+(Xk, Xc), we
mean that observing a higher value for Xk does not
decrease the likelihood of higher values of Xc,
regardless of the values of the other variables pa(Xc) \ Xk.
We take assignment a = {xj , . . . , xu} to be an
instantiation of the set of pa(Xc) = {Xj, . . . , Xu}. Formally
we define Xk ∈ pa(Xc) having a positive influence on
Xc, S+(Xk, Xc), as
P (Xc > xc | xk,n, a¬k) ≥ P (Xc > xc | xk,o, a¬k). The definition of a negative influence, S−(Xk, Xc), is completely analogous and would involve only reversing the above inequality.
We define a conditioning variable Xk ∈ pa(Xc) to be positive dominant, if the following two (sets of) assignments of pa(Xc) lead to the same probabilities: (I) all assignments of pa(Xc) in which Xk is in its most favourable state for high values of Xc and (II) the assignment in which each Xl ∈ pa(Xc) is in its most favourable state for higher values of Xc (i.e. all Xp ∈ pa(Xc) with S+(Xp, Xc) are at their highest value, and all Xn ∈ pa(Xc) with S−(Xn, Xc) are at their lowest value).
So if a positive dominant parent is in its most favourable state for high values of Xc, then, regardless of the states of the other parents, Xc will have the same probabilities as when conditional on the assignment in which all parent nodes are in their most favourable state. Negative dominant variables are defined analogously.
In the remainder of this section we will first discuss the
assessments needed from the expert for the derivation
of the CPT of Xc. We will then show how to obtain
the CPT from these assessments and end the section
with an illustrative example of the method, taken from
the Hailfinder network
It is assumed that the assessor has confirmed that the values of variable Xc are ordered and that the assessor can order the values of each of the variables Xk ∈ pa(Xc) such that (s)he judges either S+(Xk, Xc) or S−(Xk, Xc) to hold. Then the following assessments are required to determine the CPT for variable Xc with conditioning variables pa(Xc)1:
1As mentioned in Section 1 we will not discuss here how these assessments can be best elicited from the assessor 1. (ordering). For each of the conditioning variables Xk ∈ pa(Xc): order the values of Xk such that Xk has either a positive or a negative influence on Xc. Fix and record this ordering of the values and the nature of the influence. 2. (typical probabilities). For each of the values xc of Xc: (a) determine the assignment pa(Xc) = axc such that the probability P (Xc = xc | axc) is as large as possible. (b) assess the probabilities P (Xc | axc ).
Due to dominance of one of the conditioning variables axc,min (axc,max ) need not be unique, where xc,min (xc,max) is the lowest (highest) value of Xc. Therefore axc,min (axc,max ) is by default set to be the assignment in which all the conditioning variables are in their most favourable state for low (high) values of Xc, referred to as aneg (apos). 3. (weights). For each of the conditioning variables Xk ∈ pa(Xc), assess P (Xc = xc,max | aneg,k+) and P (Xc = xc,min | aneg,k+), where xc,max and xc,min are resp. the maximum and minimum value of Xc, and aneg,k+ is the assignment of pa(Xc) in which Xk is in its most favourable state for high values of Xc, and all Xl ∈ pa(Xc) \ Xk are in their least favourable state for higher values of Xc. 4. (dominance). For each of the conditioning variables Xk ∈ pa(Xc), determine whether Xk has either no, a positive or a negative dominance over Xc. 2.2
The derivation of the CPT of Xc is done in a two-step procedure, using the assessments from Section 2.1. In the first step we will express the probabilities P (Xc) as a function of an influence factor i. In the second step individual and joint influence factors are determined for all assignments of pa(Xc), which are then used to derive the probabilities P (Xc) from the functions of step 1.
The influence factor i is an expression of the positiveness (or negativeness) of the joint influence of the parent variables pa(Xc) on Xc. It is a function of values of the parent variables, with 0 ≤ i(a) ≤ 1. We set i(aneg) = 0, where pa(Xc) = aneg is the assignment in which all the conditioning variables are in their most favourable state for low values of Xc (see item 2, Section 2.1). And, at the other extreme, i(apos) is set to 1. For all other assignments i ∈ (0, 1). If assignment a2 has a strictly more positive influence on Xc than a1 - i.e. P (Xc > xc | a2) > P (Xc > xc | a1) for all xc - then the influence factor corresponding to a2 should be bigger than the influence factor corresponding to a1.
We make use of two separate influence factors: the individual influence factor ik for each conditioning variable Xk ∈ pa(Xc) and the joint influence factor ijoint. As will become more clear later on, ik will contain information about the influences exercised by each of the parent variables individually, ijoint about the ‘general tendency’ of all of the parent influences together. We determine the individual influence factor ik for Xk ∈ pa(Xc) as follows:
rank(xk) − 1 ik(xk) := rank(xk,max) − 1
rank(xk,max) − rank(xk) rank(xk,max) − 1 if S+(Xk, Xc) if S−(Xk, Xc) (1) where the rank of the smallest value is set to be 1 and xk,max is the highest value of Xk. So if Xk ∈ {low, medium, high} has a positive influence on Xc, we find that ik(low) = 0, ik(medium) = 0.5 and ik(high) = 1. The joint influence factor ijoint for assignment pa(Xc) = a is derived as: ijoint(a) :=
X {k:Xk∈pa(Xc)}
X {k:Xk∈pa(Xc)} ik(xk) · (rank(xk) − 1) (rank(xk,max) − 1) (2) Verify that indeed ijoint(aneg) = 0 and ijoint(apos) = 1. Also note that the individual influence factor of Xk, ik, is equal to the joint influence factor ijoint if pa(Xc) = {Xk}, i.e. if the set of parents of Xc merely consists of Xk.
Step 1. Estimating P (Xc) as a function of joint influence factor ijoint In this step P (Xc = xc) is estimated as a function of joint influence factor ijoint, for each value xc of Xc. For this we use the orderings determined at item 1 in Section 2.1, and the assignments axc and probabilities P (Xc = xc | axc ) assessed at 2. We construct the piecewise linear functions fxc : [0, 1] → [0, 1] through the points (ijoint(axc ), P (Xc = xc | axc )). It can be easily verified that using these linear interpolations ensures that Pxc fxc (i) = 1, i.e. the sum of the probabilities of occurrence of the different values of Xc equals unity for all i ∈ [0, 1]. Coherency requires that if xc,n > xc,m, also ijoint(axc,n ) > ijoint(axc,m ). In Figure 1 an example is given for how this estimation of P (Xc) as a function of ijoint might look like. In this example Xc ∈ {low, medium, high}, and the points (ijoint(axc ), P (Xc = xc | axc )) are assessed as in Table 1.
1 In Step 1 we obtained P (Xc) for all possible values of ijoint via linear interpolation, and equation (2) provides us with an expression for ijoint for all assignments pa(Xc) = a. We can now determine P (Xc | a) via P (Xc | ijoint(a)) from the functions fxc of Step 1. Yet this mapping from assignments a for the conditioning variables pa(Xc) to an expression ijoint is not unique. Suppose pa(Xc) = {Xj, Xk, Xl}, Xj and Xl both exercise the same type of influence (positive or negative), and Xj , Xk, Xl ∈ {low, medium, high}, then ijoint({medium, medium, medium}) = ijoint({low, medium, high}) = 0.5. As pointed out earlier, ijoint is an expression for the ‘general tendency’ of the influence of the conditioning variables. It does not take into account the (dis)agreement of the influences of each of the conditioning variables individually. To account for both the ‘general tendency’ and the individual influences of the conditioning variables, we calculate for each conditioning variable Xk ∈ pa(Xc) the average of the probabilities Pk(Xc | a) over the interval (min(ik(xk), ijoint(a)), max(ik(xk), ijoint(a))). An example of this average, denoted with Pk(Xc | a), is illustrated in Figure 2. We derive the desired probabilities P (Xc | a) as the average over the distributions Pk(Xc | a). Or actually the weighted average
X k:Xk|pa(Xc) P (Xc | a) = wk · Pk(Xc | a), (3) since one parent could have a stronger influence on Xc than another. For the same relative change in states, i.e. changes in states resulting in the same absolute change in each of the individual influence factors, the probabilities for Xc might change more for one parent variable than for another. Therefore we calculate the weight wk for each parent Xk ∈ pa(Xc), in the following way: To a large extent the choice of the base assignment aneg and the probabilities P (Xc = xc,max | aneg,k+) and P (Xc = xc,min | aneg,k+) to derive the weights is arbitrary. Even though, we feel the choice for these assignments is one of the most natural choices that can be made. And, more importantly, we feel these assignments are relatively easy for assessors to consider and assess. It is of course possible to use more assessments to determine the weights more accurately. However, we feel that the possible added value does not weigh against the burden of the extra elicitation effort needed.
We derive the desired probabilities P (Xc | pa(Xc) = a) by rewriting (3) using (1), (2) and (4), as Z imax,k
f (i) · di
X k:Xk|pa(Xc) wk · imin,k imax,k − imin,k P (Xc | pa(Xc) = a) = where imax,k (5) imin,k = min(ik(xk), ijoint(a)), = max(ik(xk), ijoint(a)) and f (i) = fxc,min(i), . . . , fxc,max(i) .
Finally, we deal with negative and positive dominance of one of the parent variables in the following straightforward way: for all the assignments ad in which a negative (positive) dominant parent is in its least (most) favourable state for high values of Xc, we set P (Xc | ad) to be equal to P (Xc | aneg) (P (Xc | apos)). We will now demonstrate the method by means of an illustrative example. 2.3 Illustrative example from the Hailfinder network wk = For the derivation of the weights we have taken the situation in which each parent is in its least favourable state for high values of Xc, aneg, as the base. We use the probabilities P (Xc = xc,max | aneg,k+) and P (Xc = xc,min | aneg,k+) assessed at item 3 in Section 2.1. Each δ+ and δ− now expresses the changes in the k k probabilities of resp. the highest and lowest state of Xc, if the one parent Xk is set to its most favourable state for high values of Xc whilst leaving the other parents in their least favourable states (aneg,k+). We obtain the weights from these δ’s via the normalisations (4).
For the variable CompPlFcst we have the fully subjectively specified CPT, consisting of 4·33 ·3 = 324 probabilities. In this example we derive the required assessments for EBBN, as specified in Section 2.1, from this CPT, but treat them as if they were directly elicited: 1. (ordering). The ordering of the states of the variables is given in Figure 3, where the highest states are on top. For the conditioning variables we find the following influences: S−(AreaMeso ALS,CompPlFcst); S+(CldShadeOth,CompPlFcst); S−(CldShadeConv,CompPlFcst); S+(Boundaries,CompPlFcst). AreaMeso_ALS
StrongUp WeakUp Neutral Down
CldShadeOth
Cloudy PC Clear
CldShadeConv
None Some Marked
CompPlFcst
IncCapDecIns LittleChange
DecCapIncIns Boundaries
None Weak Strong Based on the assessments and (4), we find the weights: wAreaMeso ALS = 0.459, wCldShadeOth = 0.303, wCldShadeConv = 0.165 and wBoundaries = 0.073. We can now use (5) to derive the desired probabilities and find P (CompPlFcst | pa(CompPlFcst) = aexpl) = {0.17, 0.32, 0.51}. We can derive the full CPT of Xc (consisting of 324 probabilities) in the same way, requiring in this case only 17 probabilities to be assessed. When we look up the probabilities in de original CPT, we find P (CompPlFcst | pa(CompPlFcst) = aexpl) = {0.20, 0.32, 0.48}. The probabilities estimated with the methodology are in this case ‘not far off’. Yet, before we can assess how well our method approximates the directly assessed probabilities, we first need to discuss how we can measure the quality of the approximation.
Assuming you have knowledge of the ‘true’
probabilities of a certain CPT, how can you assess the quality
of an approximation to that CPT? A measure to assess
the similarity between two (discrete conditional)
probability distributions, with possibly different support, is
the Jensen-Shannon divergence
take into account the context of the CPT, the belief
0.45 network. Both
parents.
Now we have all the information (containing only 17 probability assessments!) we need to derive all the 324 probabilities of the CPT of CompPlFcst.
By means of an example we calculate the
probabilities P (CompPlFcst | pa(CompPlFcst) = aexpl), where
aexpl ={AreaMeso ALS=Down, CldShadeOth=P C,
CldShadeConv=N one, Boundaries=Strong}. For these
parent node states we find the individual influence
factors: iAreaMeso ALS (Down) = 1, iCldShadeOth (P C) = 12 ,
iCldShadeConv (N one) = 0 and iBoundaries (W eak) = 0,
and a joint influence factor ijoint(aexpl) = 49 . So we
see in this case that the individual influence factors of
the parents give a diverse picture, two are very
negative (0), one is between negative and positive ( 12 ) and
one is very positive (1). This is reflected by the joint
influence factor, which has a very average value (0.44),
expressing no general tendency of the parent influences
We have investigated the performance of the
methodology by applying it to a well-known belief network
from the literature that contained suitable large
subjectively assessed CPTs, and comparing its
performance with the copula vine approach from
We have searched for belief networks that contained nodes that satisfy the following requirements: • the CPT of the node was subjectively assessed, • the CPT of the node has to be reasonably challenging in size for elicitation from an expert. For this we decided the node needed to have two or more parents, and • the states of the node are ordered.
We found these networks are difficult to come by. This
is not surprising of course, since these networks would
require a huge elicitation effort. Practitioners would
usually try to avoid having to specify these large CPTs
because the elicitation process would be too time
consuming, the very problem we are aiming to deal with
in this article. In the BBN repository of the University
of Pittsburgh2 we found the Hailfinder network, which
does contain 7 nodes that satisfy our requirements.
For the Hailfinder network we created three
alternative versions. In each of these alternative versions we
replaced the CPTs of the 7 nodes satisfying the above
requirements (and kept the remaining CPTs as they
were). In the first alternative implementation these
CPTs were replaced with the approximations resulting
from the method introduced in this paper. We treat
the CPTs from the literature as the ‘true’ CPTs. We
assume that the probabilities needed for our
methodology would have been assessed as they are in these
CPTs and treat the difference between approximations
of the method and the corresponding CPTs as
inaccuracies of the approximation. So we have not tried to
find parameters for EBBN that minimise the distance
of the resulting CPT to the original, but have derived
the parameters needed from the original CPT.
The second alternative implementation has the
selected 7 CPTs derived according to the copula vine
approach
Finally we constructed a third alternative implementation of the Hailfinder network in which all altered CPTs consist of uniform distributions for all assignments of conditioning variables, to serve as a second benchmark. We have assessed the performance of our interpolation method and both benchmarks using the measures specified in Section 3. We found the variable InsInMt to be a positive dominant parent of CldShadeConv, and treated it as such in all three alternative implementations. 4.2
The results of the comparison of the ‘true’ versions of the selected 7 CPTs of the Hailfinder network with each of the three alternative derivations of these CPTs are given in Table 2 3. The table displays how the EBBN, copula vine and the uniform versions of the CPTs score on 9 performance measures. The first four measures are the measures m1.-m4. from Section 3 for the direct comparison between the ‘true’ and the approximating versions of the CPTs. The measures in the last five columns, m1.-m3., m5. and m6. from Section 3, consider posterior probabilities for each of the 7 selected nodes under all possible scenarios for neighbouring nodes, i.e. all logically possible combinations of states of neighbouring nodes (both parent and child nodes).
For the first seven measures in the table we have that the smaller the measure, the better the performance of the approximating CPT on that measure. For the last two columns to opposite holds: the higher the percentage, the better the performance. If a number is underlined in Table 2, this means that the corresponding approximating method (EBBN, copula vine or uniform) has the best performance for that measurement on that variable.
If we look at the underlined values in Table 2, it seems
that EBBN and the copula vine versions are of
comparable performance on all performance measures apart
from ‘unmatched 0/1’, on which EBBN performs best
on all CPTs. It is comforting to see that both EBBN
and the copula vine approach clearly perform better
than when the CPT is populated with merely
uniform distributions. Further investigation reveals that
the EBBN method scores relatively well on the so
called ‘collector’ variables CombMoisture, CombVerMo
3The EBBN and copula vine versions of the Hailfinder
network (.xdsl format) can be obtained from the authors.
and CombClouds. These are nodes in the Hailfinder
network that “summarize information from different
sources about moisture, vertical motion and clouds,
respectively”
In this paper we have developed a method for deriving large conditional probability tables based on expert judgement, that can hugely reduce the number of assessments needed from the experts. The quantitative assessments needed from the experts are relatively easy to understand: the experts still need to assess only probabilities. We believe that the experts will also be capable of providing the qualitative judgements described in Section 2.1 at items 1, 2(a) and 4.
In order to evaluate the performance of EBBN we
applied it to a well-known belief network from the
literature, the Hailfinder network. EBBN’s performance
was compared with the results achieved by applying
both the normal copula vine approach from
In the development of the method, the application to a real-life example (the Hailfinder network) has proven very valuable. We would like to test the method on more examples. But, as noted before, because of cost and effort required to elicit large CPTs, these examples are difficult to find. Any help with finding more examples would be greatly appreciated.
It should be noted that the EBBN method does not always lead to a large reduction in the number of probabilities that need to be assessed. In fact, the method could even require more probabilities to be assessed than there are in the CPT. Roughly this occurs when the number of states of the variable for which the CPT is to be derived is greater than the number of conditions (i.e. the number of different assignments of the conditioning variables).
Finally we would like to remark that the interpolation used, in its current form, does not take into account synergetic effects that may exist between conditioning variables.
We are grateful to the Institute of Applied Mathematics of the TU Delft for allowing us to use their Unicorn and Uninet software to derive the normal copula vine versions of the 7 CPTs of the Hailfinder network.
t m s h so le g ir ik in
l
m ra e J
8 .
y rM is r e o V b b
M oD lo e
M b
v A sd th on t
s u O C c
e F d d l V B C C A C C C C
v yA sd th on ts
u O C c e in re v
M is r V M oD lo e o
M b
e F d d l
C a a P
m m ea u o o r op C C A C C C C
h h p om ldS lS m d o rM is r V M oD lou e e o
M b h
v yA sd th on ts
O C c m b b r
m m ea ifo o o r