This article discusses a design pattern for building belief networks for application domains in which causal models are hard to construct. In this approach we pursue a modular belief network structure that is easily extended by the users themselves, while remaining reliable for decision support. The Hypothesis Management Framework proposed here is a pragmatic attempt to enable analysts and domain experts to construct and maintain a belief network that can be used to support decision making, without requiring advanced knowledge engineering skills.
Since their introduction by Kim and Pearl [
Well-known examples of belief networks such as the
Alarm [
Besides o ering criminal investigators a method to use belief networks, also some e ort should be focused on preventing bias arising in analyses. Where much attention goes into getting unbiased and accurate prior probabilities, in this paper we are more concerned with any bias within the topology; the choice of variables included in the model. When an analyst looks for support for a certain hypothesis, it is easy to get into a so-called tunnel view in which contradicting evidence and alternate hypotheses are neglected. When a plausible alternative perspective is missing in the model, a potential bias is present yet invisible. It seems impossible to always exclude such a bias, but applying certain strategies in the design of a belief network may lead to more balanced and less biased models. Among others, the following strategies might be considered. Firstly, di erent domain experts can add an alternative point of view to the same model. Secondly, each domain expert can work independently on a di erent hypothesis or counter-hypothesis. And nally, domain experts can design reusable templates that are not tailored for a speci c case, but for generic classes of cases. Whatever combination of strategies may work best to avoid a bias, the case for a exible and modular way 1The software packages HUGIN Expert, Netica and GeNie are respectively found at: http://www.hugin. com, http://www.norsys.com/netica.html and http:// genie.sis.pitt.edu/ to design belief networks to aid better decision making should be apparent.
To maximize its applicability in real world applications
the following two requirements should be met:
Various systematic techniques are available to guide
the modeling of a belief network in a systematic
manner. Many of these generate a belief network by
translation of another type of model, e.g. ontologies [
This led to the development of the hypothesis management framework (HMF) at TNO. This design pattern enables a domain expert to independently create and maintain a belief network, and an analyst to evaluate evidence in a criminal investigation. The HMF is a modular belief network structure that is easily expandable by the users themselves, while remaining reliable for decision support. The HMF adds a layer of abstraction to the belief network, so the belief network can be kept hidden from the user. Multiple users can independently modify or extend the model based on his or her domain knowledge. The HMF ensures that all parts of the model remain a coherent whole, suitable for consistent reasoning. 2
While devising the HMF design pattern we had one
particular goal in mind: to enable the design of
modular and extendible Bayesian models for users that are
no Bayesian specialist. Once a rst version of a model
has been developed, it should be easily extended and
maintained later-on. It is likely that the set of
variables as well as the subjective priors for conditional
probability tables require regular revisions as the eld
of investigation changes over time. Therefore it should
be possible to reconsider the set of variables, without
having to elicit all of the priors on each change of the
model. The need for multiple revisions of a
developing model was addressed by the AI group at the
University of Kentucky in [
These requirements are indeed very common, and generally accepted as basic requirements in the context of system development. We think, however, that they are hard to comply with without the use of a generalized framework. 3
The HMF places each variable of interest within a prede ned structure, as visualized in Figure 4(c). Furthermore it prescribes which variables may be instantiated with evidence, and for some variables the content of conditional probability tables. All variables must be categorized by the user in hypotheses, indicators or information sources. Each type has its own place and role within the topology of the belief network: 1 Hypotheses are statements of which we would like to get a posterior probability distribution. In general, hypotheses are unobserved. The user can specify unconditional priors for each hypothesis, or use a uniform nondiscriminative distribution instead. As an option, one can add alternative hypotheses to represent known facts that explain observed indicators in an other way than existing hypotheses. 2 Indicators are statements related to hypotheses.
Knowledge of an indicator helps to reveal the states of related hypotheses. Indicators describe events that are dependent on the occurrence of one or more hypotheses. Causal relations between hypotheses and indicators are not always obvious, or present at all. Indicators are assumed to be `caused' by hypotheses, not the other way around. For each relation between an indicator and a hypothesis, a domain expert should specify conditional probabilities for that speci c relation. 3 Information Sources are used to express the reliability of sources related to an indicator, when the user does not want to enter 'hard evidence'. For instance, an information source may be a report, a sensor or a person. An indicator can be associated to multiple information sources.
Although common, it is not necessary for an arc in a
belief network to imply causality. The HMF makes use
of this freedom by taking a more abstract perspective
on the relations between variables of interest. The
structure is based on the relatively simple notion of
hypotheses and indicators. Indicators may all support
or contradict any of the hypotheses, but the indicators
themselves are assumed independent of one another.
Hypotheses are independent (root nodes) and typically
have many children. Quite similar, so-called 'naive
Bayes' structures [
If more structure is desired, this modeling style may
be applied in a recursive fashion in which a
hypothesis may have sub-hypotheses, who are modeled in an
similar way. This is not demonstrated in this article.
It is good practice to use a causal model whenever
possible [
When there is no evidence for an indicator, the
combining indicator variable (i0) will resemble the
posterior probability of the original indicator (i) by taking
the average probability of all intermediate variables.
This information is useful to predict the likelihood of
unobserved indicators or for selecting the most in
uential unobserved indicator. Equation 1 is used to
construct the conditional probability table of the
combining indicator variable. Note, that the HMF does not
use a logical function (e.g. OR/MAX, AND/MIN or
XOR). Logical functions that assume independence of
causal in uence, in a discrete or noisy variant, have
been long in use [
An analyst processes information sources and selects
evidence for indicators to support or contradict
hypotheses. For analysts the GUI (Figure 2) shows
indicators in a foldable tree-like structure. The indicators
are organized in categories and sub-categories. For
each indicator the analyst can choose a state (e.g. true
or false) based on observed evidence. If the analyst is
uncertain about an observation, the analyst is given
the ability the express the reliability of each
information source for that speci c indicator. This requires
a prior probability for both positive observations and
false positives, given that the indicator is a boolean.
Domain experts evaluate the conditional probabilities
of an indicator given an hypothesis, and choose prior
probabilities for hypotheses. The GUI should enable a
domain expert to construct and maintain a list of
indicators and hypotheses. A domain expert is
responsible for relating indicators to hypotheses in a sensible
manner, and assign conditional probabilities to each
relation. Figure 3 shows how this may be presented to
the domain expert. There is a column for each
hypothesis. Assuming only booleans are used, the respective
column requires only two elicited priors: one prior for
the likelihood of observing the indicator given the
hypothesis is true, and another for when the hypothesis
is false. Qualitative descriptions or frequencies can be
more e ective than probabilities [
To explain how the HMF may be used and why we
have chosen this speci c topology, we will now
discuss three di erent models based on a civil case
concerning a car accident. The rst is a logical causal
model by Prakken and Renooij [
Given evidence for: pulled position, driver's testimony, passenger drunk and crash, it is highly likely that the passenger pulled the handbrake ( 100%). Since the evidence against the passenger explains away the car crash, it is unlikely that the crash was caused by lost control of the vehicle after speeding through the Scurve (0:1%). The bayesian belief network comes to the same conclusion as the causal map of Prakken and Renooij.
When we model the same case using the HMF, we get a radically di erent topology (Figure 4(c)) that does not resemble the causal map of Prakken and the belief network of Huygen. Both claims are modeled as hypotheses in the HMF model: accident caused by speeding and passenger pulled handbrake of moving vehicle. These hypotheses correspond to similarly named predicates in Figure 4a and probability variables in Figure 4b. Uniform probability distributions were used as priors for these hypotheses. We use indicators to support our beliefs in the hypotheses, these are: driver's testimony directly after incident, handbrake in pulled position after incident, passenger had drunk alcohol, observed yawmarks of sliding vehicle and observed skidmarks beyond the curve.
By choosing di erent priors, the evidence for tire marks is now usable. Some intermediate variables that relate facts with the two hypotheses are no longer in use. These are locking of wheels and loss of control over vehicle. The information source of passenger had drunk alcohol is undisclosed. Suppose the source was a guest at the party, than the reliability of this testimony is represented by an information source variable (Figure 4(c)).
Given the available evidence, we get a high likelihood for the passenger pulling the handbrake of the moving vehicle ( 100%). The propability for speeding is much lower ( 27%), and therefore far less convincing. All three approaches can adequately model the case and derive equally sensible conclusions. Abductivelogical reasoning over a causal map explains the logical correctness and contradictions of propositions. The advantage of a Bayesian approach is that by quantifying in uence, it is able to give insight in what hypothesis is most credible as well as the relevance of evidence. The models of Prakken, Renooij and Huygen are based on a causal map. Although HMF follows a di erent approach to the construction of belief networks, and therefore uses a rather di erent topology, it does derive the same conclusions. 5
Extendibility as well as modularity are important requirements. The models by Prakken and Huygen are 'static' models in the sense that they were designed to model one single case with a xed set of evidence and hypotheses. This is feasible when consensus has been developed on all aspects of the case. However, supporting decision making at an earlier stage requires a high level of exibility. The HMF was developed to facilitate decision making when the set of evidence (or indicators) and hypotheses is still evolving and a constant topic of discussion. Models designed with the HMF are exible, meaning that a model is decomposable into independent modules. So that each module can be maintained or extended by a di erent domain expert. This section will discuss issues that concern the extendibility of models developed with the HMF. These issues will be illustrated by extending the existing models from the previous section.
We have pursued extendibility by modular independence of the elicited priors. When an indicator is added to the model, the only priors to elicit are those for the intermediate nodes of that speci c indicator. Priors that were elicited before do not have to be reconsidered. The same holds for adding hypotheses. We will illustrate this by considering an additional hypothesis for the car accident case. Suppose the driver pulled the handbrake of the moving vehicle. If the driver was under in uence of alcohol, that would have also in uenced the driving behavior and therefore the likelihood of speeding as well as the possibility of pulling the handbrake of the moving vehicle. In all three models we would have to add and update existing prior knowledge.
To add the alternative hypothesis to the logical model of Prakken and Renooij a proposition is needed for the new hypothesis, and another to represent the possibility that the driver was under the in uence of alcohol. These additional causal relations are highlighted in red in Figure 4(a). Together, these additions extend the existing set of 12 rules with 6 more. When we add similar variables and relations to the belief network of Huygen, we need to specify new conditional probability tables for locking of wheels, handbrake in pulled position and driver's testimony. Furthermore, we would have to replace the prior probability distributions of speeding through S-curve with a new conditional probability table. These changes comprise the elicitation of 34 new priors that substitute 14 previously elicited priors.
To add to the HMF model the hypothesis driver pulled the handbrake of the moving vehicle, requires a new column in the model in Figure 4. The possibility of the driver being under the in uence of alcohol is modeled as an indicator, which adds a new row to the model. Table I shows how many elicited priors are required for extending the models. The extensions of the HMF model comprise only 22 elicited priors, all 36 existing priors remain unchanged. This makes HMF considerably cheaper to extend than the belief network of Huygen. The original causal model of Prakken is even simpler to extend. That model, however, lacks quantitative support for probabilistic inference.
As the car accident case shows, the HMF is tolerant to extensions. Figure 5 shows the general e ect of adding hypotheses and indicators to a model by outlining the maximum number of elicited priors. While the total number of parameters grows exponentially when more hypotheses are added, the amount of elicited priors grows in a linear fashion. The gure assumes the worst case in which each indicator is associated to all hypotheses. Although the model assumes boolean variables and two priors for each intermediate variable would su ce, it is assumed that all priors for intermediate variables are elicited as well as a prior probability distribution for each hypothesis. Note that we have excluded all other parameters that require elicitation such as variable names and state de nitions. As a reference Figure 5 includes the number of priors of Hail nder (3741), Alarm (752), the original belief network of Huygen (44) and the HMF model from Section 4 (36). The extensions proposed in this Section were excluded from the HMF model.
As mentioned in Section 3 indicators are modeled by intermediate variables and one combining variable. The more hypotheses are associated to an indicator, the more probabilities of intermediate variables will have to be combined. On each extension the combining variable gets an extra parent, and as a consequence its conditional probability table (CPT) doubles in size. In the HMF an averaging function has been chosen as the preferred option for these CPTs. By default, the CPT of a combining variable e ectively takes the average posterior distribution of all intermediate variables (Equation 1).
Arguably, one might nd a logical OR-function [
Another potential problem that is associated with
ORtables is the asymmetric in uence of an indicator:
positive observations have less impact than a negative
observation. This is shown in Figure 6(b)). Where
observed indicators will only have marginal impact
on hypotheses when observed true, the impact on
intermediate variables of an indicator observed as false
is deterministic and therefore usually stronger. It is
likely that the user will be unaware of these e ects.
This makes the model relatively vulnerable to errors
in the priors. Therefore, we advice to use Equation 1
as the default method. Other methods for
constructing CPTs of combining variables may hinder extending
(a) The likelihood of an indicator.
(b) The impact of evidence for an indicator.
To evaluate the outcomes of HMF belief networks we
have translated the Asia belief network, as introduced
by Lauritzen and Spiegelhalter in [
We will use abbreviations that correspond to the rst character of each variable. The original model is shown in Figure 7 (left), the HMF version of Asia is shown on the right. In the HMF model of Asia we distinguish hypotheses: fb; l; tg, indicators: fs; v; x; dg and intermediate nodes: fsb; sl; vt; xb; xl; xt; db; dl; dtg. The variable T bOrCa is missing from the HMF model, which in the original belief network combines the probabilities of tuberculosis and lung cancer with a logical OR function has become obsolete.
In the HMF model of Asia, the prior information for the indicators is speci ed separately for each associated hypothesis. This assumes that the in uence of e.g. lung cancer on dyspnea is una ected by bronchitis. The following probabilities will have to be elicited from a domain expert, when using HMF on Asia. Unconditional priors for each hypothesis: P (b), P (l), P (t) and conditional priors for all intermediate nodes: P (sbjb), P (sljl), P (vtjt), P (xbjb), P (xljl), P (xtjt), P (dbjb), P (dljl), P (dtjt).
The Asia model uses only boolean variables and
therefore only one probability for each hypothesis has to be
elicited and two for each association of an indicator
with a hypothesis. For Asia this gives a total of 21
probabilities. In this case the priors for the
hypotheses and intermediate nodes were derived from the joint
probability table of the original Asia belief network.
We computed the posteriors of the hypotheses for all
possible scenario's of evidence for the indicators. In
each of these scenarios each indicator was either
observed or not. Note that we instantiate the
intermediate nodes for evidence, rather than the combining
variables. As mentioned in Section 3 an indicator is
represented by both intermediate variables and a
combining variable. The conditional probability table of
the combining variable is implemented by Equation
1, whereas the elicited priors are stored in the
intermediate variables. Instantiating only the combining
variable would undervalue those elicited priors.
The results are shown in Table II. For each
indicator and hypothesis, the table shows the average and
maximum absolute di erence in posteriors, as well as
the Jensen-Shannon divergence [
The current HMF design pattern is extendible and modular. In our opinion the HMF succeeds in its purpose. We have con dence that HMF comes as a relief to those application domains that so far have been relatively underequipped with practical decision support tools, due to the lack of 'hard and solid' domain knowledge that can be used as a basis for probabilistic models.
The arrangement of the HMF supports a working method which deals with tunnel-view in a well considered manner. The HMF will not explicitly reduce or prevent bias occurring within the topology of a model. However, it o ers the possibility to use certain strategies during the design of a model which lead to more balanced and thus less biased models. Using such strategies will enlarge the awareness about tunnel-view (and bias) and as such may partly prevent it. Although the requirements of reliability and usability are not validated by domain experts and analysts, several issues concerning these requirements have been discussed in this paper. The Asia example shows that posteriors via a HMF model can be quite similar to those derived via a belief network based on causality. The issues that we have encountered so far in applying belief networks for criminal investigations have been addressed in this paper. However, it is a continuous e ort to further improve the HMF. 8
One of the complementary wishes of the authors involves a bias measurement combined with automated commentary that highlights useful missing evidence. By calculating how discriminative the indicators and the evidence is to each hypothesis and counterhypothesis, we can evaluate whether tunnel vision may be present. It can also be used to investigate the added value of collecting evidence for unobserved indicators. One way of getting this information is by simulating evidence and evaluate the posteriors of all hypotheses. Since the maximum potential impact of an indicator may only occur at a certain combination of evidence for other indicators, the simulation should consider all possible combinations of evidence for all unobserved indicators. This may be a costly operation. Alternatively one may derive the maximum impact directly from the conditional probability tables of the variables, and use message passing to investigate the maximum potential impact of each indicator.
The naive structure of a HMF belief network may in some occasions not capture the targeted e ects. In those cases we would like to extend the HMF model with constraining variables that model the synergistic e ect between indicators (or in between hypotheses). We have not been able to test such mechanisms in realistic cases so far. Therefore these need further investigation to test the feasibility of adding constraints, and whether the implications of such mechanisms violate the extendibility and modularity.
The HMF has been applied on several study cases based on real data by the authors. In the foreseeable future it is expected that domain experts will work with this framework. Their experience will be very useful for validating the usability and reliability of this method, and for nding ways to further improve it.