<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Exploratory network analysis of large social science questionnaires</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Robert J. B. Goudie</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sach Mukherjee</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Frances Gri ths</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Statistics &amp;, Centre for Complexity Science, University of Warwick</institution>
          ,
          <addr-line>Coventry</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Statistics, University of Warwick</institution>
          ,
          <addr-line>Coventry</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Health Sciences Research Institute, University of Warwick</institution>
          ,
          <addr-line>Coventry</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>There are now many large surveys of individuals that include questions covering a wide range of behaviours. We investigate longitudinal data from the Add Health survey of adolescents in the US. We describe how structural inference for (dynamic) Bayesian networks can be used to explore relationships between variables in such data and present this information in an interpretable format for subject-matter practitioners. Surveys such as this often have a large sample-size, which, whilst increasing the precision of inference, may mean that the posterior distribution over Bayesian networks (or graphs) is concentrated on disparate graphs. In such situations, the standard MC3 sampler converges very slowly to the posterior distribution. Instead, we use a Gibbs sampler (1), which moves more freely through graph space. We present and discuss the resulting Bayesian network, focusing on depression, and provide estimates of how di erent variables a ect the probability of depression via the overall probabilistic structure given by the Bayesian network.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>INTRODUCTION</title>
      <p>Hypotheses of multifactorial causes of symptoms and
outcomes play an important role in the social sciences
and in public health. Regression-based approaches are
widely-used in these elds to explore such hypotheses.
A great deal of insight can be gained through such
approaches, but it is sometimes overly constraining to
x a particular quantity as the dependent variable,
especially if the goal is to explore the possibility of
unexpected relationships between the data. Instead, we
can consider a number of variables on an equal footing,
and study the possibility of unexpected relationships
in the data.</p>
      <p>
        Graphical models provide a statistical framework
within which the relationship between variables can
be studied. These models enable complex
multivariate distributions to be decomposed into simpler local
distributions. This can reveal a great deal about the
relationships between the variables, as well as provide
a statistical and computationally tractable description
of their (often large) joint distribution. The
decomposition is formed by the conditional independence
structure, which can be represented by a graph. The use of
graphs helps to make the interpretation of the model
simpler. In this paper, we focus on the structure of
the model, as given by the graph. We aim to make
inference about this using statistical model selection.
The structure of the model suggests how the di
erent components of the system interact, which may be
helpful in understanding the system as a whole. These
methods have been widely adopted in molecular
biology (
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ), and have been used in some areas of medical
sciences (
        <xref ref-type="bibr" rid="ref4">4</xref>
        ).
      </p>
      <p>Consideration of unexpected relationships between
factors requires datasets that incorporate a wide range
of topics. Such data is now widely available for
representative samples of populations in many countries,
and for many sub-groups of interest. Many of these
datasets are derived from surveys that are general in
scope, and are not collected to study any one
particular question. For example, in the US, the health
of the whole population is representatively sampled
annually for the Behavioral Risk Factor Surveillance
System (BRFSS) survey, and the Add Health study,
which we use here, followed a cohort of young
people from 1994 until 2008. Data from both of these
have been used in scores of studies, but these
commonly focus on one speci c aspect, often using the
data to evaluate existing hypotheses. Given the wide
scope inherent in the design of these studies and the
large samples available in many cases, it is possible to
broaden the scope of the analysis by considering richer
structures. In this paper, we discuss the potential that
such a more explorative approach yields. We do not
seek to make conclusive causal claims, but instead
suggest that a broader approach may uncover important
aspects that have been neglected.</p>
      <p>
        Our focus will be on depression among adolescents in
the US, drawing on data from the National
Longitudinal Study of Adolescent Health (Add Health). It is
estimated that around 1{6% of adolescents each year
are a ected by depression (
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ). The e ects of
depression in this age-group are wide-ranging (
        <xref ref-type="bibr" rid="ref7">7</xref>
        ), and
include the stigma associated with poor mental health
more generally (
        <xref ref-type="bibr" rid="ref8">8</xref>
        ). There is considerable evidence that
there are a wide range of causal factors for depression
amongst adolescents, spanning biological,
psychological and social domains. Understanding these causal
factors and separating them from the consequences of
depression has been recognised as an important aim
(
        <xref ref-type="bibr" rid="ref9">9</xref>
        ). Some of the relevant causal factors may interact
and the approach taken here accounts for this.
The remainder of this paper is organised as follows.
We rst introduce the AddHealth dataset and
describe the Bayesian network framework. Inference for
Bayesian networks is performed using Markov Chain
Monte Carlo (MCMC), but the large sample size of
the dataset we consider makes achieving convergence
di cult because the posterior distribution may be
concentrated on disparate graphs, and so we describe an
alternative sampler that has superior properties in this
situation. Whilst the PC-algorithm (
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ) has
properties that often make it attractive in such contexts,
we found that the results in this situation were not
robust (see Discussion). We then present and discuss
the results for the Add Health dataset.
2
2.1
      </p>
    </sec>
    <sec id="sec-2">
      <title>MATERIALS AND</title>
    </sec>
    <sec id="sec-3">
      <title>METHODS</title>
      <sec id="sec-3-1">
        <title>Add Health</title>
        <p>
          The data that we use are drawn from the National
Longitudinal Study of Adolescent Health (Add Health)
that explores health-related behavior of adolescents
(
          <xref ref-type="bibr" rid="ref12">12</xref>
          ) in the US. The questionnaire contains over 2000
questions that cover many aspects of adolescent
behaviours and attitudes. We consider the
representative sample of adolescents from Waves I and II of the
in-home section, and the parental questionnaire from
Wave I of the study. The analysis we perform is not
feasible when the data is not complete (see
Discussion), and so individuals with missing data were
removed from the study. Removing incomplete samples
leaves 5975 individuals in the study.
        </p>
        <p>
          Our measure of depression is a self-assessed scale based
upon the Centre for Epidemiologic Studies Depression
Scale (CES-D) (
          <xref ref-type="bibr" rid="ref13">13</xref>
          ). Two questions from the 20-item
scale are omitted from AddHealth, and two are
modi ed, and so we scale the score given by the available
questions (
          <xref ref-type="bibr" rid="ref14">14</xref>
          ). A Receiver Operating Characteristic
(ROC) analysis showed that thresholds of 24 for
females and 22 for males provided the best agreement
with clinical assessments of depression (
          <xref ref-type="bibr" rid="ref15">15</xref>
          ). We use
this threshold to create a binary indicator of
depression status.
        </p>
        <p>
          Many of the remainder of the variables that we
consider (Table 1) are drawn from the risk factors
described in the depression literature, and the mental
health literature more generally. A recent review (
          <xref ref-type="bibr" rid="ref8">8</xref>
          )
described a wide range of factors that are associated
with poor mental health in young people, including
gender, poverty, violence and the absence of social
networks in the local neighbourhood. The quality of
relationships with parents is also thought to be important,
especially with the mother (
          <xref ref-type="bibr" rid="ref16">16</xref>
          ), as are parental alcohol
problems (
          <xref ref-type="bibr" rid="ref17">17</xref>
          ) and parental discord (
          <xref ref-type="bibr" rid="ref16">16</xref>
          ). The
individual's use of alcohol, drugs, smoking and HIV/AIDS
are all also associated with depression (
          <xref ref-type="bibr" rid="ref18 ref19">18, 19</xref>
          ).
Physical exercise has been proposed in some studies as a
useful intervention for the management of depression,
but many of these studies have been deemed to be poor
quality (
          <xref ref-type="bibr" rid="ref20">20</xref>
          ).
2.2
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>Bayesian Networks</title>
        <p>Our study uses Bayesian networks to explore the
relationships between variables in the Add Health study.
Bayesian networks are a particular type of graphical
model that enable classes of probability distributions
to be speci ed using a directed acyclic graph (DAG).
A Bayesian network G is represented using a DAG
with vertices V = (V1; : : : ; Vp), and directed edges
E V V . The vertices correspond to the
components of a random vector X = [X1; : : : ; Xp]T , subsets
of which will be denoted by XA for sets A f1; : : : ; pg.
For 1 i; j p, we de ne the parents Gj of each
node Vj to be the subset of vertices V such that
Vi 2 Gj , (Vi; Vj ) 2 E. Specifying the parents of the
vertices determines the edges E of the graph G. We
denote by G the space of all possible directed acyclic
graphs with p vertices. We will use XGi to refer to the
random variables that are parents of Xi in the graph
G.</p>
        <p>The graph speci es that the joint distribution for X,
with parameters = ( 1; : : : ; p), can be written as
a product of conditional distributions p(Xi j XGi ; i),
given the variables XGi corresponding to the parents
of Xi in the graph.</p>
        <p>p(X j G; ) =
p
Y p(Xi j XGi ; i)
i=1
We will need to be able to evaluate the marginal
likelihood p(X j G) easily, and so we consider only a
conjugate analysis in which the conditional distributions
p(Xi j XGi ; i) are multinomial, with Dirichlet priors
p( i) for each i. In this case, the marginal likelihood
can be evaluated analytically. Suppose each Xi takes
one of ri values, and de ne qi as the number of levels
of the sample space of XGi , each element of which we
call a con guration. For each con guration j of XGi ,
let Nijk be the number of observations in which Xi
takes value k. We assume the Dirichlet priors for each
i, each with hyperparameters Ni0jk, are independent.
We de ne Nij = Prki=1 Nijk and Ni0j = Prki=1 Ni0jk,
and the local score p(Xi j XGi ) to be</p>
        <p>qi
p(Xi j XGi ) = Y
j=1</p>
        <p>(Ni0j ) Yri
(Nij + Ni0j ) k=1
(Nijk + Ni0jk) :
(Ni0jk)
The marginal likelihood can be shown to equal the
product p(X j G) = Qp</p>
        <p>
          i=1 p(Xi j XGi ) of these local
scores (
          <xref ref-type="bibr" rid="ref21">21</xref>
          ).
2.3
        </p>
        <p>Structural inference for Bayesian</p>
        <p>Networks
We aim to make inference about the DAG G, given
data X and so our interest focuses on the posterior
distribution Pr(G j X) on Bayesian networks. Under
the assumptions we have made, this can be written in
terms of the marginal likelihood p(X j G), and a prior
(G) for the Bayesian network structure.</p>
        <p>Pr(G j X) /</p>
        <p>
          p
(G) Y p(Xi j XGi )
i=1
The priors (G) can be chosen to encode domain
information (
          <xref ref-type="bibr" rid="ref3">3</xref>
          ). For the analyses in this paper, we choose
an improper prior (G) / 1 that is at across the
space of graphs.
        </p>
        <p>The posterior distribution Pr(G j X) is di cult
to evaluate, because cardinality of G grows
superexponentially in p. This motivates the use of
approximations to Pr(G j X), which are usually based on
Markov chain Monte Carlo (MCMC).
2.4</p>
        <p>Approximate inference for Bayesian</p>
        <p>
          Networks
The standard form of MCMC that is used for
structural inference for Bayesian networks is MC3 (
          <xref ref-type="bibr" rid="ref22">22</xref>
          ).
This is a Metropolis-Hastings sampler that explores G
by proposing to add or remove a single edge from the
current graph G. This sampler works surprisingly well
in many situations, but if the posterior distribution is
not unimodal, the local moves may fail to explore the
space fully because the sampler may become `trapped'
in one mode. This issue becomes more severe as the
sample size increases because the posterior
distribution becomes more concentrated. A natural approach
in such situations is to use the PC-algorithm (
          <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
          ),
which has been shown to be asymptotically consistent
(
          <xref ref-type="bibr" rid="ref23">23</xref>
          ), but we found in this case that the results were
not robust (see Discussion).
        </p>
        <p>
          Our analyses in this paper were performed using a
Gibbs sampler (
          <xref ref-type="bibr" rid="ref1">1</xref>
          ), which we found to converge rapidly
to its equilibrium state. A nave Gibbs sampler for
structural inference that proposes single-edge
additions and removals can easily be constructed, but
this sampler o ers no advantages over the analogous
MC3. This nave scheme, however, can be improved
by `blocking' together a number of components, and
sampling from their joint conditional distribution. In
theory, any group of components can be taken as a
block, but sampling from their joint conditional
distribution needs to be possible and, ideally,
computationally quick.
        </p>
        <p>For Bayesian networks, the most natural blocks are
those consisting of parent sets G1; : : : ; Gp. This is
natural because the marginal likelihood p(X j G) for
a graph G factorises across vertices into conditionals
p(Xj j XGj ) and these conditionals depend on the
parent set of the vertex. Therefore, since any graph G 2 G
can be speci ed by a vector G = (G1; : : : ; Gp) of parent
sets, the posterior distribution on Bayesian networks
G 2 G can be written as functions of G1; : : : Gp in the
following way.</p>
        <p>Pr(G1; : : : ; Gp j X) /
p
(G1; : : : ; Gp) Y p(Xi j XGi )
i=1
In the following, we will denote subsets of the vector
G = (G1; : : : ; Gp) by GA = fGk : k 2 Ag, and the
subset given by the complement AC = f1; : : : ; pg n A
of a set A will be denoted by G A = fGk : k 2 AC g.
In particular, the complete graph can be speci ed by
G = (G1; : : : Gp) = (Gi; G i) for any i 2 f1; : : : ; pg.
To be able to construct a Gibbs sampler using
parent sets, we need to nd their conditional
distribution, given the other parent sets G j =
fG1; : : : ; Gj 1; Gj+1; : : : ; Gpg. Parent sets Gj for
which G = (Gj ; G j ) is cyclic will have no
probability mass in the conditional distribution. Let Kj? be
the set of parent sets Gj such that G = (Gj ; G j ) is
acyclic. The conditional posterior distribution of Gj is
multinomial, with weights given by the posterior
distribution of G = (Gj ; G j ). When the cardinality of
Kj? is constrained (for example, by restricting the
maximum number of parents of each node) the conditional
posterior distribution for Gj 2 Kj? can be evaluated
1.0 ●●
0.8</p>
        <p>●
n20.6 ●
uR00..42 ●● ●●●●●●●●●●● ●●●●● ●● ●●●●●</p>
        <p>●●●●●●● ● ●● ●
0.0 ●●●●●</p>
        <p>MC3 sampler
● ● ● ●●
●
●
●
●
●
●●
●
●
●
●
●</p>
        <p>Gibbs sampler
●
●●●●●</p>
        <p>
          ●●●●
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Run 1
We can improve the speed of convergence of this
sampler by allowing pairs of parent sets to be sampled
together. At each step of the Gibbs sampler we
conditionally sample pairs of parent sets (Gj1 ; Gj2 ),
given the remainder of the graph G fj1;j2g. Parent
sets G fj1;j2g such that G = (Gj1 ; Gj2 ; G fj1;j2g) is
cyclic have no probability mass in the conditional
distribution. Let Kj?1;j2 be the set of pairs of parent
sets (Gj1 ; Gj2 ) such that G = (Gj1 ; Gj2 ; G fj1;j2g)
is acyclic. For (Gj1 ; Gj2 ) 2 Kj?1;j2 , the conditional
posterior distribution is multinomial, by analogy with
(
          <xref ref-type="bibr" rid="ref1">1</xref>
          ), with weights given by posterior distribution of
G = (Gj1 ; Gj2 ; G fj1;j2g).
        </p>
        <p>Pr(Gj1 ; Gj2 j G fj1;j2g; X)
= P</p>
        <p>
          Pr(Gj1 ; Gj2 ; G fj1;j2g j X)
(Gj1 ;Gj2 )2Kj?1;j2 Pr(Gj1 ; Gj2 ; G fj1;j2g j X)
Similarly, sets of three parent sets can be conditionally
sampled. Full technical details are presented in (
          <xref ref-type="bibr" rid="ref1">1</xref>
          ).
        </p>
        <p>
          Depressed (time point 1)
Didn't present to doctor (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Female ●
Didn't present to doctor (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Good health (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Good health (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Victim of violence (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Strong academically (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Drug user (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
●
●
        </p>
        <p>●
●
●
●
●● ●
●●●
●●
●●●
●●</p>
        <p>
          ●
The variables that we consider are detailed in
Table 1. As is common when using graphical models
(
          <xref ref-type="bibr" rid="ref24">24</xref>
          ), all of these variables were grouped, initially into
`Background', `Wave I' and `Wave II', and then
rened into whether the question asked about the
longor short-term, as shown in Table 2. These groups
dene constraints on the Bayesian networks that are
considered. Speci cally, no edges can be directed
backwards through the groups. Edges, however, are
allowed within groups. For example, no edge is allowed
to be directed into `Gender', and no edge can pass
backwards in time, for example, from Depression at
Wave II to Depression at Wave I. Additionally, no
edge can pass from a short-term variable to a
longterm variable, for example, from Depressed at Wave I
to Have HIV/AIDS at Wave I.
        </p>
        <p>We precomputed the local scores, and then drew
100,000 samples (the rst half of which were discarded
as burn-in) using the Gibbs sampler (Section 2.3),
which took 30 minutes (on a single core of a cluster
computer). The graph space was constrained such that
no node had more than 3 parents, to ensure Equation
1 could be evaluated.</p>
        <p>
          We ran 5 independent samplers, with disparate initial
states. This enables a simple test of convergence to
be performed that compares the posterior edge
probabilities obtained from each of the independent runs
(
          <xref ref-type="bibr" rid="ref25">25</xref>
          ). The agreement between runs can be examined
graphically by plotting the edge probabilities against
Been expelled (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Victim of violence (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Family bereavement (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Parents unhappy together (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Been expelled (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Been suspended (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Seen shooting (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Family bereavement (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Hisp/Latino
        </p>
        <p>
          Family poor (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>Black/Af Am</p>
        <p>
          Parent drinks (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Householder smokes (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Live with father (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Experiences prejudice (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Live with father (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Live with mother (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Live with mother (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Experiences prejudice (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          White
Parents aid decisions (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Mother warm/loving (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Mother warm/loving (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Alcohol (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Age
Victim of violence (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Strong academically (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Good health (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Exercises (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Have HIV/AIDS (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Strong academically (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          In physical fights (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          In physical fights (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>Learning disability</p>
        <p>
          Talks to neighbours (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Severely injured (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Talks to neighbours (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Exercises (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Depressed (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Severely injured (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
Skips school (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Depressed (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Drug user (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Didn't present to doctor (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Been suspended (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Seen shooting (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>Female</p>
        <p>
          Skips school (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Drug user (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Smoker (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Didn't present to doctor (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
Alcohol (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Smoker (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>
          Good health (
          <xref ref-type="bibr" rid="ref2">2</xref>
          )
        </p>
        <p>Asian/Pac Isl.</p>
        <p>
          Am Ind/Nat Am
Have HIV/AIDS (
          <xref ref-type="bibr" rid="ref1">1</xref>
          )
        </p>
        <p>
          Other race
each other (Figure 1). Mean Pearson correlation
coefcients between edge probabilities from pairs of runs
were 0:9999 0:0002 (standard deviation) for the Gibbs
sampler and 0:6322 0:0477 for MC3. The agreement
between the independent runs of the Gibbs sampler
gave us con dence in our results, in contrast to the
large disagreements between MC3 runs. In addition,
cumulative edge probability plots for each edge showed
regular excursions around the mean (
          <xref ref-type="bibr" rid="ref26">26</xref>
          ), and a
numerical diagnostic (
          <xref ref-type="bibr" rid="ref27">27</xref>
          ) monitoring the number edges
in the sampled graph also clearly suggested that su
cient samples had been drawn (R^ 1:0).
        </p>
        <p>The samples drawn using MCMC allow the posterior
distribution of Bayesian networks to be approximated.
In particular, the samples can be used to estimate the
posterior edge probability P (ejX) with e 2 E.
Figure 2 displays all edges with posterior probability of
at least 0.5.</p>
        <p>Our focus is on depression, the parents of which in
Figure 2 we observe are \Didn't present to doctor"
and \Gender". It important, however, to note that
the model does not say that these are the only factors
that are important. For example, \Drug user" at Wave
I is related to depression through \Didn't present to
doctor" at Wave I and II (Figure 2).</p>
        <p>
          This is shown in Figure 3, which gives the conditional
probability of being depressed at Wave 2 when a
particular variable is set to a speci c value. We see that
general health, violence, academic performance and
drug use all a ect the conditional probability of
depression at Wave II. Note that to compute this
probability, links from the parents of the variable in which
we `intervene' are removed; this is equivalent to the
`do-operator' in the terminology of Pearl (
          <xref ref-type="bibr" rid="ref28">28</xref>
          ).
The analysis reveals the interaction between the many
aspects of life that have an impact on depression. The
connection between the depression and its two parents
in Figure 2 have been previously discussed in the
literature. The importance of gender in depression is
particularly extensively documented in the literature
(
          <xref ref-type="bibr" rid="ref8">8</xref>
          ). The connection to a failure in seeking medical care
even when the individual thinks they should has also
been discussed in the literature, often in terms of poor
accessibility of health care services for young people
(
          <xref ref-type="bibr" rid="ref29 ref8">29, 8</xref>
          ). Several decades of research have revealed the
complex causation of depression in young people, as
suggested by this study (
          <xref ref-type="bibr" rid="ref8">8</xref>
          ).
4
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>DISCUSSION</title>
      <p>There is a large amount of information held in large
social science questionnaires. In this paper we have
examined a graphical model approach to inferring
structure amongst the variables in such questionnaires. In
contrast to the standard regression-based approaches,
a graphical model approach forgoes the need to specify
a particular variable as the response. Instead, a more
comprehensive estimate of the entire structure of the
underlying system can be obtained. Regression
approaches posit a particular conditional-independence
structure, while graphical approaches allow
consideration of more general structures.</p>
      <p>
        The limitations of this study include those of all
similar studies using observational data that are collected
for multiple audiences. These forms of data, including
the longitudinal data used here, do not permit strong
causal conclusions to be drawn. In particular there
may be important variables that we have not included
in the analysis. However, the results are consistent
with studies that have used other research approaches
including experimental designs. The connection
between an individual not seeking medical care when
they think they should and depression supports
current practice guidance in the UK (
        <xref ref-type="bibr" rid="ref30">30</xref>
        ) where there is an
emphasis on providing access to health care through
the school system rather than expecting young people
to seek health care themselves. Not seeking medical
care despite believing it should be sought is a
complex factor because it captures both barriers to getting
medical care within the individual, such as lacking
motivation to seek care, and barriers within the
individual's environment, such as poor access to care. This
may mean that the variable encapsulates a number of
di erent characteristics related to depression, and thus
may form a `marker' for depression. However, the use
of a form of the question \Has there been any time over
the past year when you thought you should get
medical care, but you did not?" as a screening question in
di erent contexts needs further consideration.
This method of analysis clari es the complexity of
depression and suggests why when using traditional
methods of analysis it can be di cult to clarify
whether or not factors, such as experiences in the
family, in the wider community and at school, impact on
the experience of depression for young people. It may
also suggest why interventions for prevention of
depression have not yet been demonstrated to be cost
e ective (
        <xref ref-type="bibr" rid="ref31">31</xref>
        ).
      </p>
      <p>
        We performed structural inference for the Bayesian
network using a Gibbs sampler (
        <xref ref-type="bibr" rid="ref1">1</xref>
        ), because MC3 did
not mix in a reasonable time. We have also found
(
        <xref ref-type="bibr" rid="ref1">1</xref>
        ) this algorithm to be superior to the REV sampler
(
        <xref ref-type="bibr" rid="ref32">32</xref>
        ), and it has the advantage of avoiding the need to
consider an order prior as required by order MCMC
methods (
        <xref ref-type="bibr" rid="ref33 ref34">33, 34</xref>
        ), which induces a bias that can only
be corrected exactly by NP-hard computation of a
correction factor.
      </p>
      <p>
        An alternative to the MCMC method used here is the
PC-algorithm (
        <xref ref-type="bibr" rid="ref10 ref11">10, 11</xref>
        ). This method is
computationally e cient and is asymptotically consistent.
However, to test whether the sample size available here is
su cient to reach the asymptotic regime, we applied
the PC-algorithm (without constraints) to 10 di
erent subsamples, each containing 90% of the data. We
found that these results di ered signi cantly, with a
mean 84 in structural Hamming distance between the
pairs of completed partially directed acyclic graphs
(CPDAGs) given for the subsamples.
      </p>
      <p>
        We used a Multinomial-Dirichlet model for the local
conditional distributions, which yields a closed-form
marginal likelihood. This model posits an entirely
general discrete distribution, allowing its form to be
guided by the data. However, the number of
parameters in the local distributions for this model increases
exponentially with the number of parents, which may
mean that overly-sparse models are preferred. This is
problematic when the sample size of the available data
is small, because models with many parameters cannot
be assessed adequately without a large dataset. The
large sample size of the dataset used here minimises
this issue, but it would nonetheless be worthwhile to
consider more compact parameterisations. However,
estimating such models (
        <xref ref-type="bibr" rid="ref35">35</xref>
        ) signi cantly increases the
complexity of the model space, which makes such an
approach computationally challenging in this setting.
For this paper, we removed samples with missing data.
It is possible to handle missing data formally, for
example by using structural EM (
        <xref ref-type="bibr" rid="ref36">36</xref>
        ), and similarly consider
latent variables (e.g. shared genetics driving both child
and parent behaviour). However, at present, doing so
whilst robustly exploring large model spaces remains
an open challenge. Tackling these computational and
inferential issues is a key area for future research.
      </p>
      <sec id="sec-4-1">
        <title>Experiences prejudice</title>
      </sec>
      <sec id="sec-4-2">
        <title>In physical ghts</title>
      </sec>
      <sec id="sec-4-3">
        <title>Didn't present to doc- 2 tor Severely injured 3</title>
      </sec>
      <sec id="sec-4-4">
        <title>Have HIV/AIDS</title>
      </sec>
      <sec id="sec-4-5">
        <title>Seen shooting</title>
      </sec>
      <sec id="sec-4-6">
        <title>Mother warm/loving Been suspended</title>
      </sec>
      <sec id="sec-4-7">
        <title>Been expelled Good health Talks to neighbours Age</title>
      </sec>
      <sec id="sec-4-8">
        <title>Live with mother Live with father Smoker Drinks alcohol</title>
      </sec>
      <sec id="sec-4-9">
        <title>Exercises</title>
        <p>Depressed
r Question
2 Interviewer, please con rm that R's sex is (male) female. (BIO SEX)
2 Are you of Hispanic or Latino origin? (H1GI4)
2 What is your race? [White] You may give more than one answer (H1GI6A)
2 What is your race? [Black or African American] You may give more than one
answer (H1GI6B)
2 What is your race? [American Indian or Native American] You may give more
than one answer (H1GI6C)
2 What is your race? [Asian or Paci c Islander] You may give more than one
answer (H1GI6D)
2 What is your race? [Other] You may give more than one answer (H1GI6E)
4 [If SCHOOL YEAR:] During this school year [If SUMMER:] During the
19941995 school year how many times HAVE YOU SKIPPED/DID YOU SKIP
school for a full day without an excuse? (H1ED2; H2ED2)
3 [If SCHOOL YEAR:] Students at your school are prejudiced [If SUMMER:] Last
year, the students at your school were prejudiced. (H1ED21; H2ED17)
4 In the past 12 months, how often did you get into a serious physical ght?
(H1DS5; H2FV16)
Has there been any time over the past year when you thought you should get
medical care, but you did not? (H1GH26; H2GH28)
Which of these best describes your worst injury during the past year? (H1GH54;
H2GH47)
2 Have you ever been told by a doctor or a nurse that you had... HIV/AIDS
(H1CO16D; H2CO19D)
3 During the past 12 months, how often did each of the following things happen?</p>
        <p>You saw someone shoot or stab another person. (H1FV1; H2FV1)
4 Most of the time, your mother is warm and loving toward you. (H1PF1; H2PF1)
2 Have you ever received an out-of-school suspension from school? (H1ED7;</p>
        <p>H2ED3)
2 Have you ever been expelled from school? (H1ED9; H2ED5)
3 In general, how is your health? Would you say... (H1GH1; H2GH1)
2 In the past month, you have stopped on the street to talk with someone who
lives in your neighborhood? (H1NB2; H2NB2)
5 Age at interview, computed from date of birth, and date of interview
(Constructed from IYEAR, IMONTH, IDAY, H1GI1Y, H1GI1M)
2 Indicator variable (Constructed from H1HR3A-T; H2HR4A-Q)
2 Indicator variable (Constructed from H1HR3A-T; H2HR4A-Q)
4 Frequency of smoking (Constructed from H1TO1/2/5; H2TO1/5)
4 Frequency and amount of drinking alcohol (Constructed from H1TO12/15/18;</p>
        <p>
          H2TO15/19/22)
3 Amount of exercise (Constructed from H1DA4/5/6; H2DA4-6)
2 Rescaled CES-D, following (
          <xref ref-type="bibr" rid="ref14">14</xref>
          ) (Constructed from H1FS1-18; H2FS1-18)
        </p>
      </sec>
      <sec id="sec-4-10">
        <title>Victim of violence Family bereavement 2 3</title>
      </sec>
      <sec id="sec-4-11">
        <title>Strong academically 4</title>
        <p>Drug user 2
Family poor 5
Parents unhappy to- 4
gether
Parent drinks 4
Householder smokes 3</p>
      </sec>
      <sec id="sec-4-12">
        <title>Has learning disability</title>
      </sec>
      <sec id="sec-4-13">
        <title>Parents aid decisions</title>
        <p>2
5</p>
        <p>Indicator variable (Constructed from H1FV2-6; (H2FV2-5)
Number of bereavements (Constructed from H1NM2/F2, H1FP24A1-5;
H2NM4/F4, H2FP28A1-3)
Quartiles (Constructed from H1ED11-4; H2ED7-10)
Indicator variable (Constructed from H1TO30/34/37/41; H2TO44/50/54/58)
Census Bureau measure of poverty (Constructed from H1HR2/3/7/8, PA55)
(Parent asked.) Do you and your partner argue/talk of separating? (Constructed
from PB19/20)
(Parent asked.) Number/frequency of drinks (Constructed from PA61/2)
(Parent asked.) Either parent or others in household smokes (Constructed from
PA63/4)
(Parent asked.) Does (he/ she) have a speci c learning disability, such as di
culties with attention, dyslexia, or some other reading, spelling, writing, or math
disability? (PC38)
(Parent asked.) How often would it be true for you to make each of the following
statements about fchild's nameg? fChild's nameg and you make decisions about
(his/ her) life together. (PC34B)</p>
      </sec>
      <sec id="sec-4-14">
        <title>Background</title>
        <p>Female
Age
Hisp/Latino
White
Black/Af Am
Am Ind/Nat Am
Asian/Pac Isl.</p>
        <p>Other race
Has learning dis.</p>
        <p>Wave I Long-term
Skips school
Experiences prejudice
In physical ghts
Didn't pres. to doctor
Severely injured
Have HIV/AIDS
Seen shooting
Mother warm/loving
Been suspended
Been expelled
Good health
Alcohol
Victim of violence
Family bereavement
Strong academically
Drug user
Family poor
Parents unhappy togth.
Parents aid decisions
Wave I Short-term
Househol. smokes
Smoker
Live with mother
Live with father
Parent drinks
Talks neighbours
Exercises
Depressed</p>
      </sec>
      <sec id="sec-4-15">
        <title>Wave II Long-term Seen shooting Alcohol Drug user</title>
        <p>Mother warm/loving
Have HIV/AIDS
Family bereavement
Experiences prejudice
Been expelled
Been suspended
Victim of violence
In physical ghts
Strong academically
Didn't pres. to doctor
Skips school
Severely injured
Good health</p>
      </sec>
      <sec id="sec-4-16">
        <title>Wave II Short-term</title>
        <p>Smoker
Live with mother
Live with father
Talks neighbours
Exercises
Depressed</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Goudie</surname>
            ,
            <given-names>R. J. B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mukherjee</surname>
            ,
            <given-names>S. M.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>An E cient Gibbs Sampler for Structural Inference in Bayesian Networks</article-title>
          .
          <source>CRiSM Working Paper 11- 21</source>
          (Dept. of Statistics, University of Warwick).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2004</year>
          ) Science,
          <volume>303</volume>
          ,
          <issue>5659</issue>
          , 799{
          <fpage>805</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Mukherjee</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Speed</surname>
            ,
            <given-names>T. P.</given-names>
          </string-name>
          (
          <year>2008</year>
          <source>) Proc Natl Acad Sci USA</source>
          ,
          <volume>105</volume>
          ,
          <issue>38</issue>
          , 14313{
          <fpage>14318</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Acid</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>de Campos</surname>
            ,
            <given-names>L. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fernandez-Luna</surname>
            ,
            <given-names>J. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rodr guez</surname>
          </string-name>
          , S.,
          <string-name>
            <surname>Rodr</surname>
            <given-names>guez</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>J. M.</given-names>
            and
            <surname>Salcedo</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. L.</surname>
          </string-name>
          (
          <year>2004</year>
          ) Artif Intell in Med,
          <volume>30</volume>
          ,
          <issue>3</issue>
          , 215{
          <fpage>232</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Costello</surname>
            ,
            <given-names>E. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mustillo</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Erkanli</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Keeler</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Angold</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2003</year>
          ) Arch Gen Psych,
          <volume>60</volume>
          ,
          <issue>8</issue>
          , 837{
          <fpage>844</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Costello</surname>
            ,
            <given-names>E. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Erkanli</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Angold</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Child Psychol</surname>
          </string-name>
          <string-name>
            <surname>Psych</surname>
          </string-name>
          ,
          <volume>47</volume>
          ,
          <issue>12</issue>
          , 1263{
          <fpage>1271</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Thapar</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Collishaw</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Potter</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Thapar</surname>
            ,
            <given-names>A. K.</given-names>
          </string-name>
          (
          <year>2010</year>
          )
          <string-name>
            <surname>Br Med</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <volume>340</volume>
          ,
          <year>c209</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Patel</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Flisher</surname>
            ,
            <given-names>A. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hetrick</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>McGorry</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2007</year>
          ) Lancet,
          <volume>369</volume>
          ,
          <issue>9569</issue>
          , 1302{
          <fpage>1313</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Barnett</surname>
            ,
            <given-names>P. A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Gotlib</surname>
            ,
            <given-names>I. H.</given-names>
          </string-name>
          (
          <year>1988</year>
          ) Psych Bull,
          <volume>104</volume>
          ,
          <issue>1</issue>
          , 97{
          <fpage>126</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Spirtes</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Glymour</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Scheines</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2000</year>
          )
          <string-name>
            <surname>Causation</surname>
          </string-name>
          , Prediction, and
          <string-name>
            <surname>Search</surname>
          </string-name>
          (The MIT Press, Cambridge, MA).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Korb</surname>
            ,
            <given-names>K. B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Nicholson</surname>
            ,
            <given-names>A. E.</given-names>
          </string-name>
          (
          <year>2011</year>
          )
          <article-title>Bayesian Arti cial Intelligence (CRC Press, Boca Raton</article-title>
          , FL).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Harris</surname>
            ,
            <given-names>K. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Halpern</surname>
            ,
            <given-names>C. T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Whitsel</surname>
            ,
            <given-names>E. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hussey</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tabor</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Entzel</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Udry</surname>
            ,
            <given-names>J. R.</given-names>
          </string-name>
          (
          <year>2009</year>
          )
          <article-title>The National Longitudinal Study of Adolescent Health: Research Design</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Radlo</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>1977</year>
          ) App Psych Meas,
          <volume>1</volume>
          ,
          <issue>3</issue>
          , 385{
          <fpage>401</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Goodman</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>1999</year>
          )
          <string-name>
            <surname>Am J Pub Health</surname>
          </string-name>
          ,
          <volume>89</volume>
          ,
          <issue>10</issue>
          , 1522{
          <fpage>1528</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Roberts</surname>
            ,
            <given-names>R. E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewinsohn</surname>
            ,
            <given-names>P. M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Seeley</surname>
            ,
            <given-names>J. R.</given-names>
          </string-name>
          (
          <year>1991</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Am Acad Child Adolesc Psych</surname>
          </string-name>
          ,
          <volume>30</volume>
          ,
          <issue>1</issue>
          , 58{
          <fpage>66</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Holt</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Buckley</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <article-title>and</article-title>
          <string-name>
            <surname>Whelan</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2008</year>
          )
          <article-title>Child Abuse</article-title>
          &amp; Neglect,
          <volume>32</volume>
          ,
          <issue>8</issue>
          , 797{
          <fpage>810</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Obot</surname>
            ,
            <given-names>I. S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Anthony</surname>
            ,
            <given-names>J. C.</given-names>
          </string-name>
          (
          <year>2004</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Child Adolesc Subst Abuse</surname>
          </string-name>
          ,
          <volume>13</volume>
          ,
          <issue>4</issue>
          , 83{
          <fpage>96</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Brown</surname>
            ,
            <given-names>R. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewinsohn</surname>
            ,
            <given-names>P. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seeley</surname>
            ,
            <given-names>J. R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>E. F.</given-names>
          </string-name>
          (
          <year>1996</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Am Acad Child Adolesc Psych</surname>
          </string-name>
          ,
          <volume>35</volume>
          ,
          <issue>12</issue>
          , 1602{
          <fpage>1610</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Battles</surname>
            ,
            <given-names>H. B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wiener</surname>
            ,
            <given-names>L. S.</given-names>
          </string-name>
          (
          <year>2002</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Adoles Health</surname>
          </string-name>
          ,
          <volume>30</volume>
          ,
          <issue>3</issue>
          , 161{
          <fpage>168</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Larun</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nordheim</surname>
            ,
            <given-names>L. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ekeland</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hagen</surname>
            ,
            <given-names>K. B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Heian</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <article-title>Cochrane Database Syst Rev, 3</article-title>
          ,
          <fpage>CD004691</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Heckerman</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Geiger</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Chickering</surname>
            ,
            <given-names>D. M.</given-names>
          </string-name>
          (
          <year>1995</year>
          ) Mach Learn,
          <volume>20</volume>
          , 197{
          <fpage>243</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Madigan</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          and York,
          <string-name>
            <surname>J. C.</surname>
          </string-name>
          (
          <year>1995</year>
          )
          <article-title>Int Stat Rev</article-title>
          ,
          <volume>63</volume>
          ,
          <issue>2</issue>
          , 215{
          <fpage>232</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Kalisch</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and Buhlmann,
          <string-name>
            <surname>P.</surname>
          </string-name>
          (
          <year>2007</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Mach Learn</surname>
          </string-name>
          <string-name>
            <surname>Res</surname>
          </string-name>
          ,
          <volume>8</volume>
          , 613{
          <fpage>636</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Cox</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wermuth</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>1996</year>
          )
          <article-title>Multivariate Dependencies Models, Analysis and Interpretation (Chapman</article-title>
          &amp; Hall, London).
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Robert</surname>
            ,
            <given-names>C. P.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Casella</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          (
          <year>2004</year>
          ) Monte Carlo Statistical Methods (Springer, New York).
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mykland</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>1998</year>
          )
          <article-title>Statistics</article-title>
          and Computing,
          <volume>8</volume>
          ,
          <issue>3</issue>
          , 275{
          <fpage>286</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Gelman</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Rubin</surname>
            ,
            <given-names>D. B.</given-names>
          </string-name>
          (
          <year>1992</year>
          ) Statistical Science,
          <volume>7</volume>
          ,
          <issue>4</issue>
          , 457{
          <fpage>472</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Pearl</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2000</year>
          )
          <article-title>Causality: Models, Reasoning</article-title>
          , and
          <string-name>
            <surname>Inference</surname>
          </string-name>
          (Cambridge University Press, New York).
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Rickwood</surname>
            ,
            <given-names>D. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Deane</surname>
            ,
            <given-names>F. P.</given-names>
          </string-name>
          and Wilson,
          <string-name>
            <surname>C. J.</surname>
          </string-name>
          (
          <year>2007</year>
          )
          <article-title>Med</article-title>
          J Aus,
          <volume>187</volume>
          , 7 Suppl, S35{
          <fpage>S39</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <article-title>National Institute for Health and Clinical Excellence (</article-title>
          <year>2005</year>
          )
          <article-title>Depression in Children and Young People (NICE</article-title>
          , London).
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Merry</surname>
            ,
            <given-names>S. N.</given-names>
          </string-name>
          (
          <year>2007</year>
          ) Curr Opin Psych,
          <volume>20</volume>
          ,
          <issue>4</issue>
          , 325{
          <fpage>329</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Grzegorczyk</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Husmeier</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2008</year>
          ) Mach Learn,
          <volume>71</volume>
          ,
          <fpage>2</fpage>
          -
          <lpage>3</lpage>
          ,
          <issue>265</issue>
          {
          <fpage>305</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Ellis</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wong</surname>
            ,
            <given-names>W. H.</given-names>
          </string-name>
          (
          <year>2008</year>
          )
          <string-name>
            <given-names>J</given-names>
            <surname>Am Stat</surname>
          </string-name>
          <string-name>
            <surname>Assoc</surname>
          </string-name>
          ,
          <volume>103</volume>
          ,
          <issue>482</issue>
          , 778{
          <fpage>789</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Koller</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2003</year>
          ) Mach Learn,
          <volume>50</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>2</lpage>
          ,
          <issue>95</issue>
          {
          <fpage>125</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Goldszmidt</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>1996</year>
          ) In
          <source>Proc. Twelfth Conference on Uncertainty in Arti cial Intelligence (UAI-96)</source>
          (Morgan Kaufmann Publishers Inc.),
          <volume>252</volume>
          {
          <fpage>260</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>1998</year>
          ) In
          <source>Proc. Fourteenth Conference on Uncertainty in Arti cial Intelligence (UAI-98)</source>
          (Morgan Kaufmann Publishers Inc.),
          <volume>129</volume>
          {
          <fpage>138</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>