=Paper=
{{Paper
|id=Vol-1274/paper7
|storemode=property
|title=Type-II Errors of Independence Tests Can Lead to Arbitrarily Large Errors in Estimated Causal Effects: An Illustrative Example
|pdfUrl=https://ceur-ws.org/Vol-1274/uai2014ci_paper7.pdf
|volume=Vol-1274
|dblpUrl=https://dblp.org/rec/conf/uai/CorniaM14
}}
==Type-II Errors of Independence Tests Can Lead to Arbitrarily Large Errors in Estimated Causal Effects: An Illustrative Example==
https://ceur-ws.org/Vol-1274/uai2014ci_paper7.pdf
Type-II Errors of Independence Tests Can Lead to Arbitrarily Large
Errors in Estimated Causal Effects: An Illustrative Example
Nicholas Cornia & Joris M. Mooij
Informatics Institute
University of Amsterdam, The Netherlands
{n.cornia,j.m.mooij}@uva.nl
Abstract influenced by a hidden confounder, an instrumental
variable is a third observed variable which is assumed
Estimating the strength of causal effects from to be independent of the confounder. In practice it
observational data is a common problem in is difficult to decide whether the instrumental vari-
scientific research. A popular approach is able definition is satisfied, and the method has aroused
based on exploiting observed conditional in- some skepticism [7]. In this paper, we study a set-
dependences between variables. It is well- ting that is similar in spirit to the instrumental vari-
known that this approach relies on the as- able model, but where all conditional independence as-
sumption of faithfulness. In our opinion, a sumptions are directly testable on the observed data.
more important practical limitation of this A similar scenario was first studied by Cooper [1] and
approach is that it relies on the ability to independently rediscovered in the context of genome
distinguish independences from (arbitrarily biology by Chen et al. [8].
weak) dependences. We present a simple An important assumption in causal discovery meth-
analysis, based on purely algebraic and ge- ods based on conditional independences is faithful-
ometrical arguments, of how the estimation ness, which means that the observed joint distribution
of the causal effect strength, based on con- does not contain any additional (conditional) indepen-
ditional independence tests and background dences beyond those induced by the causal structure.
knowledge, can have an arbitrarily large er- Usually, faithfulness is justified by the assumption that
ror due to the uncontrollable type II error of unfaithful distributions are a set of Lebesgue measure
a single conditional independence test. The zero in the set of the model parameters. By showing
scenario we are studying here is related to that one can create a sequence of faithful distributions
the LCD algorithm by Cooper [1] and to the which converges to an unfaithful one, Robins et al.
instrumental variable setting that is popular proved the lack of uniform consistency of causal discov-
in epidemiology and econometry. It is one ery algorithms [9]. Zhang and Spirtes [10] then intro-
of the simplest settings in which causal dis- duced the “Strong Faithfulness” assumption to recover
covery and prediction methods based on con- the uniform consistency of causal discovery. Using geo-
ditional independences arrive at non-trivial metric and combinatorial arguments, Uhler et al. [11]
conclusions, yet for which the lack of uniform addressed the question of how restrictive the Strong
consistency can result in arbitrarily large pre- Faithfulness assumption is in terms of the volume of
diction errors. distributions that do not satisfy this assumption. Even
for a modest number of nodes and for sparse graphs,
the “not strongly faithful” regions can be surprisingly
Introduction large, and Uhler et al. argue that this result should dis-
courage the use of large scale causal algorithms based
Inferring causation from observational data is a com- on conditional independence tests, such as the PC and
mon problem in several fields, such as biology and eco- FCI algorithms [12].
nomics. To deal with the presence of unmeasured con-
In this work, we analyse in the context of the LCD
founders of observed random variables the so-called
setting how an error in a single conditional indepen-
instrumental variable technique [2] has found applica-
dence test may already lead to arbitrarily large er-
tions in genetics [3], epidemiology [4, 5] and economics
rors in predicted causal effect strengths, even when
[6]. Given two observable random variables possibly
the faithfulness assumption is not violated. Our re- where An(X) is the set of the causal ancestors of X
sults may not be surprising for those familiar with the (which includes X itself), so this condition means that
work of [9], but we believe that the analysis we present we assume that X1 is not caused by the other observed
here may be easier to understand to those without a variables X2 , X3 .
background in statistics, as we separate statistical is-
sues (the possibility of type II errors in the conditional Cooper [1] proved that:
independence test from a finite sample) from a rather Theorem 1.1. Under the assumptions in Definition
straightforward analysis of the problem in the popu- 1.1, the causal structure must be a subgraph of:
lation setting. We use an algebraic approach, showing
how causal prediction may lead to wrong predictions X1 X2 X3
already in the simple context of linear structural equa-
tion models with a multivariate Gaussian distribution.
Here, the directed arrows indicate a direct causal rela-
In Section 1, we begin with a brief description of the tionship and the bidirected edge denotes an unobserved
problem setting in a formal way, giving the definitions confounder.
of the causal effect, instrumental variable, LCD al-
gorithm and the toy model we present. We consider Our primary interest is to predict p(X3 |do(X2 )), the
three observed random variables (X1 , X2 , X3 ), which distribution of X3 after an intervention on X2 . In
is the minimal number such that a non-trivial condi- general, this quantity may differ from p(X3 |X2 ), the
tional independence test can be obtained. In Section 2, conditional distribution of X3 given X2 [13]. In the
we show how an (arbitrarily weak) conditional depen- linear-Gaussian case, the quantity
dence that goes undetected can influence our estima-
tion of the causal effect of X2 on X3 from the observed ∂E(X3 |do(X2 ))
covariance matrix, when a confounder between X2 and ∂X2
X3 is almost off-set by a direct effect from X1 to X3 .
In fact, we show that this phenomenon can lead to an measures the causal effect of X2 on X3 .
arbitrarily large error in the estimated causal effect as It is easy to show that in the LCD setting, these quan-
the noise variance of X2 approaches zero. We finish tities are equal:
with conclusions in Section 3.
Corollary 1.1. Under the LCD assumptions in Def-
inition 1.1,
1 Problem setting
p(X3 |do(X2 )) = p(X3 |X2 ).
1.1 LCD algorithm
Therefore, in the linear-Gaussian case, the quantity
The model we are interested in arises from the work
∂E(X3 |do(X2 )) ∂E(X3 |X2 ) Cov(X3 , X2 )
of Cooper [1], who proposed the “LCD” algorithm for = = (1)
causal discovery in observational databases and the ∂X2 ∂X2 Var(X2 )
more recent paper of Chen et al.[8], who proposed the
is a valid estimator for the causal effect of X2 on X3 .
“Trigger” algorithm to infer transcriptional regulatory
networks among genes. Throughout this section we
will assume: 1.2 Relationship with instrumental variables
• Acyclicity; The other model relevant for our discussion is the so
• No Selection Bias. called instrumental variable model. Following Pearl
Definition 1.1. (LCD setting) Given three ran- [13], we define:
dom variables X1 , X2 , X3 such that the following sta- Definition 1.2. (Instrumental Variable setting)
tistical properties and prior assumptions are satisfied: Given three random variables X1 , X2 , X3 , we call X1
Statistical dependences: an instrumental variable if the following conditions are
satisfied:
• X1 6⊥
⊥ X2
Statistical dependences:
• X2 6⊥
⊥ X3
• X1 ⊥⊥ X3 |X2 • X1 6⊥⊥ X2
Prior assumptions: Prior assumptions:
• An(X1 ) ∩ {X2 , X3 } = ∅ • X1 ⊥⊥ X3 |do(X2 )
• Faithfulness • Faithfulness
The second assumption says that X1 and X3 are inde- between X1 and X3 , or between X1 , X2 , X3 . This sim-
pendent after an intervention on the variable X2 . In plification will not influence the final result of the pa-
terms of the causal graph, this means that all the un- per, because we will prove how unboundedness of the
blocked paths between X1 and X3 contain an arrow causal effect estimation error is already achieved for
that points to X2 . this special case.
Unfortunately the instrumental variable property can- Definition 1.3. We assume that the “true” causal
not be directly tested from observed data. The causal model has the following causal graph:
graph for the IV setting is a subgraph of:
X4
X1 X2 X3
X1 X2 X3
So, a possible confounder between X2 and X3 is al-
lowed, in contrast with the LCD setting. Note that
which is one of the possible causal structures that is
the LCD setting is a special case of the IV model.
compatible with the following conditions:
Lemma 1.1. Under the IV assumptions in Definition Statistical dependences:
1.2 and for the linear-Gaussian case, the quantity
• X1 6⊥⊥ X2
Cov(X1 , X3 ) • X2 6⊥⊥ X3
Cov(X1 , X2 ) • A weak conditional dependence
is a valid estimator for the causal effect of X2 on X3 . X1 6⊥⊥ X3 |X2
Prior assumptions:
1.3 Type II errors in LCD
• Faithfulness
In practice, the confidence on the result of the con- • An(X1 ) ∩ {X2 , X3 } = ∅
ditional independence test X1 ⊥⊥ X3 |X2 in the LCD
The observed random variables are X1 , X2 , X3 while
setting depends on the sample size. Indeed, it could
X4 is a hidden confounder, assumed to be independent
be hard to distinguish a weak conditional dependence
from X1 .
X1 6⊥
⊥ X3 |X2 The joint distribution of the observed variables is as-
sumed to be a multivariate Gaussian distribution with
from a conditional independence using a sample of fi- covariance matrix Σ and zero mean vector. We also
nite size. Here we study the question of what happens assume that the structural equations of the model are
to our prediction of the causal effect of X2 on X3 if linear. Then
the conditional independence test encounters a type II X = AX + E, (2)
error (i.e., erroneously accepts the null hypothesis of
independence). where T
X = X1 , . . . , X4
Note that a type I error (i.e., erroneously rejecting the
null hypothesis of independence) in the tests X1 6⊥⊥ X2 is the vector of the extended system,
and X2 6⊥ ⊥ X3 will not be as dangerous as a type II T
error in the conditional independence test. Indeed, the E = E1 , . . . , E4
probability of a type I error can be made arbitrarily
is the vector of the independent noise terms, such that
small by tuning the significance level appropriately. In
addition, a type I error would let the LCD algorithm E ∼ N 0, ∆ ∆ = diag δi2 ,
reject a valid triple, i.e., lower the recall instead of
leading to wrong predictions.
and A = (αij ) ∈ M4 R is (up to a permutation of
For these reasons we study the model described in the indices) a real upper triangular matrix in the space
following definition, which allows the presence of a hid- M4 (R) of real 4 × 4 matrices that defines the causal
den confounder X4 , and a direct effect from X1 on X3 strengths between the random variables of the system.
(not mediated via X2 ). We assume that these addi- Remark 1.1. In [14], an implicit representation for
tional features result in a possible weak conditional the confounder X4 is used, by using non-zero covari-
dependence between X1 and X3 given X2 . For sim- ance between the noise variables E2 , E3 . It can be
plicity we consider only the linear-Gaussian case. We shown that for our purposes, the two representations
also assume no confounders between X1 and X2 , or are equivalent and yield the same conclusions.
In the Gaussian case, a conditional independence is Proof. It is possible to express the covariance matrix
equivalent to a vanishing partial correlation: Σ̄ of the joint distribution of X1 , . . . , X4 in terms of
Lemma 1.2. Given a set of three random variables the model parameters as follows:
(X1 , X2 , X3 ) with a multivariate Gaussian distribution −T −1
the conditional independence Σ̄ = I − A ∆ I −A .
X1 ⊥
⊥ X3 | X2 The individual components in (6)–(11) can now be ob-
tained by straightforward algebraic calculations.
is equivalent to a vanishing partial correlation
ρ13 − ρ12 ρ23 Remark 2.1. (Instrumental variable estimator)
ρ13·2 = q =0 (3) From equation (8) it follows immediately that for
1 − ρ212 1 − ρ223 α13 = 0, we have
where ρij is the correlation coefficient of Xi and Xj . Σ13
α23 = ,
Σ12
In the model described in Definition 1.3,
∂E(X3 |do(X2 )) which corresponds to the usual causal effect estimator
= α23 . (4) in the instrumental variable setting [3].
∂X2
In contrast with the LCD model in Definition 1.1, the The lemma we present now reflects the fact that we
equality (1) no longer holds. We are interested in the are always free to choose the scale for the unobserved
error in the estimation of the effect of X2 on X3 that confounder X4 :
would be due to a type II error of the conditional inde-
pendence test in the LCD algorithm. The next section Lemma 2.1. The equations of proposition 2.1 are in-
is dedicated to the analysis of the difference between variant under the following transformation
the true value (4) and the estimated one in (1): q
ᾱ4j = δ42 α4j , δ̄42 = 1
|E X3 |X2 − E X3 |do(X2 ) | = |g A, Σ ||X2 |,
where the “causal effect estimation error” is given by: for j ∈ {2, 3}.
Σ32
g A, Σ = − α23 . (5) Proof. This invariance follows from the fact that α42
Σ22 and α43 always appear in a homogeneous polynomial
of degree 2, and they are always coupled with a δ42
2 Estimation of the causal effect error term.
from the observed covariance matrix
Without loss of generality we can assume from now on
The following proposition gives a set of equations for that δ42 = 1.
the observed covariance
matrix Σ, given the model
parameters A, ∆ and the linear structural equation Remark 2.2. (Geometrical Interpretation) From
model (2). a geometrical point of view the joint system of equa-
Proposition 2.1. The mapping Φ : (A, ∆) 7→ Σ that tions for the observed covariance matrix defines a
maps model parameters (A, ∆) to the observed covari- manifold MΣ in the space of the model parameters
ance matrix Σ according to the model in Definition 1.3 M4 (R) × Dδ2 , where M4 (R) is the space of the possi-
is given by: ble causal strengths αij and
Σ11 = δ12 (6) 3
Y
D δ2 = [0, Σii ]
Σ12 = α12 δ12 (7)
i=1
Σ13 = α13 + α23 α12 δ12
(8)
is the compact hypercube of the noise variances. Note
Σ11 Σ23 = Σ12 Σ13 that we have used the symmetry Σ̄44 = δ42 = 1 and
(9)
+ Σ11 δ22 α23 + δ42 α42 α43 + α23 α42 that
Σ11 Σ22 = Σ212 + Σ11 δ22 + δ42 α42
2
(10) δi2 ≤ Σii
Σ11 Σ33 = Σ213 from equations (6), (10) and (11). Note that the map
2 Φ : (A, ∆) 7→ Σ is not injective. This means that
+ Σ11 δ22 α23
2
+ δ32 + δ42 α43 + α23 α42 . given an observed covariance matrix Σ, it is not possi-
(11) ble to identify the model parameters in a unique way.
Indeed, the number of equations is six, while the num- Here,
ber of model parameters is eight. Geometrically, this m = Σ11 Σ22 − Σ212 > 0 (19)
means that the manifold MΣ does not reduce to a
η = Σ11 Σ33 − Σ213 > 0
single point in the space of model parameters. Nev-
ertheless it is still an interesting question whether the ω = Σ22 Σ33 − Σ223 > 0
function g is a bounded function on MΣ or not, i.e., ϑ = Σ13 Σ22 − Σ12 Σ23
whether we can give any guarantees on the estimated
causal effect. Indeed, for the instrumental variable γ = Σ11 Σ23 − Σ12 Σ13 .
case with binary variables, such bounds can be derived
(see, e.g., [13]). Proof. The proof proceeds by explicitly solving the
system of equations (6)–(11). Some useful identities
are:
A Σ Σ12 α42 α43 ϑ
Φ α13 = + ,
MΣ m m
. γ − α23 m
α42 α43 = ,
Φ −1
=? Σ11
∆ ϑ
ρ13·2 = √ ,
ωm
The following Theorem and its Corollary are the main ηm − γ 2 = Σ11 det Σ.
results of this paper. We will prove that there still re-
main degrees of freedom in the noise variances δ22 , δ32 The signs in the equations are a consequence of the
and the signs s1 , s2 , given the observed covariance ma- second degree polynomial equations.
trix Σ, that will lead to an unbouded causal effect es- Corollary 2.1. It is possible to express the error in
timation error g(A, Σ). the estimated causal effect as
Theorem 2.1. Given the causal model in Definition ϑΣ12
g Ψ(Σ, δ22 , δ32 , s1 , s2 ), Σ =
1.3, there exists a map +
mΣ22
p p (20)
Ψ : M3 (R) × D(Σ) × {−1, +1}2 → M4 (R) (12) det Σ − mδ32 m − Σ11 δ22
s1 s2 p .
m δ22
such that for all (A, ∆):
By optimizing over δ32 we get:
Ψ(Φ(A, ∆), δ22 , δ32 , s1 , s2 ) = A. (13)
α23 ∈ [b− , b+ ] ⊂ R,
2
Here D(Σ) = [0, m/Σ11 ] × [0, det Σ/m] ⊂ R is the
rectangle where the noise variances of X2 and X3 live, with √ p
γ det Σ m − Σ11 δ22
with m defined below in (19). The map Ψ gives explicit b± (δ22 ) = ± p . (21)
solutions for the causal strengths αij , given the ob- m m δ22
served covariance matrix Σ, the noise variances δ22 , δ32 The length of the interval [b− , b+ ] is a function of
and signs si = ±1. The components of Ψ are given by: (Σ, δ22 ) and satisfies
Σ12 ∂|b+ − b− |
α12 = (14) < 0.
Σ11 ∂δ22
r
m
α42 = s1 − δ22 (15)
Σ11 Proof. Equation (20) follows from (18) and:
p
det Σ − mδ32 Σ23 γ ϑΣ12
α43 = s2 p (16) = + .
δ22 Σ11 Σ22 m mΣ22
p p
Σ12 det Σ − mδ32 m − Σ11 δ22 ϑ From equation (11), combined with the results of The-
α13 = s1 s2 p + ,
m δ22 Σ11 m orem 2.1, we can obtain the following inequality, using
(17) also the fact that δ32 Σ11 > 0:
2 2
and the most important one for our purpose: mα23 − 2γα23 + η − Σ11 α43 ≥ 0.
p p
γ det Σ − mδ32 m − Σ11 δ22 The two solutions of the inequality define the interval
α23 = − s1 s2 p . (18) [b− , b+ ]. Its length is a decreasing function of δ22 .
m m δ22
Unfortunately, the causal effect strength α23 in equa- is proportional to α13 and from (17) one can deduce a
tion (18) is unbounded. This means that for all the similar growing rate of the function h in terms of the
choices of the observed covariance matrix Σ that are variance of the noise term E2 :
in accordance with the model assumptions in Defini-
tion 1.3, the set of model parameters (A, ∆) ∈ MΣ 1
|h| ∝ as δ2 → 0. (24)
that would explain Σ leads to an unbounded error g. δ2
Indeed, a singularity is reached in the hyperplane Remark 2.5. (Singularity analysis)
δ22 = 0, which corresponds to making the random Figure 2 shows a contour plot of |g| on the rectangle
variable X2 deterministic with respect to its parents D(Σ) 3 (δ22 , δ32 ). The singularity in the causal effect
X1 , X4 . Figure 1 shows the singularity of the function
|g(Σ, δ22 , δ32 )| in the limit δ22 → 0. The rate of growth is
proportional to the inverse of the standard deviation
of the noise variable E2 :
1
|g| ∝ as δ2 → 0. (22)
δ2
Figure 2: The function |g| has a singularity in the
hyperplane δ22 = 0.
function g is reached in the degenerate case, when the
conditional distribution of X2 given X1 and X4 ap-
proaches a Dirac delta function. This cannot be de-
Figure 1: Causal effect estimation error |g| as a func- tected empirically, as we can still have well-defined co-
tion of δ22 , for fixed δ32 , Σ and s1 s2 = 1. variance matrices Σ of the observed system even if the
covariance matrix Σ̄ of the extended one is degenerate.
Remark 2.3. (Lower bound for δ22 ) Corollary 2.1
is the main result of our analysis. The right hand term Let us investigate in detail the limit for δ22 → 0 from
in (20) consists of two terms: the first one, through ϑ, the point of view of the causal model. This propo-
represents the contribution of the partial correlation, sition will show a simple example of how the causal
and is small if ρ13·2 is small. The second term is a strengths can be arbitrarily large, keeping the entries
fundamental, intrinsic quantity not controllable from of the observed covariance matrix Σij finite.
the conditional independence test and the sample size.
However, in situations where one is willing to assume Proposition 2.2. Assume that the observed covari-
a lower bound on δ22 : ance matrix Σ is positive-definite. Then, for the limit
δ22 → 0 we have the following scenario for the causal
δ22 ≥ δ̂22 ,
strength parameters:
it is possible to give a confidence interval [b+ , b− ] for
the function g, depending on the choice of the lower −1
α23 ≈ ± δ2
bound δ̂22 . α43 ≈ ∓ sgn(α42 ) δ2−1
α13 ≈ ∓ sgn(α12 ) δ2−1 .
Remark 2.4. (IV estimation error)
In the instrumental variable literature the IV estima-
tor is used, presented in Lemma 1.1. Unfortunately, This limit, in which our error in the estimated causal
this estimator and its error function effect strength of X2 on X3 diverges, is illustrated in
Σ13 Figure 3.
h(Σ, A) = − α23 (23)
Σ12
X4 • Bayesian Information Criterion: We could
directly assign a score based on the likelihood
α42 ∓∞ function of the data given the model parameters
(A, ∆) and the model complexity, without assum-
α12 ±∞ ing any prior distribution for the model parame-
X1 X2 X3 ters.
∓∞ • Nonlinear structural causal equations: To
deal with nonlinearity it is possible to consider
Spearman’s correlation instead of the usual one,
Figure 3: Scenarios in which the error in the causal ef- using the following relationships:
fect strength of X2 on X3 based on the LCD algorithm
may become infinitely large. m = Σ11 Σ22 (1 − ρ212 )
η = Σ11 Σ33 (1 − ρ213 )
ω = Σ22 Σ33 (1 − ρ223 )
3 Conclusions and future work p
γ = Σ11 Σ22 Σ33 (ρ23 − ρ12 ρ13 )
Corollary 2.1 shows how the causal effect estimation
p
ϑ = Σ22 Σ11 Σ33 (ρ13 − ρ12 ρ23 )
error can be extremely sensitive to small perturbations
of our model assumptions. Equation (20) holds for any • “Environment” variable: In many applica-
value of ϑ (which is proportional to the partial corre- tions in biology, for example where X1 is geno-
lation ρ13·2 ) and the second term vanishes when the type, X2 gene expression and X3 phenotype,
confounder is not present. This shows that with a the observed random variables X2 and X3 are
finite sample, a type II error in the conditional inde- strongly dependent on the environmental condi-
pendence test may lead to an arbitrarily large error in tions of the experiment. It might be reasonable
the estimated causal effect. Even in the infinite sample to assume that most of the external variability
limit, this error could be arbitrarily large if faithfulness is carried by the covariance between the environ-
is violated. The result is in agreement with the results ment variable W and the other measured ones,
in [9], and it shows in a clear algebraic way how type including possible confounders. This leads to the
II errors of conditional independence tests can lead to following graphical model, which could be useful
wrong conclusions. in deriving some type of guarantees for this sce-
We believe that this conclusion holds more generally: nario:
even when we increase the complexity and the number
of observed variables, the influence of confounders will W
still remain hidden, mixing their contribution with the
visible parameters, thereby potentially leading to ar- X4
bitrarily large errors. This means that for individual
cases, we cannot give any guarantees on the error in X1 X2 X3
the estimation without making further assumptions.
An interesting question for future research is whether
this negative worst-case analysis can be supplemented Acknowledgements
with more positive average-case analysis of the esti-
mation error. Indeed, this is what one would hope if We thank Tom Heskes for posing the problem, and
Occam’s razor can be of any use for causal inference Jonas Peters for inspiring discussions. We thank the
problems. reviewers for their comments that helped us improve
the manuscript.
Other possible directions for future work are:
• Study more complex models, in terms of
References
the number of nodes, edges and cycles. [1] G. F. Cooper. A simple constraint-based
algorithm for efficiently mining observational
• Bayesian model selection: We hope that the databases for causal relationships. Data Mining
Bayesian approach will automatically prefer a and Knowledge Discovery, 1:203–224, 1997.
simpler model that excludes a possible weak con-
ditional dependence even though the partial cor- [2] R.J. Bowden and D.A. Turkington. Instrumental
relation from the data is not exactly zero. Variables. Cambridge University Press, 1984.
[3] V. Didelez and N. Sheehan. Mendelian random-
ization as an instrumental variable approach to
causal inference. Statistical Methods in Medical
Research, 16:309–330, 2007.
[4] S. Greenland. An introduction to instrumental
variables for epidemiologists. International Jour-
nal of Epidemiology, 29:722–729, 2000.
[5] D. A. Lawlor, R. M. Harbord, J. A. C. Sterne,
N. Timpson, and G. D. Smith. Mendelian ran-
domization: Using genes as instruments for mak-
ing causal inferences in epidemiology. Statistics
in Medicine, 27:1133–1163, 2008.
[6] J.D. Angrista, W. G. Imbens, and D.B. Rubinc.
Identification of causal effects using instrumental
variables. Journal of the American Statistical As-
sociation, 91:444–455, 1996.
[7] D. A. Jaeger J. Bound and R. M. Baker. Problems
with instrumental variables estimation when the
correlation between the instruments and the en-
dogeneous explanatory variable is weak. Journal
of the American Statistical Association, 90:443–
450, 1995.
[8] L. S. Chen, F. Emmert-Streib, and J. D. Storey.
Harnessing naturally randomized transcription
to infer regulatory relationships among genes.
Genome Biology, 8, 2007.
[9] R. Scheines J. M. Robins, P. Spirtes, and
L. Wasserman. Uniform consistency in causal in-
ference. Biometrika, 90:491–515, 2003.
[10] J. Zhang and P. Spirtes. Strong faithfulness and
uniform consistency in causal inference. In Pro-
ceedings of the Nineteenth Conference on Uncer-
tainty in Artificial Intelligence (UAI 2003), pages
632–639, 2003.
[11] C. Uhler, G. Raskutti, P. Bühlmann, and B. Yu.
Geometry of the faithfulness assumption in causal
inference. The Annals of Statistics, 41:436–463,
2013.
[12] P. Spirtes, C. N. Glymour, and R. Scheines. Cau-
sation, prediction, and search. The MIT Press,
2000.
[13] J. Pearl. Causality: models, reasoning and infer-
ence. Cambridge University Press, 2000.
[14] M. Drton, R. Foygel, and S. Sullivan. Global
identifiability of linear structural equation mod-
els. The Annals of Statistics, 39:865–886, 2011.