Inferring causal explanations for machine learning models is a challenging task for eXplainable Artificial Intelligence (XAI). Counterfactual explanations, which are techniques to decide how to modify the model input to achieve a desired outcome, represents a possible first step in this direction. However, existing counterfactual explanation methods do not produce genuinely causal counterfactuals. These methods only exploit the correlations between features and target variables while ignoring the causal mechanisms among them. The project presented in this paper (and called MaLESCaMo) aims to develop a novel local and model-agnostic XAI procedure to generate genuine causal counterfactual explanations. Given a black-box predictor and an instance of the features, the procedure computes a counterfactual query in a surrogate causal model trained from a local neighbourhood of the input instance. To ease the domain expert elicitation of the causal model, we propose to adopt algorithms for partial ancestral graphs as a possible pre-processing step. A specialisation of the expectation maximisation algorithm is used instead to practically compute the causal queries.
Consider the following example. Peter is a 32-year-old low-wage factory worker. He submits to a
bank a loan request. The bank processes the request with a black-box classifier and eventually
rejects the loan. Peter asks the reasons for the decision and what he can do to get the loan accepted.
For such kinds of scenarios, counterfactual explanations (CEs) are generally considered in the
Explainable AI (XAI) literature [
However, CEs have the severe limitation of ignoring the causal relations among features often
present in real scenarios [
The MaLESCaMo project, presented in this paper, aims at countering these issues by
developing an XAI procedure to get causally-robust CEs based on surrogate causal models. The
procedure is local and model-agnostic. Given a supervised learning setup with features and
target , the procedure takes in input a black box predictor , a single instance of the features ,
and a causal query , hence it answers the query via learning a surrogate causal model on a local
neighbourhood of . As we want the query to be genuinely counterfactual, e.g., like Pearl’s
probabilities of causation [
In the above setup, a SCM is defined as a tuple ⟨, ℱ , { ( )}∈ ⟩ where is a causal
directed graph whose nodes are in one-to-one correspondence with a set of endogenous variables
(, ) and a set of exogenous variables denoting latent factors and graphically represented by
the root nodes of (see e.g. Fig. 1), ℱ is a collection of structural equations, each determining
the value of an endogenous variable as a deterministic function of its exogenous parents in , and
( ) is a probability distribution ranging over ∈ [
A complete SCM specification Pneumonia Electrolytes might require extensive specific domain knowledge to identify the common latent causes, determine the oriFloroquirolones Hospitalisation Epilepsy entation of the edges in the graph, and specify both the exogenous disFigure 1: An SCM with two exogenous variables (in grey). tributions and the structural equations.
In the lack of background domain knowledge to elicit the structural equations, we adopt a
conservative approach consisting of enumerating all the possible deterministic relations between
an endogenous variable and its endogenous parents in the causal graph [
Regarding the exogenous distributions, we consider the possibility of inferring them from
a joint endogenous distribution, trained, for instance, from a dataset of endogenous
observations. Following [
Like the bounding of counterfactual queries [20], exact CN inference is a hard task [21]. As
an alternative to approximate CN inference algorithms [22, 23], there are approaches proposed
explicitly for the bounding of counterfactual queries based on sampling [
The last challenge that remains is the identification of the causal graph with a complete
specification of the confounders (i.e., common exogenous causes) [
In a PAG, directed edges encode so-called ancestral relations between endogenous variables. Specifically, an edge from to ′ marks the existence of a causal path from to ′ (possibly mediated by some unknown exogenous variables) and excludes the possibility of ′ to be a cause of . A bi-directed edge ↔ ′ excludes that both is a cause of ′ and ′ is a cause of , which means that it must exist some latent common variable explaining the correlation between and ′. Circles on the edges mark uncertainty regarding the existence of a causal dependence, i.e., ⊸ ′ indicates that we do not definitively know whether there is a causal path from to ′ or not. Finally, directed edges labelled by mark a direct causal dependence that excludes the presence of possible confounders [28].
The significant advantage of PAGs is that they can be learned directly from data via standard structural learning algorithms [29, 30, 31]. In contrast, the learning of causal directed graphs is a much more challenging task and typically necessitates substantial assumptions about the causal-generating mechanism beyond the available data [32]. However, PAGs are less informative than standard causal diagrams since they encompass uncertainties in both edge orientations and the presence of confounders. Specifically, a PAG corresponds to an equivalence class of causal graphs, with each graph representing a distinct causal-generating mechanism compatible with the available endogenous information (see, Fig. 2). To obtain a causal graph from a PAG, we can then proceed by selecting one of the graphs in the equivalence class using domain-specific background knowledge.
1 1
3 2 (b) 2 1 1
3 2 1
3 (a) 2 (c) 1 1
3 2 (d) 2
The ideas sketched in the previous section are the basis for defining a complete XAI procedure to extract genuinely causal CEs based on a query , an instance of the features , and minimal access to expert domain knowledge. The procedure is articulated in several distinct steps, graphically outlined in Fig. 3 and described below.
• The procedure is model-agnostic and the training of is obtained from the supervised data , through any (supervised) machine learning (ML) algorithm. • The procedure is local aiming to explain a single test instance of the features. A synthetic neighbourhood of is generated by perturbing the input space locally on and then selecting the instances whose distance from is lower than a threshold * . The distance can be a standard metric (e.g., Hamming) or some ad-hoc distance functions considered suitable for the application. Similarly, the choice of * is up to the user, depending on the specific application context. • The instances of the features in the neighbourhood are automatically annotated by , .
using as an oracle. The resulting annotated dataset is denoted as • If the number of features is too high to be tractable and explainable in our setup, the , can be reduced by applying standard feature dimensionality of the instances in selection (FS) methods. The resulting dataset is denoted as ˜, . • Structural learning (SL) algorithms, such as [31], infer the PAG PAG over (, ) from the dataset ˜′, . • Domain knowledge (DK) in the form of a knowledge base or a human expert is required to select a causal directed graph in the equivalence class of PAG. • The structural equations (SEs) of the surrogate SCM based on can be also based on
DK. Alternatively, the conservative approach above described should be considered. • Finally, the EM procedure of [20], already embedded in a software library for causal inference [33], can be used to answer the causal query .
,
ML , * , FS ˜ , SL PAG
DK SE s SCM
EM
CE
The research in this paper has been supported by the Hasler Foundation grant No. 22050. Models, volume 138 of PMLR, JMLR.org, 2020, pp. 613–616. [19] I. Shpitser, J. Pearl, What counterfactuals can be tested, in: Proceedings of the Twenty-Third
Conference on Uncertainty in Artificial Intelligence, 2007, pp. 352–359. [20] M. Zaffalon, A. Antonucci, R. Cabañas, Causal expectation-maximisation, in: Proceeding of the WHY-21 NeurIPS Workshop, 2021. [21] D. D. Mauá, C. P. De Campos, A. Benavoli, A. Antonucci, Probabilistic inference in credal networks: new complexity results, Journal of Artificial Intelligence Research 50 (2014) 603–637. [22] A. Antonucci, Y. Sun, C. de Campos, M. Zaffalon, Generalized loopy 2U: A new algorithm for approximate inference in credal networks, International Journal of Approximate Reasoning 51 (2010) 474–484. [23] A. Antonucci, C. P. de Campos, D. Huber, M. Zaffalon, Approximate credal network updating by linear programming with applications to decision making, International Journal of Approximate Reasoning 58 (2015) 25–38. [24] A. Balke, J. Pearl, Counterfactual probabilities: Computational methods, bounds and applications, in: Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, 1994, pp. 46–54. [25] M. Zaffalon, A. Antonucci, R. Cabañas, D. Huber, D. Azzimonti, Bounding counterfactuals under selection bias, in: Proceedings of The 11th International Conference on Probabilistic Graphical Models, volume 186 of Proceedings of Machine Learning Research, 2022, pp. 289–300. [26] J. Zhang, Generalized do-calculus with testable causal assumptions, in: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, volume 2 of Proceedings of Machine Learning Research, PMLR, 2007, pp. 667–674. [27] J. Zhang, Causal reasoning with ancestral graphs, Journal of Machine Learning Research 9 (2008) 1437–1474. [28] A. Jaber, A. Ribeiro, J. Zhang, E. Bareinboim, Causal identification under markov equivalence: calculus, algorithm, and completeness, Advances in Neural Information Processing Systems 35 (2022) 3679–3690. [29] B. Huang, K. Zhang, J. Zhang, J. Ramsey, R. Sanchez-Romero, C. Glymour, B. Schölkopf, Causal discovery from heterogeneous/nonstationary data, The Journal of Machine Learning Research 21 (2020) 3482–3534. [30] J. M. Mooij, T. Claassen, Constraint-based causal discovery using partial ancestral graphs in the presence of cycles, in: Conference on Uncertainty in Artificial Intelligence, PMLR, 2020, pp. 1159–1168. [31] J. Zhang, On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias, Artificial Intelligence 172 (2008) 1873–1896. [32] P. Spirtes, An anytime algorithm for causal inference, in: International Workshop on
Artificial Intelligence and Statistics, PMLR, 2001, pp. 278–285. [33] R. Cabañas, A. Antonucci, D. Huber, M. Zaffalon, CREDICI: a Java library for causal inference by credal networks, in: International Conference on Probabilistic Graphical Models, PMLR, 2020, pp. 597–600.