We introduce a conceptualisation for generating argumentation frameworks (AFs) from causal models for the purpose of forging explanations for models' outputs. The conceptualisation is based on reinterpreting properties of semantics of AFs as explanation moulds, which are means for characterising argumentative relations. We demonstrate our methodology by reinterpreting the property of bi-variate reinforcement in bipolar AFs, showing how the extracted bipolar AFs may be used as relation-based explanations for the outputs of causal models.
The field of explainable AI (XAI) has in recent years become a major focal point of the
eforts of researchers, with a wide variety of models for explanation being proposed (see
[
Despite these findings, many of the approaches for generating explanations for AI
models have, nevertheless, neglected causality as a potential drive for explainability. Some
of the most popular methods are heuristic and model-agnostic [
https://www.doc.ic.ac.uk/~afr114/ (A. Rago); https://briziorusso.github.io/ (F. Russo);
https://www.imperial.ac.uk/people/f.toni (F. Toni)
they are useful, particularly with regards to their wide-ranging applicability, they neglect
how models are determining their outputs and therefore the underlying causes therein.
This has arguably left a chasm between how explanations are provided by models at the
forefront of XAI technology and what users actually require from explanations [
Meanwhile, computational argumentation (see [
In this paper, we introduce a conceptualisation for generating argumentation
frameworks (AFs) with any number of dialectical relations as envisaged in [
Overall, we make the following main contributions: • We propose a novel concept for defining relation-based explanations for causal models by inverting properties of argumentation semantics. • We use this concept to define a novel form of reinforcement explanation (RX) for causal models. • We show deployability of RXs with two machine-learning models, from which causal models are drawn.
Our method relies upon causal models and some notions from computational argumentation. We provide core background for both.
Causal models. A causal model [
); • is a (finite) set of structural equations that, for each endogenous variable ∈ , define ’s values as a function of the values of ’ parents ( ) ⊆ ∪ ⧵ { Example 1. Let us consider a simple causal model ⟨ , , ⟩ comprising = { = { 1, 2} and for all ∈ ∪ , (
) = {⊤, ⊥}. Figure 1i (we ignore Figure 1ii for the moment: this will be discussed later in §4) visualises the variables’ parents, and Table 1 group chooses to enter the pizzeria. gives the combinations of values for the variables resulting from the structural equations . This may represent a group’s decision on whether or not to enter a restaurant, with variables 1: “margherita” is spelt correctly on the menu, not like the drink; 2: there is pineapple on the pizzas; 1: the pizzeria seems to be legitimately Italian; and 2: the }. 1, 2}, the assignment to exogenous variables u ∈ such that 1[u] = ⊤ and 2[u] = ⊤. example { 1, 2} = (
1), i.e. 1 and 2 are the parents of 1). (ii) SAF explanation (see §3) for Given a causal model ⟨ , , ⟩ where = { 1, … , }, we denote with = ( 1)×…× ( ) the a set of all possible combinations of values of the exogenous variables (realisations). With an abuse of notation, we refer to the value of any variable ∈ ∪ given u ∈ as [u]: if is an exogenous variable, [u] will be its assigned value in u; if is 1 ⊤ margherita ⊤ margherita ⊥ margarita ⊥ margarita
2 ⊤ pineapple ⊥ ∼pineapple ⊤ pineapple ⊥ ∼pineapple
1 ⊥ ∼Italian ⊤ Italian ⊥ ∼Italian ⊥ ∼Italian
2 ⊥ ∼enter ⊤ enter ⊥ ∼enter ⊥ ∼enter an endogenous variable, it will be the value dictated by the structural equations in the causal model.
We use the do operator [
Argumentation. In general, an argumentation framework (AF) is any tuple ⟨ , ℛ 1, … , ℛ ⟩,
with a set (of arguments), > 0 and ℛ ⊆ × , for ∈ {1, … , } , (binary and directed)
dialectical relations between arguments [
The meaning of AFs (including the intended dialectical role of the relations) may be
given in terms of gradual semantics (e.g. see [
The choice of gradual semantics for AFs may be guided by properties that the mappings
should satisfy (e.g. as in [
In this section we see the task of obtaining explanations for causal models’ assignments of values to variables as a two-step process: first we define moulds characterising the core ingredients of explanations; then we use these moulds to obtain, automatically, (instances of) AFs as argumentative explanations. Moulds and explanations are defined in terms of influences between variables in the causal model, focusing on those from parents to children given by the causal structure underpinning the model, as follows.
be a causal model. The influence graph corresponding to Definition 1. Let = ⟨ , , ⟩ is the pair ⟨ , ℐ ⟩ with: • = ∪ • ℐ ⊆ × influences).
is the set of all (exogenous and endogenous) variables; is defined as ℐ = {( 1, 2)| 1 ∈ (
Note that, while straightforward, the concept of influence graph (closely related to the
notion of causal diagram [
Next, the idea underlying explanation moulds is that, typically, inside the causal model, some variables afect others in a way that may not be directly understandable or even cognitively manageable by a user. The influence graph synthetically expresses which variables afect which others but does not give an account of how the influences actually occur in the context (namely, the values given to the exogenous variables) that a user may be interested in. Thus, the perspective we take is that each influence can be assigned an explanatory role, indicating how that influence is actually working in that context. The explanatory roles ascribable to influences can be regarded as a form of explanatory knowledge which is user specific: diferent users may be willing (and/or able) to accept explanations built using diferent sets of explanatory roles as they correspond to their understanding of how variables may afect each other. We assume that each explanatory role is specified by a relation characterisation, i.e. a Boolean logical requirement, which can be used to mould the explanations to be presented to the users by indicating which relations play a role in the explanations.
Definition 2. Given a causal model ⟨ , , ⟩ an explanation mould is a non-empty set:
and its corresponding influence graph ⟨ , ℐ ⟩ , { 1, … , } where for all ∈ {1, … , } , ∶ × ℐ → {⊤, ⊥} is a relation characterisation, in the form of a Boolean condition expressed in some formal language. Given some u ∈ and ( 1, 2) ∈ ℐ, if (u, ( 1, 2)) = ⊤ we say that the influence ( 1, 2) satisfies for u.
Note that we are not prescribing any formal language for specifying relation characterisations, as several such languages may be suitable.
Given an assignment u to the exogenous variables, based on an explanation mould, we can obtain an AF including, as (diferent) dialectical relations, the influences satisfying the (diferent) relation characterisations for the given u. Thus, the choice of relation characterisations is to a large extent dictated by the specific form of AF the intended users expect. Before defining argumentative explanations formally, we give an illustration. Example 1 (Cont.). Let us imagine a situation where one would like to explain the behaviour of the causal model from Figure 1i and Table 1 with a SAF (see §2). We thus require one single form of relation (i.e. support) to be extracted from the corresponding influence graph ⟨{ 1, 2, 1, 2}, {( 1, 1), ( 2, 1), ( 1, 2)}⟩. In order to define the explanation mould for such a situation, we note that the behaviour defining this relation could be characterised as changing the state of rejected arguments that it supports to accepted when the supporting argument’s state is accepted. In our simple causal model, accepted arguments may amount to variables assigned to value ⊤ and rejected arguments may amount to variables assigned to value ⊥. Thus, the intended behaviour can be captured by a relation characterisation such that, given u ∈ and ( 1, 2) ∈ ℐ: (u, ( 1, 2)) = ⊤ if ( 1[u] = ⊤∧ 2[u] = ⊤ ∧ 2[u, ( ( 1[u] = ⊥ ∧ 2[u] = ⊥ ∧ 2[u, ( 1 = ⊥)] = ⊥)∨ 1 = ⊤)] = ⊤).
Then, for the assignment to exogenous variables u ∈ such that 1[u] = ⊤ and 2[u] = ⊥, we may obtain the SAF in Figure 1ii (visualised as a graph with nodes as arguments and edges indicating elements of the support relation). For illustration, consider ( 1, 1) ∈ ℐ for this u. We can see from Table 1 that 1[u] = ⊤ and also that 1[u, ( 1 = ⊥)] = ⊥ and thus from the above it is clear that (u, ( 1, 1)) = ⊤ and thus the influence is of the type of support that characterises. Meanwhile, consider ( 2, 1) ∈ ℐ for the same u: the fact that 2[u] = ⊥ and 1[u] = ⊤ means that (u, ( 2, 1)) = ⊥ and thus the influence is not cast as a support. Indeed, if we consider the first and second rows of Table 1, we can see that 2 being true actually causes 1 to be false, thus it is no surprise that the influence is not cast as a support and plays no role in the resulting SAF. If we wanted for this influence to play a role, we could, for example, choose to incorporate an additional relation of attack into the explanation mould, to generate instead BAFs (see §2) as argumentative explanations. This example thus shows how explanation moulds must be designed to fit causal models depending on external explanatory requirements dictated by users. It should be noted also that some explanation moulds may be unsuitable to some causal models, e.g. the explanation mould with the earlier would not be directly applicable to causal models with variables with non-binary or continuous domains.
In general, AFs serving as argumentative explanations can be generated as follows. Definition 3. Given a causal model ⟨ , , ⟩ , its corresponding influence graph ⟨ , ℐ ⟩ , some u ∈ and an explanation mould { 1, … , }, an argumentative explanation is an AF ⟨ , ℛ 1, … ℛ ⟩, where • ⊆
, and • ℛ1, … , ℛ ⊆ ℐ ∩ ( × ) )| (u, ( 1, 2)) = ⊤}.
such that, for any = 1 … , ℛ = {( 1, 2) ∈ ℐ ∩ ( ×
Note that we have left open the choice of (as a generic, possibly non-strict subset of ). In practice, may be the full , but we envisage that users may prefer to restrict attention to some variables of interest (for example, excluding variables not “involved” in any influence satisfying the relation characterisations).
Example 1 (Cont.). The behaviour of the causal model from Figure 1i and Table 1 for u such that 1[u] = ⊤ and 2[u] = ⊤, using the explanation mould { } given earlier, can be captured by either of the two SAFs (argumentative explanations) below, depending on the choice of : • the SAF in Figure 1ii, where every variable is an argument; • the SAF with the same support relation but 2 excluded from , as not “involved” and thus not contributing to the explanation.
Both SAFs explain that 1[u] = ⊤ is supported by 1[u] = ⊤, in turn supporting 2[u] = ⊤ . Namely, the causal model recommends that the group should enter the pizzeria because the pizzeria seems legitimately Italian, given that “margherita” is spelt correctly on the menu. Note that the pineapple not being on the pizza could also be seen as a support towards the pizzeria being legitimately Italian, the inclusion of which could be achieved with a slightly more complex explanation mould.
The choice (number and form) of relation characterisations in explanation moulds is crucial for the generation of explanations concerning the value assignments to endogenous variables in the causal models. Even after having decided which argumentative relations to include in the AF/argumentative explanation, the definition of the relation characterisations is non-trivial, in general. In this section we demonstrate a novel concept for utilising properties of gradual semantics for AFs for the definition of relation characterisations and the consequent extraction of argumentative explanations.
The common usage of these properties in computational argumentation can be roughly
equated to: if a semantics, given an AF, satisfies some desirable properties, then the
semantics is itself desirable (for the intended context, where those properties matter).
We propose a form of inversion of this notion for use in our XAI setting, namely: if some
desirable properties are identified for the gradual semantics of (still unspecified) AFs,
then these properties can guide the definition of the dialectical relations underpinning
the AFs. For this inversion to work, we need to identify first and foremost a suitable
notion of gradual semantics for the AFs we extract from causal models. Given that, with
our AFs, we are trying to explain the results obtained from underlying causal models, we
cannot impose just any gradual semantics from the literature, but need to make sure that
we capture, with the chosen semantics, the behaviour of the causal model itself. This is
similar, in spirit, to recent work to extract (weighted) BAFs from multi-layer perceptrons
(MLPs) [
A natural semantic choice for causal models, since we are trying to explain why endogenous variables are assigned specific values in their domains given assignments to the exogenous variables, is to use the assignments themselves as a gradual semantics. Then, the idea of inverting properties of semantics to obtain dialectical relations in AFs can be recast to obtain relation characterisations in explanation moulds as follows: given an influence graph and a selected value assignment to exogenous variables, if an influence satisfies a given, desirable property, then the influence can be cast as part of a dialectical relation in the resulting AF.
Naturally, for this inversion to be useful, we need to identify useful properties from an explanatory viewpoint.
We will illustrate this concept with the property of
bivariate reinforcement for BAFs [
Property 1. Given a causal model ⟨ , , ⟩ such that, for each ∈ ∪ , the domain ( ) is equipped with a pre-order <,2 and given its corresponding influence graph ⟨ , ℐ ⟩ , an argumentative explanation ⟨ , ℛ
−, ℛ+⟩ for u ∈ + ∈ (
1) such that + > 1: any ( 1, 2) ∈ ℐ where 1 = 1[u], for any − ∈ ( satisfies causal reinforcement if for 1) such that − < 1, and for any • if ( 1, 2) ∈ ℛ−, then 2[u, ( 1 = +)] ≤ 2[u] and 2[u, ( 1 = −)] ≥ • if ( 1, 2) ∈ ℛ+, then 2[u, ( 1 = +)] ≥ 2[u] and 2[u, ( 1 = −)] ≤ 2[u]; 2[u].
We can then invert this property to obtain an explanation mould. In doing so, we
introduce slightly stricter conditions to ensure that influencing variables that have no efect
on influenced variables do not constitute both an attack and a support, a phenomenon
which we believe would be counter-intuitive from an explanation viewpoint.
1Here, we ignore the intrinsic basic strength of arguments used in the formal definition in [
Definition 4. Given a causal model ⟨ , , ⟩ such that, for each ∈ ∪ , the domain ( ) is equipped with a pre-order <, and given its corresponding influence graph ⟨ , ℐ ⟩ , a reinforcement explanation mould is an explanation mould { −, +} such that, given some u ∈ and ( 1, 2) ∈ ℐ, letting 1 = 1[u]: • −(u, ( 1, 2)) = ⊤ if: • +(u, ( 1, 2)) = ⊤ if: 1. ∀ + ∈ ( 2. ∀ − ∈ ( 1) such that + > 1, it holds that 2[u, ( 1) such that − < 1, it holds that 2[u, ( 1 = +)] ≤ 2[u]; 1 = −)] ≥ 2[u]; 3. ∃≥1 + ∈ ( 1) or ∃≥1 − ∈ ( in points 1 and 2 above.
1) satisfying strictly the inequality conditions 1. ∀ + ∈ ( 2. ∀ − ∈ ( 1) such that + > 1, it holds that 2[u, ( 1) such that − < 1, it holds that 2[u, ( 1 = +)] ≥ 2[u]; 1 = −)] ≤ 2[u]; 3. ∃≥1 + ∈ ( 1) or ∃≥1 − ∈ ( in points 1 and 2 above.
We call any argumentative explanation resulting from the explanation mould { −, +} a reinforcement explanation (RX).
Note that, as for generic argumentative explanations, we do not commit in general to any choice of in RXs.
Proposition 1. Any RX satisfies causal reinforcement.
Proof. Follows directly from the definition of Property 1 and Definition 4.
The satisfaction of the property of causal reinforcement indicates how RXs could be used counterfactually, given that the results of changes to the variables’ values on influenced variables are guaranteed. For example, if a user is looking to increase an influenced variable’s value, supporters (attackers) indicate variables whose values should be increased (decreased, respectively). In the following sections, we will explore the potential of this capability when causal models provide abstractions of classifiers whose output needs explaining.
In this section, we instantiate causal models for two diferent AI models commonly used for classification in the literature and preliminarily discuss a potential use of RXs in this context.
The two classification models that we use to instantiate causal models are Bayesian network classifiers (BCs) and classifiers built from feed-forward neural networks (NNs). Given some assignments to input variables I (from the variables’ domains), these classifiers can be seen as determining the most likely value for classification variables, which, in this paper, we assume to be binary, in a given set C. Thus, the classification task may be seen as a mapping ℳ(x) returning, for assignment x to input variables, either 1 or 0 (for the classification variables in C) depending on whether the probability exceeds a given threshold . We summarise the classification process in Figure 2. Note that the choice of threshold is crucial to guarantee that a single value is determined by ℳ for each classification variable : if is too high, then no value may be computed, whereas if is too low, the probability of both values may exceed it. Note also that, in the case of NNs, the probabilities may result from using, e.g., a softmax activation for the output layer. Furthermore, note that for the purposes of this paper, the underpinning details of these classifiers and how they can be obtained are irrelevant and will be ignored. In other words, we treat the classifier as a black-box, as standard in much of the XAI literature, and explain its outputs in terms of its inputs.
We represent the classification task by a (naive) BC or by a NN with the following causal model: Definition 5. A causal model for a naive BC or classifier built from a NN is a causal model ⟨ , , ⟩, where: • consists of the input variables I of the classifier, with their respective domains; • = C such that, for each ∈ C, ( ) = { 1, 0}; • corresponds to the computation of the probability values ( = 1)) by the classifier (see Figure 2).
ℐ = × represents the influences in the causal model for the classifier; these are such that the exogenous variables are densely connected to the endogenous variables .
In line with our assumptions for RXs, we assume that the variables’ domains are equipped with a pre-order.
On this basis, we envisage the following use of RXs as actionable explanations in contexts where the user has a classification goal to reach and has control on (some of) the input variables.
Given a situation with an undesired classification outcome (e.g. a rejected loan application) and an explanation indicating the relevant attackers and supporters, if a user would like to decrease the probability of the current classification, s/he would look to increase (decrease) the value of the corresponding variable’s attackers (supporters, respectively), in line with Property 1.
A broader investigation of the possible uses of RXs is left to future work.
The role of causality within explanations for AI models has received increasing attention
of late. [
Computational argumentation has been widely used in the literature as a mechanism for
explaining AI models, from data-driven explanations of classifiers’ outputs [
Various works have explored the links between causality and argumentation. [48] shows that a propositional argumentation system in a full classical language is equivalent to a causal reasoning system, while [49] develops a formal theory combining “causal stories” and evidential arguments. Somewhat similarly to us, [50] present a method for extracting argumentative explanations for the outputs of causal models. However, their method requires more information than the causal model alone, namely, ontological links, and the argumentation supplements the rule-based explanations, rather than being the main constituent, as is the case in our approach.
We have introduced a novel approach for extracting AFs from causal models in order to explain the latter’s outputs. We have shown how explanation moulds can be defined for particular explanatory requirements in order to generate argumentative explanations. We focused, in particular, on inverting the existing property of argumentation semantics of bi-variate reinforcement to create an explanation mould, before demonstrating how the resulting reinforcement explanations (RXs) can be used to explain causal models representing diferent machine-learning-based classifiers.
One of the most promising aspects of this preliminary work is the vast array of directions for future investigation it suggests. First, an experimental validation of the proposed ideas and a comparison with other explanation approaches on a suficiently large variety of case studies is needed.
Clearly, the wide-ranging applicability of causal models broadens the scope of explanation moulds and argumentative explanations well beyond machine learning models, and we plan to undertake an investigation into other contexts in which they may be useful, for example for decision support in healthcare.
We also plan to study inversions of diferent properties of argumentation semantics and
diferent forms of AFs to understand their potential, e.g. counting for AAFs [ 51]. Within
the context of explaining machine learning models, we plan to assess RXs’ suitability
for diferent data structures and diferent classifiers, considering in particular deeper
explanations, e.g. including influences amongst input variables and/or intermediate, in
addition to input and output, variables, in the spirit of [
Finally, while we posit that, when properly defined, the meaning and explanatory role of the dialectical relations can be rather intuitive at a general level, providing efective explanations to users through AFs will require the investigation of proper presentation and visualization methods, possibly tailored to users’ competences and goals and to diferent application domains.
Toni was partially funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101020934). Further, Russo was supported by UK Research and Innovation [grant number EP/S023356/1], in the UKRI Centre for Doctoral Training in Safe and Trusted Artificial Intelligence (www.safeandtrustedai.org). Finally, Rago and Toni were partially funded by J.P. Morgan and by the Royal Academy of Engineering under the Research Chairs and Senior Research Fellowships scheme. Any views or opinions expressed herein are solely those of the authors listed, and may difer, in particular, from the views and opinions expressed by J.P. Morgan or its afiliates. This material is not a product of the Research Department of J.P. Morgan Securities LLC. This material should not be construed as an individual recommendation for any particular client and is not intended as a recommendation of particular securities, financial instruments or strategies for a particular client. This material does not constitute a solicitation or ofer in any jurisdiction.
Bringing Order to the Web, WWW: Internet and Web Inf. Syst. 54 (1998) 1–17. [44] L. Amgoud, H. Prade, Using arguments for making and explaining decisions,
Artificial Intelligence 173 (2009) 413–436. [45] K. Cyras, D. Letsios, R. Misener, F. Toni, Argumentation for explainable scheduling, in: Proc. AAAI, 2019, pp. 2752–2759. [46] Q. Zhong, X. Fan, X. Luo, F. Toni, An explainable multi-attribute decision model based on argumentation, Exp. Syst. Appl. 117 (2019) 42–61. [47] N. Oren, K. van Deemter, W. W. Vasconcelos, Argument-based plan explanation, in: Knowledge Engineering Tools and Techniques for AI Planning, Springer, 2020, pp. 173–188. [48] A. Bochman, Propositional argumentation and causal reasoning, in: Proc. IJCAI, 2005, pp. 388–393. [49] F. Bex, An integrated theory of causal stories and evidential arguments, in: Proc.
ICAIL, 2015, pp. 13–22. [50] P. Besnard, M. Cordier, Y. Moinard, Arguments using ontological and causal knowledge, in: Proc. FoIKS, 2014, pp. 79–96. [51] L. Amgoud, J. Ben-Naim, Axiomatic foundations of acceptability semantics, in:
Proc. KR, 2016, pp. 2–11. [52] C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, A. Mordvintsev,
The building blocks of interpretability, Distill 3 (2018) e10. [53] T. Kyono, Y. Zhang, M. van der Schaar, CASTLE: regularization via auxiliary causal graph discovery, in: Proc. NeurIPS, 2020.