Explainable AI aims at making the decisions of autonomous systems human-understandable. Argumentation frameworks are a natural tool for this purpose. Among them, bipolar abstract argumentation frameworks seem well suited to explain the efect of features on a classification decision and their formal properties can potentially be used to derive formal guarantees for explanations. Two particular interesting properties are correctness (if the explanation says that afects , then afects ) and completeness (if afects , then the explanation says that afects ). The reinforcement property of bipolar argumentation frameworks has been used as a natural correctness counterpart in previous work. Applied to the classification context, it basically states that attacking features should decrease and supporting features should increase the confidence of a classifier. In this short discussion paper, we revisit this idea, discuss potential limitations when considering reinforcement without a corresponding completeness property and how these limitations can potentially be overcome.
Automatic decision making is increasingly driven by black-box machine learning models.
However, their opaqueness raises questions about fairness, reliability and safety. Explanation
methods aim at making the decision process transparent [
One easily comprehensible argumentation model are bipolar abstract argumentation
frameworks. They represent arguments in a graph, where nodes correspond to abstract arguments
(entities that can be accepted or rejected) and edges to attack or support relationships between
them. This representation seems well suited to represent the influence of features in a
classification problem on the class decision. For example, in a credit approval setting, the income may
have a positive efect (support), while existing debts may have a negative efect (attack) on the
decision. This is in line with the idea of the reinforcement property in bipolar argumentation [
The faithfulness of an argumentative explanation to the classifier can then naturally be
evaluated by checking to which extent the reinforcement property is satisfied. In recent work,
authors actually guaranteed perfect faithfulness by showing only those connections that respect
reinforcement [
We argue that, while reinforcement is a desirable property, it can be too strong when the arguments in the argumentation framework are not carefully selected. In particular, introducing a single argument per feature is often not suficient. One reason is that classifiers are often non-monotonic and a feature may have a positive efect in one and a negative efect in another region of the input domain. Another reason is that the efect of features often cannot be captured independently of the other features. Of course, this does not mean that we should give up reinforcement. Rather, we should refine the bipolar abstract argumentation frameworks based on the application domain to capture more complicated efects of features and to improve the completeness of the explanation.
The abstract goal of classification is to map inputs x to outputs . We think of the inputs as vectors x = (1, . . . , ), where the i-th value is taken from some domain . We let = × =1 denote the cartesian product of the individual domains. Given an input x = (1, . . . , ), with a slight abuse of notation, we let (x− , ) = (1, . . . , − 1, , +1, . . . , ) denote the input where the i-th component has been replaced with . The output is taken from a finite set of class labels. A classification problem = ((1, . . . , ), , ) consists of feature domains , class labels and a set of training examples = {(x, ) | 1 ≤ ≤ , x ∈ , ∈ }. For example, in a credit approval scenario, the first feature could be Age with corresponding domain 1 = N, the second feature Income with domain 2 = R and the third feature Debts with domain 3 = {0, 1}. In this case, the label set could be = {0, 1} indicating whether the credit application is accepted or rejected. then consists of previous cases, that are used to train a classifier. We will not be concerned with training here and just assume that a classifier is given and is supposed to be explained.
A probabilistic classifier is a function C : × → [
A bipolar argumentation graph (BAG) is a tuple (, Att, Sup) consisting of a set of abstract
arguments and attack and support relationships between them [
In order to explain a classifier, we can associate the underlying classification problem with a set of abstract arguments that allow us to argue about the classification decision. The most straightforward way to do this is to introduce one argument per feature and one argument per class label.
Definition 1 (Naive Classification Arguments) . Given a classification problem ((1, . . . , ), , ), the naive classification arguments associated with are the = • feature arguments , 1 ≤ ≤ for the features and • || class arguments , 1 ≤ ≤ for the class labels
We can then build an explanation BAG from a probabilistic classifier by taking the naive classification arguments and adding support and attack edges from feature arguments to class arguments.
Roughly speaking, we say that an explanation BAG satisfies reinforcement if the following two conditions are satisfied: 1. If there is an attack from to , then increasing (decreasing) the value of the -th feature decreases (increases) the probability of the -th class and 2. If there is an support from to , then increasing (decreasing) the value of the -th feature increases (decreases) the probability of the -th class.
While there is a natural order for boolean and ordinal features, the definition has to be made more precise for categorical features. However, it is suficient for our purposes as we mainly want to create awareness for potential limitations of this idea and how they can be addressed.
As discussed in the introduction, reinforcement is an interesting correctness/faithfulness property of an explanation. However, we should not only consider correctness, but also completeness of the explanation. As an extreme example, the empty graph defined over the naive classification arguments satisfies reinforcement. However, it is not a very interesting explanation because it does not explain anything. In general, a corresponding completeness property is desirable that explains to which extent the explanation graph captures the existing relationships. As a first step in this direction, we explain reasons for why an explanation BAG can result in incomplete explanations.
To begin with, let us focus on boolean data. That is, we assume that we have = {0, 1} for all features. In this case, there is a straightforward (even though computationally expensive) way to create an explanation BAG that satisfies reinforcement.
Definition 2 (Naive Explanation BAG). The naive explanation BAG for a probabilistic classifier
C : × → [
Let us note that one may also want to introduce attack arguments between the class arguments (only one class argument should be accepted). We refrain from doing so here because, at this point, we consider the explanation BAG merely as a visualization of the relationships between feature and class arguments. However, when applying argumentation semantics to the explanation BAG, these additional edges may be necessary.
While constructing the naive explanation BAG in a straightforward way takes exponential time, it is easy to check that it satisfies reinforcement. However, in many cases, it will only explain a fraction of the actual efects of features because it only looks at the individual efects of features.
To illustrate the problem, consider the XOR function (, ) that returns 1 if exactly one of and is 1. In this case, our explanation BAG would actually not contain any edges. However, and clearly have an efect on the decision, so that the explanation BAG is not an accurate explanation even though it satisfies reinforcement. The problem in this example is that the efect of features cannot be determined independently of each other. While the XOR function is an extreme example, similar dependencies naturally occur in many datasets. For example, in a credit approval setting, the highest educational degree may be relevant for an application if the applicant has a low income (e.g., a student after graduation), but not if the applicant has a high income. In a medical scenario, a drug may be an efective treatment for most patients, but could be detrimental for patients with particular medical or physical conditions.
In order to take account of such joint efects, we need to refine our naive explanation BAG. Two natural candidate refinements are the following.
1. Increase the BAG: Introduce joint feature arguments that capture the state of multiple
arguments simultaneously.
2. Consider more expressive BAGs: Introduce joint attacks/ joint supports that capture
the joint efect of arguments rather than considering them independently. Joint efects
can be represented, for example, by logical formulas similar to ADFs [
Let us now look at continuous data. That is, we assume that = [, ], , ∈ R, < is a real interval for all features. Now, the efect of features may not only depend on the efect of other features, but the efects of features can actually change on the domain.
To illustrate the problem, consider a simple binary classification problem over a single feature with domain = R. Intuitively, an input should be classified as positive if the value of the feature is “suficiently far away from 0". A typical example is anomaly detection, where the feature value corresponds to the deviation of an observation from the mean or median. Consider the classifier C1() = (2), where () = 1+ex1p(− ) is the logistic function. Figure 1 shows the graphs of the functions () and C1().
Intuitively, () squashes its input between 0 and 1. While the logistic function is monotonically increasing, our classifier C1() = (2) is monotonically decreasing for < 0 and monotonically increasing for > 0. Hence, similar to our previous XOR example, we are again in a situation where our feature can neither be characterized as an attacker nor as a supporter even though it clearly has an efect on the classification. Let us note that nonmonotonic behaviour naturally occurs in many domains. For example, when predicting health risks, the probability often increases when particular health markers deviate substantially from a default value. For instance, both underweight and overweight and both low and high blood pressure could be seen as red flags. For spatial features, even multiple changes in the behaviour of a classifier can occur. One example are latitude and longitude when predicting property demand in real estate datasets because areas with high and low popularity are often spread throughout cities.
In order to deal with non-monotonicity, we basically have to consider value intervals for features rather than just features. For example, C1() in Figure 1 is decreasing on the open interval (−∞ , 0) and increasing on (0, ∞). However determining these intervals can be challenging in general. For example, when the classifier is a large neural network, it is hard to tell at which points the model changes its behaviour. Taking many small intervals will increase the chance that the model behaves monotonically on the interval, but can potentially blow up the explanation unnecessarily. Taking too large intervals increase the chance that the model is non-monotonic on the interval, so that the explanation may miss important efects (reducing completeness) or may misrepresent the efect (violating reinforcement). However, let us note that finding good intervals is easier for discrete classifiers like tree ensembles. This is because they internally discretize continuous features, so that the potential critical points can be found by traversing all trees in the ensemble.
To summarize, for continuous features, we do not only have to take account of joint efects of features, but also of their potential non-monotonicity. In addition to the two refinements proposed at the end of the previous section, we may consider the following variations to improve completeness when features are continuous.
1. Increase the BAG: Instead of having a single feature argument per feature, we can consider multiple arguments that represent the influence of the feature in a particular region. 2. Consider more expressive BAGs: similar to the boolean setting, joint attacks/supports can capture the joint efects of continuous feature arguments that take a value in a particular region or conditional efects based on the state of boolean feature arguments.
Before closing the paper, let us briefly discuss a suficient condition under which the naive explanation BAG can accurately represent a probabilistic classifier. The condition that we consider here is that the classifier behaves monotonically with respect to every individual feature (independent of the remaining features). We call this property Strong Monotonicity. To talk about monotonicity, we have to assume that all domains are ordered. Note that this applies to boolean features (0 < 1) as well as discrete ( < + 1) and continuous features. Formally, we call a classifier monotonically increasing (resp. decreasing) wrt. the label and the -th feature if for all inputs x ∈ , and ′ ∈ , < ′ implies that C(x) ≤ C((x− , ′)) (resp. C(x) ≥ C((x− , ))) and > ′ implies that C(x) ≥ C((x− , ′)) (resp. C(x) ≤ C((x− , ))). If these conditions hold with strict inequality, we call the classifier strictly monotonically increasing (resp. decreasing) wrt. and the -th feature. We call a classifier strongly monotonic if for every label and for every feature, the classifier is strictly monotonically increasing or strictly monotonically decreasing.
Let us note that if our data is boolean and the classifier is strongly monotonic, then the naive
explanation BAG accurately captures all efects by definition. If we have continuous features,
then building up the explanation BAG can be dificult because an edge can only be added when
the reinforcement condition can be verified for all domain values. As the domain is infinite,
this is not always possible. However, if the classifier is composed of diferentiable functions
like in the case of many neural networks, then it is theoretically possible to apply symbolic
diferentiation techniques to compute the partial derivatives with respect to all features. Note
that the classifier is strongly monotonic if and only if all partial derivatives depend only on the
feature itself (independence) and are guaranteed to be always negative (decreasing) or positive
(increasing). The conditions for the naive explanation BAG can be checked analogously. is
an attacker (supporter) of if the partial derivative of ︀( x) with respect to x depends only
on x and is always negative (positive). In this setting (continuous data, diferentiable classifier),
the gradient does indeed have several advantages over other feature attribution methods that
evaluate features by numerical scores [
Formally, it would be interesting to characterize exactly under which conditions certain explanation BAGs can correctly and completely capture the behaviour of classification models. Intuitively, it seems that the naive explanation BAG can satisfy both reinforcement and completeness if and only if the classifier satisfies a notion (probably a refinement) of strong monotonicity. However, a formal investigation requires a more rigorous definition of completeness that is out of scope here.
We revisited the notion of reinforcement of bipolar abstract argumentation classifiers. While the property is important, we believe that it should be considered in combination with a completeness property that guarantees that reinforcement is not satisfied in a trivial way. As we illustrated with some examples, reinforcement and completeness cannot be satisfied simultaneously when the explanation BAG is not suficiently expressive. We identified two main reasons for this: 1. The efect of features cannot always be determined independently of the state of other features. 2. The efect of features can be non-monotonic and may be positive in one region and negative in another.
For these reasons, a naive explanation BAG that only considers individual features and their efects on class labels, is necessarily incomplete or incorrect. Possible workarounds include adding joint-features or joint-attacks and -supports, and discretizing continuous features into bins on which the classifier behaves monotonically. However, the latter task can be challenging for some models, and one may have to sacrifice correctness to improve completeness. One notable exception may be decision tree ensembles where the critical points can directly be extracted from the trees in the ensemble.
An interesting research direction corresponding to the correctness-completeness-tradeof is to characterize for which classification models we can find explanation BAGs that satisfy both reinforcement and completeness. We conjecture that, for the naive explanation BAG, the characterizing property of the classification model is some notion of strong monotonicity, that is, the classifier should be 1. monotonic with respect to every feature, 2. independent of the state of the other features.
Finally, let us note that another way to guarantee an acurate representation of classification
models is to establish a one-to-one relationship between classifiers and argumentation
frameworks. For example, many multilayer perceptrons can be perfectly represented by gradual BAGs
[
This research was partially funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 101020934). Francesca Toni was partially funded by J.P. Morgan and by the Royal Academy of Engineering under the Research Chairs and Senior Research Fellowships scheme. Any views or opinions expressed herein are solely those of the authors listed.