Post hoc analyses are used to provide interpretable explanations for machine learning predictions made by an opaque model. We modify a top-level model (AF-CBA) that uses case-based argumentation as such a post hoc analysis. AF-CBA justifies model predictions on the basis of an argument graph constructed using precedents from a case base. The efectiveness of this approach is limited when faced with an inconsistent case base, which are frequently encountered in practice. Reducing an inconsistent case base to a consistent subset is possible but undesirable. By altering the approach's definition of best precedent to include an additional criterion based on an expression of authoritativeness, we allow AF-CBA to handle inconsistent case bases. We experiment with four diferent expressions of authoritativeness using three diferent data sets in order to evaluate their efect on the explanations generated in terms of the average number of precedents and the number of inconsistent a fortiori forcing relations.
Both machine learning (ML) and rule-based classification approaches involve a trade of between accuracy and transparency, specifically the ability of end-users to understand decisions (class predictions) [1]. Deep neural networks in particular tend to produce predictions with a high degree of accuracy at the cost of transparency due to their technical complexity. However, the perceived complexity may vary according to a person’s level of understanding, so even much simpler approaches might be thought of as relatively opaque by some people. Another reason for poor transparency can be proprietary protection of the approach, which can render even a relatively simple approach opaque. Regardless of the underlying reason, the term ‘black box’ is often used to refer to a particularly opaque approach [1, 2]. A black box model is more dificult to trust. It is harder to see its shortcomings, including biases and ethical concerns, which is why Explainable Artificial Intelligence (XAI) is aimed at increasing the transparency of black box models [3]. In the case of a binary classification problem in ML, this entails that we can explain why one class label was predicted by the model and not the other in a particular instance.
Methods for explaining ML decisions vary in a number of respects. A distinction can be made between methods that generate local explanations (explaining individual instances) and those that generate global explanations (explaining a whole model). Some methods have access to the learnt model, while others are model agnostic. We use the term ‘justifications’ for explanations generated without model access to signify that such explanations do not explain exactly how a decision was reached, but instead they explain the assumptions under which the model’s decision can be justified. Justifications are thus not intended as separate predictions. This is related to the notion of post hoc analysis, which implies that an explanation is produced after the fact [1]. In this paper, we are concerned with local justifications produced by a model-agnostic, post hoc analysis for binary classification.
One way to justify a classifier’s prediction is to show a most similar case to one whose class is being predicted (the focus case). Pointing out a similar case constitutes an argument that requires no knowledge of ML to understand. To this end, Prakken & Ratsma [4] draw on AI & law research to propose a top-level model using case-based argumentation (CBA) to explain black-box predictions based on Horty’s model of a fortiori reasoning [5, 6] and inspired by CATO [7], hereafter referred to as ‘A Fortiori Case-Based Argumentation’ (AF-CBA). As ML classification is a supervised approach, there exists a training set used to train a classifier. AFCBA requires that this set be accessible. AF-CBA produces a human-interpretable justification of that classifier’s binary prediction for a focus case by treating this training set as a case base (CB) and comparing precedents and their outcomes from that CB to the focus case. The underlying a fortiori assumption of AF-CBA is that the focus case should have the same outcome as a precedent case if the diferences between these cases only serve to add further support for that same outcome [4].
When asked to provide a justification, AF-CBA constructs an argument graph through a grounded argument game consisting of a fixed set of allowed moves. A proponent defends why the focus case should receive the same outcome as a best precedent (a most similar case) and the opponent argues against this. In doing so, they cite examples and counterexamples from the CB. There are distinguishing moves that set cases apart and moves to downplay these diferences. When a precedent case has no relevant diferences with the focus case, deciding for the focus case is said to be ‘forced’. The efectiveness of AF-CBA hinges in part on the distance measure between cases and any feature selection technique used to promote an interpretable argument graph [4].
CBs may contain inconsistencies. Because training sets are used as CBs, they constitute annotated data, i.e. data instances labelled by a person or process with the intention of allowing the ML model to learn to perform the same classification task. Annotators (people who label data to this end) produce a labelled data set specifically for the purpose of training a model, but may not necessarily be fully consistent when doing so [8]. Multiple annotators might disagree or an annotator can make an occasional mistake, thus leading to an inconsistent case base. Labels may also be produced by decision makers—people who produce labels as part of their role in some decision process, such as judges who decide on court cases, with their verdict being the label that is stored in a body of case law. This can also lead to contradictory classifications, as case law can contain conflicting opinions and interpretations. Finally, the feature vector itself may be a subset of all relevant details, thereby potentially lacking necessary data to discriminate between seemingly similar cases [9]. These sources of noise make the labelling seem inconsistent, since identical feature vectors might receive conflicting labels. Under the a fortiori assumption, this notion of inconsistency becomes even broader: a case which is at least as good as another yet receives the opposite outcome is a source of inconsistency. For these reasons, CB consistency is generally not a safe assumption in practice.
AF-CBA does not strictly require that the CB be consistent, but inconsistencies are often due to exceptional cases (with a surprising outcome) and these can be problematic for the explanation due to the focus case being forced for both outcomes. In experiments by Prakken & Ratsma [4], significant portions of a CB had to be ignored (by removing a minimal number of cases when instantiating the CB) in order to make them consistent—namely 0.32%, 11.35% and 3.20% for three diferent inconsistent data sets. We would preferably use the whole training set as a CB, without having to take this consistent subset to circumvent the problem. The problem is exasperated by feature selection techniques, which would otherwise benefit the simplicity of AF-CBA’s explanations. In conclusion, CB consistency forms a problematic constraint for AF-CBA.
In this paper, we present a modification of AF-CBA that takes into account the degree (which we call ‘authoritativeness’) to which the CB is consistent in regards to a best precedent. This measure is used to prevent inconsistent forcing by modifying the selection of best precedents to cite, as it makes intuitive sense to cite cases with the highest authoritativeness. We investigate the desirability of this modification through exploratory experiments with several possible alternatives of quantifying authoritativeness, demonstrating it to have a beneficial efect on AF-CBA without adversely afecting its explanations. The rest of this paper is structured as follows. We describe AF-CBA and its background in Section 2. We consider how to address the problem of inconsistency in Section 3. We subsequently experiment with our proposed solution in Section 4. We discuss the results and future work in Section 5.
In this section, we present the CBA framework by Prakken & Ratsma (with some diferences in notation) for explanations with dimensions [4]. As our running example, we make use of the Telco Customer Churn data set [10], which describes the customers of a telecommunications provider and whether or not they churned (switched providers). Table 1 describes the dimensions (‘features’ in ML) used. The optional superscript arrow reflects the tendency of a dimension, i.e. whether a higher value promotes a result of 1 for the class label. Here, only the dimension of ℎℎ makes it likelier for a customer to churn; the other three dimensions make it less likely for a customer to do so.
We take Table 2 as our example CB. Let us presume that Alice and Bob are previous customers and Charlie is a new customer whose predicted outcome we want to justify (the focus case). Formally, we denote this as follows. Let and ′ be the two possible outcomes of a case in the CB. The variables and ¯ denote the two sides, meaning that = if ¯ = ′ and vice versa. A dimension is defined as a tuple = (, ≤ , ≤ ′ ), with value set and two partial orderings on , ≤ and ≤ ′ , such that ≤ ′ if ′ ≤ ′ for , ′ ∈ . A value assignment is a pair (, ). We denote the value of dimension as (, ) = for case ∈ . Value assignments to all dimensions ∈ (where is nonempty) constitute a fact situation . A case is defined as = (, ()) for such a fact situation and an () ∈ {, ′}. In this context, a case base specifically refers to the set of cases with value assignments for . We denote the fact situation of a case as (). In the rest of this paper, we assume that any two fact situations assign values to the same set .
Say we have a ML model that predicts Charlie will stay. An explanation of this outcome could be that it is forced by another case. We model Horty’s [5] a fortiori reasoning using Definitions 1 and 2, meaning that the outcome of a focus case is forced if there is a precedent with the same outcome where all their diferences make the focus case even stronger for that outcome. Definition 1 (Preference relation for fact situations). Given two fact situations and ′, ≤ ′ if ≤ ′ for all (, ) ∈ and (, ′) ∈ ′.
Definition 2 (Precedential constraint). Given case base and fact situation , deciding for is forced if CB contains a case = ( ′, ) such that ′ ≤ .
A fact situation could be forced for both and ¯, which brings us to the following definition of CB consistency: Definition 3 (Case base consistency). A case base is consistent if it does not contain two cases = (, ) and ′ = ( ′, ¯) such that ≤ ′. Otherwise it is inconsistent.
An explanation takes the form of an argument game for grounded semantics [11] played between a proponent and opponent of an outcome, in which they take turns to attack the other’s last argument. Since neither of the cases in Table 2 is identical to the focus case, it is not forced and the proponent and opponent argue about the outcome. An argument is justified if the proponent has a winning strategy, meaning the opponent runs out of moves. The proponent starts by citing a best precedent. This is a case which has the outcome for which the proponent is arguing and has a minimal subset of relevant diferences with the focus case. Determining the relevant diferences between two cases is defined according to Definition 4 (as presented in [ 4]). Definition 4 (Diferences between cases) . Let = ( (), ()) and = ( ( ), ( )) be two cases. The set (, ) of diferences between and is: 1. (, ) = {(, ) ∈ () | (, ) ≰ (, )} if () = ( ) = . 2. (, ) = {(, ) ∈ () | (, ) ≱ ¯ (, )} if () ̸= ( ) and () = .
So one relevant diference between Charlie (assuming he is predicted to stay) and Alice (who stayed) in Table 2 would be for dimension 1, where Alice received a gift and Charlie did not, making her case better for staying.
Definition 5 (Best precedent). Let = ( (), ()) and = ( ( ), ( )) be two cases, where ∈ and ∈/ . is a best precedent for if: • () = ( ) and • there is no ′ ∈ such that (′) = () and (′, ) ⊂ (, ).
Definition 5 defines a best precedent to cite. Multiple cases can meet these criteria. A lower number of best precedents is preferable, because of computational reasons and because one could say that a higher number of possible citations would make a single explanation somewhat arbitrary. This is why Prakken & Ratsma evaluated AF-CBA in part on the average number of best precedents found for three diferent data sets [4].
The opponent can reply to the initial citation by playing a distinguishing move or by citing a counterexample. The proponent can reply in turn with similar distinguishing moves. The distinguishing moves are (, ) — the focus case is on some dimensions worse than the precedent for () — , (, , ) — the dimensions on which the focus case is not at least as good as the precedent for () are compensated by dimensions on which the focus case is better for () than — and (, ′) — the initial citation of a most similar case for ( ) can be transformed by the distinguishing moves into a case for which (, ) = ∅ and which can therefore attack the counterexample. For the sake of brevity, see [4] for formal motivations of these moves and the need to allow the move to be empty in order to state that the diferences with the focus case do not matter.
Returning to our example, Figure 1 presents the resulting explanation as an argument game, which can be read as follows. P1: Alice stayed and her case is similar to Charlie’s. O1: Charlie’s scores for ↓1 and ↓3 make him worse for staying than Alice. P2: Charlie’s score for ↓ 2 compensates for O1. O2: Bob churned and his case is similar to Charlie’s. P3: Charlie’s score for ↑4 makes him worse for churning than Bob. O3: Charlie’s score for ↓1 compensates for P3. P2: Charlie’s score for ↓2 compensates for O3. After this, the opponent has run out of possible moves to make and the proponent wins. The similarity to Alice’s case has held up and acts as a explanation for the prediction that Charlie will stay as well.
Formalising this brings us to the following definition (after [ 4]) for the AF-CBA framework: Definition 6 (Case-based argumentation framework). Given a finite case base , a focus case ∈/ , and definitions of compensation , an abstract argumentation framework AAF is a pair < , >, where: • = ∪ , with = { (, ) | ∈ , ̸= ∅ and = {(, ) ∈ ( ) | (, ) <() (, )}} ∪ {(, , ) | ∈ , ⊆ { (, ) ∈ ( ) | (, ) <() (, )}, = {(, ) ∈ ( ) | (, ) <() (, )} and compensates according to } ∪ { (, ′) | ∈ and can be transformed into ′ and (′, ) = 0} • attacks if: – , ∈ and () ̸= () and (, ) ̸⊂ (, ); – ∈ with () = ( ) and is of the form (, ); – is of the form (, ) and is of the form (, , ); – ∈ and () ̸= ( ) and is of the form
(, ′).
In summary, AF-CBA provides justifications for individual binary classifications predicted by a ML model by presenting a winning strategy for a grounded argument game in favour of the predicted class label. This winning strategy represents a dialogue between a proponent and opponent on the basis of citations from the labelled training set (the case base) and shows how the opponent runs out of moves and the proponent thus wins the argument.
As we argued in Section 1, CB consistency is not always a safe assumption to make. Explanations containing inconsistent forcings essentially explain that a decision cannot be justified without acknowledging the inconsistency of the CB, which weakens the value of those explanations. The larger the number of inconsistent forcings (), the larger the number of explanations where this problem occurs.
Instead of mitigating the problem through case deletion [4], we explicitly take inconsistencies into account. Informally, one might say that when there is consistency, a precedential case has a strong backing when cited and should indeed immediately force the outcome; if there is inconsistency, it has less backing and thus should not. We therefore introduce the concept of ‘authoritativeness’, by which we mean that, given any case ∈ , the authoritativeness () numerically expresses (normalised between 0 and 1) the degree to which the rest of the CB supports the citing of for (). We subsequently use () as an additional criterion in the selection of best precedents. The intuition behind authoritativeness is that whereas the a fortiori rule applied to a consistent CB can be expressed as the phrase ‘cases like this always receive outcome ,’ our idea of authoritativeness changes this phrase to ‘cases like this usually receive outcome ’—where ‘usually’ has to be quantified in some manner which expresses the inconsistency of the CB with regards to the focus case. Since () is a number, we can have a total ordering ≤ on the authoritativeness of cases.
Table 3 is another instance of our Churn example. Depending on how one chooses to define (), 1 and 2 should arguably receive a higher value for () than 3 due to 4 having the opposite outcome.
First of all, the definition of best precedent has to be modified to reflect the additional criterion of maximising the authoritativeness:
= ( (), ()) and = ( ( ), ( )) be two cases, where ∈ and ∈/ . is a best precedent for if: • () = ( ), • there is no ′ ∈ such that (′) = () while (′, ) ⊂ (, ) and (′) ≥ ().
In order to quantify authoritativeness, we require expressions of agreement and disagreement between a precedent and the rest of the CB: Definition 8. (Agreement) Let be a case base. Given ∈ , the agreement () is defined as: () = | {′ ∈ | (′) = () and (, ′) = ∅} | Definition 9. (Disagreement) Let be a case base. Given ∈ , the disagreement () is defined as: () = | {′ ∈ | (′) ̸= () and (, ′) = ∅} |
We understand () as the number of cases which have the same outcome as the precedent case and are at least as good for that outcome as (thereby lending support to ). Similarly, () is the number of cases which have the opposite outcome yet are at least as good for (). The agreement () has at least a value of 1 due to itself being a member of the CB. The disagreement () can have a value of 0.
Exactly how the level of agreement relates to authoritativeness is not self-evident, as various
expressions may have equal merit. For example, given a case ∈ , we could express the
authoritativeness () as the relative number of cases which lend further support to (
() =
() () + ()
However, this overlooks any intuitive understanding of authoritativeness which stems from
the absolute number of cases that can act as precedents (
() =
()
| |
(
Both (
Alternatively, (
() () ()+() · || ()+() + () () ||
How desirable each expression is, is dificult to say. In the next section, we attempt to answer
this question through experimentation. However, two observations can be made here regarding
the four expressions. One is that () = 1 implies that the CB is consistent with regard to ,
but only in case of (
Proof. Recall that is consistent with regards to if there exists no other case ′ ∈ with
() ̸= (′) and () ≤ () (′). Recall also that this can be expressed
as () = 0 and that () ≥ 1 for any ∈ since is in agreement with itself. Suppose
that () = 1 according to (
The other observation is that () = 0 is not obtainable for any of the expressions for authoritativeness.
0 for any ∈ .
Proof. Recall that () is the cardinality of the set of cases for which the condition holds that
() = (′) and (, ′) = ∅ for ′ ∈ . Suppose that () = 0. Then
() = 0 when evaluating () according to (
These observations do not afect our evaluation in the next section and do not form a limitation of our current approach, but they do result in the following question regarding the intuitive understanding of authoritativeness: should the minimum value of 0 and maximum value of 1 for () be of significance? If so, this would steer our choice for an expression for authoritativeness. We return to this point in Section 5.
Using authoritativeness as a criterion when selecting a best precedent is intended to improve the ability of AF-CBA to generate useful justifications in light of CB inconsistency. Since AF-CBA generates justifications for the same outcome as the ML model predicts, we cannot use ifdelity (the agreement between an XAI approach and the ML model it explains) to assess the eficacy of our modification. Evaluation of our approach therefore requires a more investigative experimentation and interpretation of the results.
To this end, we follow a similar strategy to Prakken & Ratsma [4]. We also rely on the same data sets as they do, namely Graduate Admission [12], Telco Customer Churn [10] and Mushroom [13]. As an expression of how inconsistent each data set is, we determine the minimum number of case deletions required to make each CB consistent. The result is 26 (3.20%), 647 (9.20%) and 16 (0.32%) for the Admission, Churn and Mushroom data set, respectively. The tendencies of all dimensions are determined using the Pearson correlation coeficient. Prakken & Ratsma [4] attempt to gain insights into the feasibility of AF-CBA in terms of the justifications themselves and in the treatment of inconsistencies. As they explain, desirable characteristics for AF-CBA include fewer best precedents (reducing the solution space for citing a precedent, see Section 2). This is one of the metrics on which we compare our four alternative formulations of authoritativeness to the base method. We treat each case in the CB as a focus case and compute the number of best precedents for that case given the rest of the CB, reporting the mean number ( ). Changes in would depend on the average distribution of inconsistent cases per focus case. There is no well-motivated cut of point for when these numbers become too high, but it is worthwile to consider whether increases by orders of magnitude and whether there are surprising diferences between the alternative formulations of authoritativeness.
We also report the number of inconsistent forcing relations () given each experiment. As described in Section 3, this is the number of forcing relations between two cases that contradict each other on the outcome. = 0 for a consistent data set and our intention is to achieve this without having to take a consistent subset of the data. We therefore expect to drop to very low numbers (if not zero) for all experiments where we make use of authoritativeness.
We present the results of these experiments1 in Table 5. A qualitative assessment of these
results suggests that inconsistent forcing is indeed largely avoided by taking the
authoritativeness of precedents into account, without having a costly impact on the best precedent
distributions. The relative version of authoritativeness (
Post hoc analyses often constitute classifiers themselves, although evidently worse than the actual models (or they would be used as the models instead). This is not the case with AF-CBA. One can still hold the view that a simpler albeit more transparent model is preferable to a post hoc analysis. However, it is our experience that this is often unfeasible, as there exist many problems for which the only satisfactory solutions are too opague—especially for people who are not researchers or data scientists. It is our belief that this warrants the use of post hoc analyses in many situations.
Our modification of AF-CBA relies on the intuition that one precedent can be more authoritative than another. We demonstrate its consequences in numerical terms ( and ), given that lower numbers indicate better explanations (as was argued in the original paper [4]). However, it could be argued that these metrics are but proxies for the capacity to justify ML predictions in an intuitive fashion. Testing this would require a usability study to evaluate the explanatory power and interpretability of various explanations. A variety of alternative modifications and additional metrics could then be compared to study their eficacy in a real-world setting.
None of our expressions for authoritativeness would ever reach a value of 0 for any case in the CB. This seems intuitive, since any case should have at least some authoritativeness simply due to its being a precedent. A value of () = 1 is only realistic when using our relative expression of authoritativeness 1. This would only be a problem if values due to diferent expressions of authoritativeness would have to be compared to each other, which is not the aim of our method. If multiple explanations are ever to be compared as part of some overarching approach, these (and possibly other) characteristics of alternative authoritativeness expressions would have to be taken into account.
Additional modifications to AF-CBA could include other criteria for ranking precedents, incorporating complex arguments in the explanations (AF-CBA is qualified as a ‘top-level’ model due to the possibility of providing it with a set of definitions as to why specific downplaying moves can be played) or accounting for dimensions which are highly dependent. Another possibility is an alteration that allows dimensions to have a more complex efect on predictions than the tendencies used in this paper. There exist binary classification tasks for which this would be desirable. For example, a dimension such as blood pressure could be a predictor for illness both at very low and very high values, with a value in the intermediate range being a predictor for the patient not being ill. We intend to include this in our future work.
In this paper, we have presented an extension of an earlier top-level model (AF-CBA) for case-based argumentation used to provide post hoc justifications for opaque machine learning predictions. We have modified its definition of best precedent to include a quantified expression of how authoritative that precedent is, thereby afecting which cases are likely to be cited. This is not strictly in conflict with the a fortiori assumption underpinning the approach. Instead, it recognises the limitations of that assumption in light of the inconsistency that can be expected from real-world case bases. We have experimented with multiple versions of this expression to study which appears to be the most fruitful regarding the handling of inconsistency without adversely afecting the explanations. Our evaluation suggests our two somewhat more elaborate expressions of authoritativeness are more suitable. Future work is to be aimed at other modifications and usability studies.
Acknowledgements The authors would like to thank the anonymous reviewers for their feedback and suggestions.