Case-based explanation (CBE) goes by many names, and in this paper I will argue why we should tend towards alternative choices when designing XAI systems. My argumentation for the stated claim rests on four broad points: (1) people seem to dislike CBE; (2) CBE relies on weak semantic linkage; (3) CBE is epistemically outmatched; (4) CBE is restrictive. This paper expounds on these arguments and concludes with thoughts about characteristics of possible alternatives.
What is case-based explanation? Broadly speaking,
casebased explanation is the notion of providing example(s)
from the training data which are “similar” to the instance
being explained, for some definition of similar. I have
deployed case-based explanation [
To ground our discussion, here are examples of CBE:
identical to Iliana; 6 of them reofended (60%)” [
Gradient Boosting Machines 59.5
80.3 74.1 39.6 Convolutional NNs 28.7 Bayesian Things 27.6 Dense NNs 26.7 Recurrent NNs 17.1 Transformers
≫
Thus, explanations by perturbation (e.g., Sensitivity from
With examples in hand, I’d like to draw a distinction Binns [
Conjecture: It is a bad idea to deploy Case-Based Explanations when not using -Nearest-Neighbors, or similar.
1Disclaimer: other criticisms raised in the paper may still apply when using CBE for -NN—but at least it is sound then.
propriateness, fair process perception, and (in the
loans case) deservedness, consistently compared to
sensitivity based styles and occasionally compared to
other styles. This is an effect primarily observed, like
most efects in the quantitative part of our study, in
the within subject study design, indicating that the
act of comparison in a particular scenario is
important for these diferences to become apparent.
Casebased explanations seem to have the most
consistent negative impact on justice perceptions when
presented alongside alternative explanation styles.” [
When we created a program to generate explanations
following the same four templates, we observed the same
result—CBE was the worst of the bunch [
Low soundness alone might be enough earn a “bad
idea” label. One consequence of such labelling is that
sometimes bad ideas should remain unused, as Correll [
Some readers may ask, “Why do you care enough to write this paper?” I was presenting an invited talk and during Q&A had the following exchange while answering a question on the explanation types just presented:
erally a very bad idea. Part of the reason I say this is, ifrst of all, Binns, when they researched case-based explanation, they found it to be the least preferred. We also found that [result]. And then also it has these issues you kind of saw, with that self-refuting explanation. The reason that is happening is because it is not sound. The classifier isn’t a K-means clusterer or something, and so when your explanation is assuming that structure to the classifier, you are lying to people” Interviewer (Ali El-Sharif): “You just shocked me, by the way. I always thought case-based explanations were more intuitive, that they presented something logical2.”
As mentioned, Binns, et al. [
More specifically, in those authors’ words: “Tukey’s post-hoc paired tests showed that casebased explanations result in lower perceptions of ap2https://www.youtube.com/watch?v=bD5_q2t-4S8, timestamp ≈ 37:00
3https://royalsociety.org/-/media/policy/projects/
explainable-ai/AI-and-interpretability-policy-briefing.pdf
Afterwards, I decided that perhaps these thoughts were
worth developing further and writing about it. As
negative about CBE as I was at the time of this exchange, the More recently, van der Waa, et al. [
4“We found that 16% of the test data exhibited the failure mode of
contradicting the claim (< 50% of individuals with identical features
share label).” [
B B
A A
? C C
A
B C
As evidence that few researchers conduct user eval- the neighborhood contains items, regardless of label;
uations, consider that Keane and Kenny [10] surveyed (3) expand until the neighborhood contains items, of a
1,102 papers on “post-hoc explanation by example” be- particular label (as in [
CBE and an alternative, to the extent we have evidence The distance problem only gets worse if we assume at all, most of it seems to suggest that users will prefer the feature space is impoverished, which is typically true the alternative. of decisions involving humans and many other applications which violate the closed-world assumption5.
It seems probable that every explanation style will suf2. CBE Relies on Weak Semantic fer when deployed under violation of this assumption.
Linkage However, it also seems that CBE will fare particularly badly—due to reliance on distance. Suppose my current Under the hood, CBE relies on some notion of distance, representation uses dimensional features, and a (posas Sarkar et al. [11] explain: sibly theoretical) perfect representation uses + dimensions. To convert between the representations can “Since in order to explain the -NN algorithm’s be- be viewed as a vector space projection. However, projechavior it sufices to represent proximity, rather than tions can mangle distances when not done carefully6, as variation along any particular dimension, we sac- Figure 3 illustrates. rifice concrete interpretations of the spatial axes in To be concrete, consider using CBE in a domain like favor of expressing “nearness” and “farness”.” chess. This might make perfect sense, because the domain is fully captured by the representation. As a result, one might expect nearby points in feature space to have a strong semantic linkage, e.g., the same board has the same good actions available, a similar board might be a successor, predecessor, or sibling.
To contrast, consider Example #2 and implications of the closed world assumption. The explanation contains only some information about driving history, while one could imagine many other features being useful to predict insurance costs, such as risk tolerance, alcohol consumption, etc. This is not to say that such information should be obtained, merely that we should be wary of over-relying on distance when we know in advance that the features are incomplete. The reason is that the strength of the semantic linkage of distance decreases as the feature set gets further from a total representation.
Burdens of Proof The spectrum of certainty
Absolute Certainty Beyond a Reasonable Doubt Clear and Convincing Evidence Preponderance of the Evidence Probable Cause Reasonable Suspicion
Mere Speculation
Due to unsoundness, CBE can generate self-refuting meanings with an example, based on a tale about bank explanations—a form of epistemic mismatch. Footnote 4 robber Willie Sutton [14], and informed by definitions hints at this efect, Dodge and Burnett [ 12] previously found in Sørmo, et al [15]: argued this more thoroughly (see Figure 2 in that paper). To briefly restate that argument: weak evidence Willie Sutton, a bank robber, was asked why he occurs when the neighborhood around the instance to robbed banks, and his response was “That’s where be explained contains mixed examples (“I labelled this an the money is.” This description just ofers a fact , 50% of nearby things are s.” ); contradictions occur about the context of banks—notably the story does when it contains few or no examples of the same label not include the explainer. An example transpar(“I labelled this an , 10% of nearby things are s.” ). ent answer might be: “I wanted the money in the
By epistemic outmatch, I refer to the strength of the bank.” The latter example contextualizes the acclaim far exceeding “burden of proof.” An assessor con- tion with respect to why the actor performed it, suming explanations might be trying to determine “If it is explaining how the system reached an answer [15]. ift for the purpose” [13]. Here, this is a fairly strong claim, Note that to justify that action, the explanation and so the legal regime might want a high burden of would need to ofer a reason as to why it is good proof, depicted in Figure 4. Does consuming CBE confer for Willie to want to rob banks. “absolute certainty”, akin to a mathematical proof?
CBE is definitely not proof. To clarify, CBE is essen- Some explanations do not change based on the instance tially a form of “proof-by-example,” which is a known to be explained, and so are called global explanations. logical fallacy7. Instead of as “proof-by-example”, could As a result of being static in this way, it seems impossiCBE be considered disproof-by-counterexample, which ble for a single global explanation to ofer transparency is a valid proof technique8? Occasionally, yes; however, into every decision. However, while this can be taken many ML/AI systems do not support such formalism, as as evidence that being a local explanation is a necessary they are statistical machines. In particular, even a 99.9% condition for transparency, it is not suficient. In particuaccurate classifier will mishandle specific instances. Even lar, CBE is a local explanation since each instance will the existence of many such examples does not disprove have diferent neighbors. anything—if there are appropriately many more correctly Thus, since CBE merely describes the decision, ofering handled instances. little transparency to aford the assessor introspection
If CBE isn’t proof, and instead is merely evidence, what on the system, it amounts to weak evidence attempting
kind of evidence is it? It describes the decision, providing to support a strong claim. Case dismissed!
potentially important context; but does not justify it, as
stated by van der Waa et al. [
CBE requires two properties to be true of the training data.
First, the training data must still be accessible; second, the explainer must be allowed to present training data to the user, possibly in an anonymized form. ML/AI techniques make predictions in many domains where one or the other criteria is violated, which means there are constraints for when researchers can deploy CBE responsibly.
Accessing training data is easiest when the training data is tabular. Tabular data stands in contrast with unstructured data: “A very naive definition for unstructured data is anything that cannot be put up into traditional rowcolumn or tabular database. The common examples of unstructured data are text or document based data, network or graph data, image data, video, audio, web-based logs, sensor data, etc.9” Concretely, when OpenAI trained GPT-3 [16], they input a vast quantity of unstructured text data—and so the training data would not be tabular. In situation like these, XAI system designers cannot always rely on the training data being cheaply accessible, if at all.
Our second criterion, presenting training data to
assessors, is easiest when the training data is not private. For
example, suppose a robot is delivering objects and that
Alice is the current package recipient, but the robot botches
the delivery somehow. Should Alice be able to request
information about previous deliveries? How about if
Alice were instead a developer? The main goal of privacy
preserving machine learning (see e.g., [17]) is to prevent
leakage of private information. CBE stands in tension
with that, because even if the explanation is anonymized, NOT reoffend
it may provide enough features (e.g., “a combination of
gender, race, birth date, geographic indicator, and other
descriptors” 10) to allow “indirect identification”.
we saw three other alternative textual explanation
templates from Binns, et al. [
Having concluded my argumentation for the conjecture, the reader may ask what they should use instead? Personally, I like the visualization strategies, (e.g., search trees, charts, saliency maps), but recognize that not everyone does. Returning to my geometric framing, there are essentially three entities available for explanation: 1. the decision boundary; 2. the instance to be explained; 3. the training data.
Some explanations use the entities directly, or characterize the relationship between them, or some mixture.
CBE relies heavily on the relationship between (2) and (3), and essentially does not use (1). In the introduction 9https://medium.com/analytics-vidhya/ sql-to-nosql-4dd15ab121b0 10https://www.dol.gov/general/ppii priors count : 0 (------) 1 to 3 (---) 4 to 6 (0) 7 to 10 (+++) >10 (++++) Last, as shown in Figure 5, Sensitivity focuses on the relationship between (1) and (2): If the individual was age ‘‘30 to 39’’ they would have been predicted as likely to reoffend If the individual had priors count ‘‘7 to 10’’ they would have been predicted as likely to reoffend
In conclusion, decision boundaries may be hard to characterize, but neglecting boundaries in explanation seems to expose consumers to possibly falling prey to logical fallacy.
This material is based upon work supported by DARPA #N66001-17-2-4030 and joint support by NSF and USDANIFA under #2021-67021-35344.