Longitudinal Distance: Towards Accountable Instance Attribution Rosina O. Weber1 , Prateek Goel1 , Shideh Amiri3 , and Gideon Simpson4 Information Science, Department of Mathematics, Drexel University {rw37, pg427, ss4588, grs53}@drexel.edu Abstract. Previous research in interpretable machine learning (IML) and explainable artificial intelligence (XAI) can be broadly categorized as either focusing on seeking interpretability in the agent’s model (i.e., IML) or focusing on the context of the user in addition to the model (i.e., XAI). The former can be categorized as feature or instance attribu- tion. Example- or sample-based methods such as those using or inspired by case-based reasoning (CBR) rely on various approaches to select in- stances that are not necessarily attributing instances responsible for an agent’s decision. Furthermore, existing approaches have focused on in- terpretability and explainability but fall short when it comes to account- ability. Inspired in case-based reasoning principles, this paper introduces a pseudo-metric we call Longitudinal distance and its use to attribute instances to a neural network agent’s decision that can be potentially used to build accountable CBR agents. Keywords: explainable artificial intelligence, interpretable machine learn- ing, case-based reasoning, pseudo-metric, accountability 1 Introduction and Background The literature in interpretable machine learning (IML) and explainable artificial intelligence (XAI) have proposed various categorization schemes and taxonomies to describe their methods. One of those is through attribution – attribution methods attempt to explain model behavior by associating a classified (or pre- dicted) test instance to elements of the model (e.g., [28, 32]). There are two main categories of attribution methods. Instance attribution methods select instances as integral elements to associate with a classification or prediction (e.g., [8, 15, 17, 24, 34, 31]). Instance attribution methods have been used for debugging models, detecting data set errors, and creating visually- indistinguishable adversarial training examples [8, 17, 24, 34]. Feature attribution methods select features to attribute to a classification or prediction indicating which features play a more significant role than others [5, 9, 20, 29, 30, 32]. When using case-based reasoning (CBR), a third category is often mentioned: example- or sample-based. CBR can offer instances as examples but it does not make attributions. The most successful use of CBR for explainability or interpretability is in the ANN-CBR twin by [14]. In their work, Kenny and Keane Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 Weber et al. [14] were able to transfer the accuracy of a CNN into CBR for full transparency. This was done by applying the feature attribution approach DeepLift [30]. Both instance and feature attribution methods suffer strong criticisms. DeepLift [30] demonstrated good performance when used in the investigation reported in [14]. However, feature attribution methods have been shown to be insensitive to changes in models and in data [1]. In addition, they do not work in deep learning (DL) models that use a memory [18]. One important limitation of feature at- tribution is how to evaluate them because of how the data distribution changes when a feature is removed [13, 2]. If we extend the analysis beyond interpretability and explainability and con- sider accountability, then feature attribution methods pose more challenges. An accountable decision-making process should demonstrate its processes align with legal and policy standards [19, 25]. This notion is expanded in the literature around two goals. The first is to generate a fully reproducible [19] audit of the system generating the decision (e.g.,[2, 6, 26, 27]); the second is to change the system when its decision is unwarranted (e.g., [11, 25]). The additional challenge to feature attribution lies therefore on the difficulty to change the behavior of a system that produces unwanted decisions based on simply knowing the role of features in each decision. Notwithstanding, it seems plausible to make changes when the instances responsible for each decision are known. Instance attribution methods are also target of criticisms. One criticism has been directed to instances attributed when using influence functions [17] by Brashan et al. [8]. It refers to the fact that the selected instances are mostly outliers, a problem they overcome with relative influence [8]. Another criticism is how time and processing consuming it is to compute influence functions [15]. A third criticism is that selected sets of instances have substantial overlap across multiple test instances to be classified. Consequently, there seems there is still much to gain from further examining instance attribution. This paper proposes a new instance attribution method. It is intuitive and thus simple to explain to users. It is also simple to compute. It has limitations as well. It shares with influence functions the overlap between sets of instances at- tributed to multiple test instance. It shares with ROAR [13] and HYDRA [10] the requirement to retrain the model. It is a step into a different direction that may spur further research. The proposed method is based on a new pseudo-metric we call Longitudinal Distance (LD). This paper focuses exclusively on classifica- tions. For simplicity, we henceforth refer to solutions as classifications and the NN structures as classifiers. In the next sections, we describe the pseudo-metric LD and a non-pseudo-metric variant and how they can be used for explanation. We then present some preliminary studies. We conclude with a discussion and future steps. 2 Introducing Longitudinal Distances Longitudinal distances are a class of distances based on a heuristic that an itera- tive learning model such as NN can be used as an oracle for instance attribution. Longitudinal Distance: Towards Accountable Instance Attribution 3 Longitudinal distances operate in the metric space of instance elements that are used to train and to be solved by NN methods through a series of iterations (or epochs). Given a classification model trained with neural networks (NN) on a space where x ∈ X are instances mapped by features f ∈ F , Xtrain ∈ X are training instances and hence include labels y ∈ Y to indicate the class they belong, and Xtest are testing instances. The classifier Ce (xi ) is a NN classifier that learns to assign labels y using a set of n training instances xi through e epochs, e = 1, . . . , k, xi = 1, . . . , n. We assume that the ability of a classifier to solve a previously unseen instance is represented in the weights learned through a sequence of learning iterations from a set of training instances. We therefore hypothesize that there is at least one (or more) training instance(s) responsible for the classification of an unseen instance, be it correct or incorrect, and that the relation between the training and unseen instances are somehow represented throughout the sequence of learning iterations. We justify the proposed hypothesis based on the fact that when a previously known solution, i.e., a known class, is selected to classify an unseen problem, that both the training instance and the unseen instance, now both labeled as members of the same class, meet some condition that causes their solutions to be interchangeable, that is, their labels are the same. This condition is an abstract concept that we wish to represent, let us call it the oracle condition. We know that two instances together met this oracle condition when they are members of the same class. Consequently, a trained classifier can be perceived as a function that indicates that two instances meet said oracle condition. However a trained classifier may not be ideal to distinguish, among all instances that meet the oracle condition, which ones are responsible for the classification of an unknown instance. We consider, for example, the problem known as catastrophic interference [23] to possibly degrade the quality of a classifier with respect to some classes as it continues to learn new classes. Because some abilities may be forgotten as an NN method moves through new epochs, we contend that the complete experience learned by a classifier is better represented by every weight matrix learned in all epochs of its learning cycle. The last weight matrix represents the learned classifier that has refined its ability to classify all instances through generalization. Therefore, to successfully generalize, it is reasonable to assume that particular abilities with respect to some instances or labels were sacrificed. Hence, we propose to use the sequence of all intermediary classifiers, the weight matrices resultant at the end of each learning iteration, to measure the relationship between an unknown instance and all the training instances and identify the one(s) responsible for the classification. We first propose the pseudometric Longitudinal Distance, dL to produce the ratio of incorrect classifications to the total number of epochs. e=k 1� dL (xi , x) = 1 − ( δe (xi , x)) (1) k e=1 4 Weber et al. Now we propose the non-pseudometric Strict Longitudinal Distance, �� � e=k e=1 we (xi )δe (xi , x) dSL (xi , x) = 1 − �e=k (2) e=1 we (xi ) We define the distance space as (d, X) for the distance pseudo-metric dL (xi , x), along with dSL (xi , x) where xi , x ∈ X are instances mapped by features f ∈ F that are classified by a classifier Ce at epoch e and receive a label y ∈ Y to indicate their outcome class. In the above expressions, δe (xi , x) = 1Ce (xi )=Ce (x) (3) the 1Ce (xi)=Ce (x) in Equation 3 is the indicator function taking the value of 1 when they are equal and 0 otherwise. For assigning relevance, we incorporate we (xi ) as a binary weight to Equation 2 that indicates whether the classifier Ce (xi ) is correct or wrong. When we (xi ) = 1, this means the label y predicted by the classifier Ce (xi ) for xi is equal to the label of the instance and thus it is correct, otherwise we (xi ) = 0. We observe that Equation 1 assumes xi as a training instance and therefore there is a label y designated as its class and thus we (xi ) can be computed. Note that only the correctness of the classification of the training instance is verified as the correct label of unseen instances is unknown. Consequently Equation 2 is suited for computing the distances between a training instance xi ∈ Xtrain and a testing instance x ∈ Xtest , which is the goal of this paper. Note that we can rewrite e=k e=k 1� 1� dL (xi , x) = (1 − δe (xi , x)) = 1C (x )�=Ce (x) (4) k e=1 k e=1 e i which is the mean number of classifier mismatches over all the epochs. Next we demonstrate that dL a pseudo-metric. Recall the pseudo-metric properties: 1. d(x, x) = 0 2. d(x, y) = d(y, x) 3. d(x, y) ≤ d(x, z) + d(z, y) What makes d a pseudo-metric and not necessarily a metric is that in order to be a metric, if d(x, y) = 0, then x = y. Because of mapping into feature space to evaluate the classifier, there may exist predictors, xi and x, for which xi �= x, but d(xi , x) = 0. For dL (xi , x), e=k e=k 1� 1� dL (xi , x) = 1C (x )�=Ce (x) = 0 = 0. (5) k e=1 e i k e=1 Longitudinal Distance: Towards Accountable Instance Attribution 5 The symmetry property is also obvious. For the triangle property, given x, y, and z, suppose Ce (x) = Ce (y). Then, 1Ce (x)�=Ce (y) = 0 ≤ 1Ce (x)�=Ce (z) + 1Ce (z)�=Ce (y) , (6) regardless of what the two terms on the right are. If, instead, Ce (x) �= Ce (y), then 1Ce (x)�=Ce (y) = 1 (7) Now we argue by contradiction. Suppose for some choice of z, 1Ce (x)�=Ce (z) + 1Ce (z)�=Ce (y) = 0 (8) For this to be true, each of these two terms must be zero, so Ce (x) = Ce (z) = Ce (y) (9) But this contradicts the assumption that Ce (x) �= Ce (y). Thus, 1Ce (x)�=Ce (z) + 1Ce (z)�=Ce (y) ≥ 1, and 1Ce (x)�=Ce (y) ≤ 1Ce (x)�=Ce (z) + 1Ce (z)�=Ce (y) (10) Consequently the triangle equality holds for each epoch, and it must then hold over any finite sum of epochs: e=k e=k 1� 1� dL (x, y) = 1Ce (x)�=Ce (y) ≤ 1C (x)�=Ce (z) +1Ce (z)�=Ce (y) = dC (x, z)+dC (z, y) k e=1 k e=1 e (11) Consequently, this is a pseudo-metric. 2.1 Strict Longitudinal Distance The variant Strict Longitudinal Distance considers a relevance weight based on how correct the classifier Ce (x) is throughout its life cycle. If the we were independent of the xi and x arguments, then the analogous computations would demonstrate dSL , Equation 2, to also be a pseudo-metric. However, because of dSL ’s dependence on the binary weights, we (xi ), on the first argument, xi , it fails to be a pseudo-metric when x ∈ Xtest is an element of the training set. It does, however, satisfy the properties that dSL (x, y) ≥ 0 and dSL (x, x) = 0, so it provides some weaker notion of distance between points. 3 Explaining with Longitudinal Distances To use the proposed metrics dL and dSL for explaining decisions of an NN, it is necessary that the weight matrices produced at every epoch are preserved. If those have not been preserved, then it is necessary to retrain the NN preserving 6 Weber et al. its weight matrices. Due to its non-deterministic nature, the instance to be ex- plained has to be solved again by the trained classifier for an explanation to be provided. To explain a given target instance, we propose to compute the distance between the target instance and all training instances using the longitudinal dis- tance or the strict longitudinal distance. With the results of distances, it is then possible to determine the minimum (i.e., shortest) distance observed between the target and training instances. Now note that multiple training instances can be equidistant to the target, and this is exactly why longitudinal distances fail the axioms for being metrics. Definition 1 The shortest distance observed between a given target instance and all training instances computed through longitudinal distances is defined as the explainer distance. Definition 2 The set of instances that are equidistant to the target instance at the explainer distance constitute the positive explainer set. The positive explainer set and the negative explainer set are further specified in that the explainer set computed via longitudinal instances is, by nature of equations (1) and (2), the positive explainer set. The premise of the longitudinal distances may be reversed to compute the negative explainer set by modifying Equation (3) as follows: δe (xi , x) = 1Ce (xi )�=Ce (x) (12) the 1Ce (xi )=Ce (x) in Equation 12 is the indicator function taking the value of 1 when they are not equal and 0 otherwise. Definition 3 The negative explainer set is the set of instances computed via longitudinal distances that represent negative instances in classifying a given target instance. Definition 4 The explainer set results from the combination of the positive explainer set and the negative explainer set. The instances belonging to the explainer set we theorize are responsible for producing the solution to the target instance and consequently can explain its so- lution. Once the explainer set is known, some considerations are needed. First, the explainer distance needs to be defined based on a precision value E. The value for E depends on the domain given that explanations tend to be domain- dependent. Second, a training instance may produce a direct or indirect expla- nation. A direct explanation would be one that does not require any further processing as in all contents of the training instance(s) suffices to explain the target instance. An indirect explanation may require further processing such as comparing whether all features of the target instance match the instances of the training instance chosen to explain the target. Third, the explainer set may Longitudinal Distance: Towards Accountable Instance Attribution 7 include one or more training instances. When cardinality is greater than one, then a process to select the most loyal explanation may be needed. As a result of this process, it may be that no explanation is given. Fourth, if the explanation is meant to foster human trust, the explainer set may need to be interpreted. When the set is meant to produce accountability, then it has to be logical. We note that the negative explainer set is needed to demonstrate positive and negative instances that could be potentially used to train a classifier. When used to select the best candidate for explanation, the positive explainer set suffices. 4 Studies Fig. 1. Image of 9 on the left is the target instance, the set circled in red is the positive explainer set with all members at dL = 0, the explainer distance for the positive set. The set circled in blue is a sample of a 3,569 (0.04%) image instances at the negative explainer distance dL = 1 4.1 Visual Inspection We used the MNIST data set consisting of 55k training instances, and 10k testing instances. We implemented a convolutional neural network design consisting of 1 convolutional layer with 28 3x3 kernels, 1 max pool layer of size 2x2 and stride of 1. The resultant layer is flattened and then connected to 1 hidden layer with 128 nodes with ReLU activation function for every node. A dropout of 20% is introduced following the hidden layer. The resultant output is passed to the output layer with 10 nodes (each corresponding to the label) with the Softmax activation function. This entire network is trained with 55,000 28x28 MNIST 8 Weber et al. images in batches of 64 over 10 epochs. This training reached an accuracy of 83.9%1 . Fig. 1 shows the images of the entire positive explainer set for one of the testing instances from MNIST labeled as a number nine. This positive explainer set was selected for illustration because its cardinality is 12 and thus we can show all the 12 images that are at distance zero from the target testing instance, measured by dL . The negative explainer set contains 3,569 images. Figure set shown circled in blue is a 0.4% sample of the complete set of images at distance 1, which included 3,569 images. Although not a validation of accuracy, images are often presented to illustrate feature attribution methods. In Fig. 1 the negative explainer set is a random selection of the large overall set. In that sample, the types of nines are still nines but they are of a different category from those in the positive explainer set and the target instance. 4.2 Fidelity of explanations with longitudinal distances Fidelity of an explanation is a metric that aims to determine how well an expla- nation generated by an XAI method aligns with ground-truth [4].This concept has been discussed from multiple perspectives (e.g., [3, 7, 12, 16, 21, 22, 32, 33]), and also related to gold features [29]. In this study, we use the data and method from [4]. Their approach is to create artificial data sets by populating equation variables with values based on a given interval so that each instance can be ex- plained by the equation used to create it. This way, an explanation is loyal to the data when it approximates the relationship between the variables in the original equation. In this section, we use a data set created from seven equations. The equations have three variables, which were randomly populated. The result of these equations, the numeric value produced when applying the values and exe- cuting the equation, is the fourth feature of the data set. The label corresponds to the equation, hence there are seven labels, l = {0, 1, 2, 3, 4, 5, 6}. The data set consists of 503,200 instances. The experiment is as follows. Data set and deep neural network classifier. We selected 403,200 instances for training and 100,000 for testing and validation. We trained a deep learning architecture with 1 hidden layer and 8 nodes. The activation for the input and hidden layers was done using ReLU functions and for the output layer using the Softmax activation function. The loss calculated was categorical cross entropy. The 403,200 training instances were trained in batches of 128 over 15 epochs and reached an accuracy of 95% in the testing set. As required to implement the longitudinal distances, we preserved the classifiers at each epoch. Accuracy metric and hypothesis. As earlier described, the data in this experiment was designed with equations that are known and are used to des- ignate the class of the instances. This way, an XAI method produces a correct 1 Code and results are available https://github.com/Rosinaweber/LongitudinalDistances Longitudinal Distance: Towards Accountable Instance Attribution 9 explanation if it selects to explain a target instance with a training instance that implements the same equation with which the instance being explained was built. The explanation will be given by an existing training instance, which has a label. Consequently, correctness is observed when the class label of the training instance selected to explain the target instance has the same label as the target instance being explained. With the definition of a correct explanation, we can define accuracy as the ratio of correct explanations to the total explanations pro- duced by the proposed approach. Our hypothesis is that the proposed method using both longitudinal distances will select instances to explain the classifica- tions produced by the deep NN architecture with average accuracy above 95%. Methodology. With the classifier trained, we randomly selected 1,000 from the 100,000 testing instances to measure the accuracy of the proposed approach. This evaluation, as the metric describes, only seeks the selection of the class of explanations, and thus it does not use the negative explainer set. For each instance, we followed these steps: 1. Compute the positive explainer set using dL and dSL . 2. Use the label of each instance to create subsets of instances for each label. 3. Determine the largest subset and use its label as explanation. 4. Assess whether label is correct. 5. Compute average accuracy. Results. Of the 1,000 testing instances, there were 978 and 980 correct expla- nations for the approach when using dL and dSL , respectively. Both average accuracy levels are above 95%, which is consistent with our hypothesis. The classifier is correct for 968 instances out of 1,000 (96.8%), being wrong in 32 in- stances. Both distances led to the selection of the wrong explanation only when the classifier’s predictions were wrong. However, the distances do not indicate the wrong explanations whenever the classifier is wrong. Interestingly, both dis- tances dL and dSL were correct, respectively, in 10 and 12 instances for which the classifier was incorrect. Discussion. It is curious that the distances led to the selection of the correct explanation in instances when the classifier was wrong because the distances are based on the results of the classifier. We examined whether there was anything else unusual about those instances. We found that when dL was able to select cor- rect explanations while the classifier was wrong, the cardinality of the explainer sets was much smaller than in other instances. Considering when the classifier was wrong, the average size of the explainer set was 274 instances when the cor- rect explanation was selected in comparison to 23,239 when the explanation was not correct. We then examined the predictions of the classifier throughout the 15 epochs and computed two values, namely, the number of distinct predictions and the number of times the prediction changed. Table 1 shows the values for all instances where dL selected the correct explanation when the classifier was 10 Weber et al. Table 1. Analysis of the predictions when dL selected the correct explanation when the classifier was wrong. The correlation between the explainer set (1st column) and the respective number of distinct predictions (4th column) and changes (5th column) is 0.60 and 0.74 Explainer Majority 15 Predictions Distinct Changes set size label set predic- size tions 841 787 0, 0, 0, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4 2 1 527 480 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4 2 2 527 480 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4 2 2 321 321 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4, 4, 4 2 2 321 321 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 0, 4, 4, 4 2 2 120 120 5, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1 2 2 32 32 6, 6, 6, 6, 6, 6, 0, 4, 0, 4, 0, 0, 4, 4, 4 3 6 29 29 6, 6, 6, 6, 6, 6, 0, 4, 4, 4, 4, 0, 4, 4, 4 3 4 15 15 6, 6, 6, 6, 6, 6, 0, 4, 4, 4, 0, 0, 4, 4, 4 3 4 5 5 3, 3, 3, 3, 3, 3, 3, 3, 0, 3, 3, 0, 0, 0, 0 2 3 wrong. The correlation between the explainer set and the respective number of distinct predictions and changes is 0.60 and 0.74. We also computed the correla- tion between the explainer set and the respective number of distinct predictions and changes for when dL was not correct, and the results are 0.82 and 0.72. These correlations suggest that the more changes in labels the classifier makes as it is learning, the more demanding the distances become causing the explainer sets to be more efficient. There is a lot to investigate further on this finding. 5 Conclusions and Future Work In this paper, we introduced longitudinal distances to be computed between an unseen target instance classified by an NN and all training instances used to train it. The training instances at the shortest distance from the target instance constitute its explainer set. The training instances that are members of the explainer set are hypothesized to have contributed to classify the target instance. As such, they can be potentially used to explain the target instance. Longitudinal instances are inspired by the similarity heuristics and the prin- ciple that similar problems have similar solutions. Although not demonstrated yet, instance attribution methods have the potential to bring to example-based XAI, particularly when implemented with CBR, the facet of attribution, cur- rently missing in those approaches. In this paper, the positive explainer set was used to select the best candidate for explanation. We did not use the negative explainer set. It is obvious that, if we are interested in accountability, as this paper described in its motivation, then we first need to demonstrate that the explainer set produces the explained decision; this is when we will use the negative explainer set. Ultimately, we also need to determine how to modify the set to change its decisions. Longitudinal Distance: Towards Accountable Instance Attribution 11 New XAI and IML approaches should demonstrate how they address any criticisms. Demonstrating how they perform in presence of noise, the proportion of outliers in the explainer set, and the proportion of overlap across explainer sets for multiple instances are future work for longitudinal distances. Acknowledgments. Support for the preparation of this paper to Rosina O We- ber and Prateek Goel was provided by NCATS, through the Biomedical Data Translator program (NIH award 3OT2TR003448-01S1). Gideon Simpson is sup- ported by NSF Grant no.DMS-1818716. References 1. Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: 32nd NeurIPS. pp. 9525–9536 (2018) 2. Adler, P., Falk, C., Friedler, S.A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., Venkatasubramanian, S.: Auditing black-box models for indirect influence. Knowl- edge and Information Systems 54(1), 95–122 (2018) 3. Alvarez-Melis, D., Jaakkola, T.S.: Towards robust interpretability with self- explaining neural networks. arXiv preprint arXiv:1806.07538 (2018) 4. Amiri, S.S., Weber, R.O., Goel, P., Brooks, O., Gandley, A., Kitchell, B., Zehm, A.: Data representing ground-truth explanations to evaluate xai methods. arXiv preprint arXiv:2011.09892 (2020) 5. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one 10(7), e0130140 (2015) 6. Baldoni, M., Baroglio, C., May, K.M., Micalizio, R., Tedeschi, S.: Computational accountability. In: Deep Understanding and Reasoning: A Challenge for Next- generation Intelligent Agents, URANIA 2016. vol. 1802, pp. 56–62. CEUR Work- shop Proceedings (2016) 7. Barr, B., Xu, K., Silva, C., Bertini, E., Reilly, R., Bruss, C.B., Wittenbach, J.D.: Towards ground truth explainability on tabular data. arXiv preprint arXiv:2007.10532 (2020) 8. Barshan, E., Brunet, M.E., Dziugaite, G.K.: Relatif: Identifying explanatory train- ing samples via relative influence. In: International Conference on Artificial Intel- ligence and Statistics. pp. 1899–1909. PMLR (2020) 9. Bhatt, U., Ravikumar, P., et al.: Building human-machine trust via interpretability. In: AAAI Conference on Artificial Intelligence. vol. 33, pp. 9919–9920 (2019) 10. Chen, Y., Li, B., Yu, H., Wu, P., Miao, C.: Hydra: Hypergradient data relevance analysis for interpreting deep neural networks. arXiv:2102.02515 (2021) 11. Garfinkel, S., Matthews, J., Shapiro, S.S., Smith, J.M.: Toward algorithmic trans- parency and accountability. Communications of the ACM 60(9) (2017) 12. Guidotti, R.: Evaluating local explanation methods on ground truth. Artificial Intelligence 291, 103428 (2021) 13. Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretability methods in deep neural networks. In: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Pro- cessing Systems. vol. 32. Curran Associates, Inc. (2019), https://proceedings. neurips.cc/paper/2019/file/fe4b8556000d0f0cae99daa5c5c5a410-Paper.pdf 12 Weber et al. 14. Kenny, E.M., Keane, M.T.: Twin-systems to explain artificial neural networks us- ing case-based reasoning: Comparative tests of feature-weighting methods in ann- cbr twins for xai. In: Twenty-Eighth International Joint Conferences on Artificial Intelligence (IJCAI), Macao, 10-16 August 2019. pp. 2708–2715 (2019) 15. Khanna, R., Kim, B., Ghosh, J., Koyejo, S.: Interpreting black box predictions using fisher kernels. In: The 22nd International Conference on Artificial Intelligence and Statistics. pp. 3382–3390. PMLR (2019) 16. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al.: Inter- pretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In: International conference on machine learning. pp. 2668–2677. PMLR (2018) 17. Koh, P.W., Liang, P.: Understanding black-box predictions via influence functions. In: International Conference on Machine Learning. pp. 1885–1894. PMLR (2017) 18. Koul, A., Greydanus, S., Fern, A.: Learning finite state representations of recurrent policy networks. arXiv preprint arXiv:1811.12530 (2018) 19. Kroll, J.A., Huey, J., Barocas, S., Felten, E.W., Reidenberg, J.R., Robinson, D.G., Yu, H.: Accountable algorithms. UNIVERSITY of PENNSYLVANIA LAW RE- VIEW pp. 633–705 (2017) 20. Lundberg, S., Lee, S.I.: A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874 (2017) 21. Mahajan, D., Tan, C., Sharma, A.: Preserving causal constraints in counterfac- tual explanations for machine learning classifiers. arXiv preprint arXiv:1912.03277 (2019) 22. Man, X., Chan, E.P.: The best way to select features? comparing mda, lime, and shap. The Journal of Financial Data Science 3(1), 127–139 (2021) 23. McCloskey, M., Cohen, N.: Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation - Ad- vances in Research and Theory 24(C), 109–165 (Jan 1989) 24. Mercier, D., Siddiqui, S.A., Dengel, A., Ahmed, S.: Interpreting deep models through the lens of data. In: 2020 International Joint Conference on Neural Net- works (IJCNN). pp. 1–8. IEEE (2020) 25. Mittelstadt, B., Russell, C., Wachter, S.: Explaining explanations in ai. In: Proceed- ings of the conference on fairness, accountability, and transparency. pp. 279–288 (2019) 26. Mittelstadt, B.D., Allo, P., Taddeo, M., Wachter, S., Floridi, L.: The ethics of al- gorithms: Mapping the debate. Big Data & Society 3(2), 2053951716679679 (2016) 27. NOW, A.: Annual report–new york university (2018) 28. Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., Mordv- intsev, A.: The building blocks of interpretability. Distill 3(3), e10 (2018) 29. Ribeiro, M.T., Singh, S., Guestrin, C.: ” why should i trust you?” explaining the predictions of any classifier. In: 22nd ACM SIGKDD. pp. 1135–1144 (2016) 30. Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: International Conference on Machine Learn- ing. pp. 3145–3153. PMLR (2017) 31. Sliwinski, J., Strobel, M., Zick, Y.: Axiomatic characterization of data-driven in- fluence measures for classification. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 718–725 (2019) 32. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning. pp. 3319–3328. PMLR (2017) 33. Yang, M., Kim, B.: Benchmarking attribution methods with relative feature im- portance. arXiv preprint arXiv:1907.09701 (2019) Longitudinal Distance: Towards Accountable Instance Attribution 13 34. Yeh, C.K., Kim, J.S., Yen, I.E., Ravikumar, P.: Representer point selection for explaining deep neural networks. arXiv preprint arXiv:1811.09720 (2018)