The paper describes a non-parametric approach to the Similarity Based XAI, based on Shepardʼs universal law of generalization, which states that “the probability of a response to one stimulus being generalized to another is a function of a “distance” between the two stimuli in a psychological space”. In the theory of pattern recognition, the analogue of this law is the compactness postulate of Averyanov and Braverman, which states that objects of one class in a feature space, as a rule, are located closer to each other than to objects of other classes. It is common to consider objects to be similar if they are close in a feature space. Meanwhile, features of objects in real life often are random values. Such objects are described not by a feature vector, but by a random sample or several samples of features and the compactness hypothesis should be replaced by a statistical homogeneity hypothesis. Objects are considered homogeneous if their features obey the same distributions, and their similarity measures form vectors in a similarity space. The chapter describes a nonparametric measure of homogeneity and provides an illustration of their use in medical applications, in particular for the diagnosis of breast cancer in the framework of the Similarity-Based XAI. We formulate new statistical postulates of machine learning and propose to consider a machine learning algorithm explainable and interpretable if it satisfies these postulates. The proposed concepts are illustrated by an application of XAI principles for the breast cancer diagnosis.
The concept of explainable artificial intelligence has become the subject of intensive research,
since the mechanical application of artificial intelligence does not meet the natural requirements that
apply to human-machine systems. In automatic systems, one subsystem unconditionally accepts the
results of the work of another subsystem, since this feature is inherent in their designs. Medical
applications are of a completely different nature, since they can be schematically described as a
patient–physician–artificial intelligence–doctor–patient interaction. As you can see, the role of a
person in this scheme is many times greater than the role of artificial intelligence. Instead of a
mechanical interaction that implies blind trust and automatism, a person dictates to artificial
intelligence the properties that it should possess: trust, causality, portability, and informativeness [
Particular attention is drawn to the cognitive aspects of the explainability of deep learning and
machine learning algorithms. First, we note that in many works (see, for example, [
In [
The answer to the first question is reduced to explaining what kind of data is used for training and why. In particular, it requires the formulation of a working hypothesis, which reflects the choice of data. For example, in medical applications in the field of oncology, the input of algorithms is information collected from patients with various types of cancer, as well as from healthy people. It is logical to assume that this information has a different value for diagnosis. Who and how will select the necessary information to ensure the greatest accuracy of the diagnosis? The classical approach implies the participation in this process of an expert in the subject area, who has a priori knowledge about the significance of certain data. This expert can, for example, use prior knowledge that in the body of a person with cancer, biochemical reactions occur that affect the distribution of chromatin in the nuclei of buccal epithelial cells. In this case, the expert will offer the algorithm photographs of cell nuclei taken from sick and healthy people. At the same time, a second, no less important question arises: how objective are these data? For example, if a photograph contains some kind of systematic artifact (label), then the classification apriori will be compromised. Therefore, a data validation mechanism is needed that guarantees their unbiased and random selection from the entire data set. By the way, this raises, for example, the question of the criteria for the randomness of the available data.
The second question follows from the first. If the algorithm, acting on the basis of a black box, received input data, it can independently select features that have neither biological nor physical meaning, nor any other meaning except statistical. In other words, the results to which it will lead will be correlated rather than causal. This does not affect the recognition accuracy in any way, because the algorithm strives to classify objects as accurately as possible, but the recognition process itself in this case can only be described from a statistical point of view. This makes it difficult to acquire new knowledge. Yes, we have correctly divided the two sets, but what does it give from a meaningful point of view. What features did not the human eye see in these photographs, and what is their biological or physical meaning? The modern theory of machine learning does not provide mea ningful answers to these questions. As a rule, the features are some function or a combination of several functions that were chosen by the algorithm without any knowledge of the content of the photo.
The third question concerns the design of the model. In the context of artificial neural networks, it
comes down to architecture analysis and layer manipulation. A variety of architectures and methods
of working with them leave the impression of blind juggling. If this or that architecture has led to a
successful result, it seems superfluous to explain why. At the same time, such a network can be like a
house of cards that will crumble as soon as new data enters it. The ability of such algorithms to
generalize is very problematic, and their stability is questionable. An interesting classification of
classification methods in terms of their explainability is given in [
It seems to us that such a classification of methods in the field of artificial intelligence applications in medicine is not entirely accurate. If you think about it, it becomes obvious that this classification evaluates explainability from the point of view of the researcher, that is, it makes sense only for the developer of the algorithm. At the same time, in the patient–doctor–AI–doctor–patient scheme, there are two more people who do not belong to the category of developers, but users of algorithms. A doctor wants to understand and trust an algorithm, and a patient wants to trust and understand the doctor. Therefore, explainability must be considered from their point of view too. There are both similarities and differences in these approaches.
From the point of view of data explainability, the doctor, like the developer, wants and is obliged to understand the meaning and quality of the input data. Here, their interests coincide. In addition to the increased quality of data preparation by a competent doctor who will correctly set up the mechanics with an understanding of the specifics of input data preparation (for example, set the correct illumination level, choose the right reagent, and set the required microscope resolution), a priori understanding of the input data will allow you to get feedback in time, if there is a drift of concept, due to which the algorithm may be compromised.
The explainability of the results also affects the level of confidence of the doctor and the patient in the results obtained. A doctor verifies the conclusions made by the algorithm, one way or another, using additional methods (for example, by directing the patient to be examined by other methods). If the algorithm constantly demonstrates high accuracy, then the doctor and the patient can make the right clinical decisions, rightly believing that this technique has been tested and is reliable.
At the same time, neither a doctor nor a patient should be interested in how many layers are used in the neural network architecture. This information is beyond his competence and has nothing to do with explainability from the point of view of either the doctor or the patient. Ther efore, the explainability of the algorithm for the doctor and the patient is of no value.
Letʼs move on to interpretability, the second aspect of XAI [
Closely related to the concept of interpretability is the concept of transparency, which boils down
to three properties: imitability, decomposability, and algorithmic learning [
A detailed and extensive analysis carried out in the works [
Note also remarkable papers of Rudin [
However, without being able to open the black box and interfere with its operation, we can try to explore how well it meets the criteria of explainability and interpretability. This is what our work is devoted to, in which we show, what criteria can be used to evaluate the interpretability of a black box, if one had to apply it, and how to build a transparent Rudin-interpretable model for diagnosing breast cancer that also meets these criteria. Thus, we propose a compromise between these two alternatives.
As we will show below, the construction of a formal XAI theory based on a rigorous mathematical approach leads to the conclusion that the explanation of the work and conclusions of artificial intelligence is impossible without the application of machine learning postulates. The goal of this chapter is to propose a new XAI formalism based on alternative statistical postulates of machine learning.
Confidence, causality, portability, and informativeness are essential properties of any machine learning method. To prove this thesis, consider the general scheme of any machine learning method.
Let X be a set of objects, Y be a set of class lables, and f: X Y be a function attaining values at elements of a training sample drawn from X. The goal of the algorithm learning it to extent f: X Y on all the set X to construct a solving function g: X Y mapping all objects to their lables and mimimizing a risk function (for example, an error).
Since the very definition of a machine learning problem includes minimizing the error and specifying the function on the entire set X, accuracy and portability (ability to generalize) are guaranteed by default. At the same time, the concepts of causality and informativeness are of an informal nature, which is difficult to describe with the help of a mathematical apparatus. However, difficult does not mean impossible.
One of the most meaningful mathematical formalisms of interpretable artificial intelligence was
proposed in [
It should be noted that this formalism has a significant drawback, explained by the Bayesian nature of the entire method. It requires knowledge of the a priori probability of recognizable classes, which must be given by experts. The authors also propose to calculate the Sloman similarity function as a cosine measure between the vectors generated by the silency-map and the experts participating in the experiment. It is obvious that all this makes the methodology local and subjective. Its correctness depends on the competence of specific experts.
We propose to use a more rigorous and objective approach, based not on the subjective assessments of experts, but on objectively determined values of a measure of the objectʼs homogeneity with other objects and its statistical depth. In short, we propose to consider the results of the work of artificial intelligence as interpretable from the point of view of a doctor if they satisfy the postulate of statistical compactness. On the other hand, we suggest that these results be considered interpretable from the patientʼs point of view if they allow one to determine the patientʼs individual risk. The accuracy of any machine learning algorithm in medical diagnostics is treated according to Fisher. Assume that the sensitivity of an algorithm is 95%. This means that out of 100 randomly selected patients, with repeated repetition of this choice, the diagnosis is correctly determined on average in 95 patients and 5 patients receive a false diagnosis. If the algorithm has a specificity of 95%, this means that out of 100 randomly selected healthy people, if this choice is repeated many times, 5% will receive the diagnosis “sick”. These ratings do not answer the natural questions that every patient has: “What is the probability that I am really sick? Which group do I fall into: 95 correct diagnoses or 5 false ones?” This question can be answered using two concepts: statistical homogeneity and statistical depth. On the one hand, by comparing the patientʼs data with the patientʼs etalon, one can assess the degree of similarity between them, on the other hand, by evaluating the statistical depth of the data one can assess the typicality for a given diagnosis.
Machine learning is predicated on two premises, both of which we have previously mentioned
implicitly above: The first postulate is that an object should be represented as a feature vector in a
feature vector space, and the second is the Averyanov and Braverman postulate on compactness [
Despite the obvious success of the methods listed above, it cannot be denied that the postulates of vector space and compactness in the form formulated above are not applicable to all problems. In many medical and biological studies, the patient does not correspond to a vector of features, that is, an ordered set of numbers characterizing his various properties, but a random sample, that is, an unordered set of random results of measurements of certain parameters (for example, nuclear area, optical density of nuclei and the like). Such samples arise, for example, when analyzing probes taken from a patient. These probes typically contain several tens of cells, so the patient does not correspond to a single point in the vector space, but rather to a cloud of points that are not in any way arranged in any particular order. It is true that this process can be made simpler by computing the average sample values and using the accepted postulates, but it is also clear that in this scenario, a significant portion of the information regarding the distribution of the measured parameters is lost. Imagine you were given no further information save the fact that the median of a conventional Gaussian distribution, even an empirical one, is zero.
Alternative statistical hypotheses that we put forth include: 1) that things can be represented by sample parameter values, 2) that parameters of objects within the same class have similar distributions, and that parameters of objects within different classes have different distributions. With the use of this strategy, we can reduce the challenge of determining how similar two things are to verifying whether two or more samples are homogeneous. The approach we suggest using to determine how similar two things are is shown below. It has statistical universality, which means that it performs equally well for samples with different means and the same standard deviations as it does for samples with the same means but different standard deviations, unlike traditional methods like Kolmogorov-Smirnov statistics and Mann-Whitney-Wilcoxon statistics.
Take into account a sample of size n made up of continuous random variables coming from a
swappable distribution. The likelihood that a random value from the same distribution is larger than
ith and less than j-th order statistics of the sample equals (j-i)/(n+1), according to the Hill's assumption
[
For a detailed review of the various concepts of statistical depth, see [
Consider some distribution F in Rn . A depth function is a function DF (x) that orders points x
from distribution F monotonically decreasing from center to outward. A depth of x F is a value
attained by DF (x) at x [
When we have no information about a distribution F but have a sample containing n points from F , we shall use notation Dn ( x) . Let us consider some examples of a depth function.
Tukey depth [
Convex hulls peeling [
Oja depth [
The simplex depth [
Zonoid depth [
n n 1 D (x1,..., xn ) i xi : i 1, 0 i ,i : i .
i1 i1 n
Mahalanobis depth [
Elliptical statistical depth [
. Thus, the elliptical statistical depth is a monotonous function that attains a maximum at the n 1 deepest point and decrease from the center to outward.
Depth-ordered regions [
Consider is a set of random points X x1,..., xn , xi d . Let us divide the description of the algorithm for construction of Petuninʼs ellipsoids into two cases: two-dimensional d = 2 and multidimensional d > 2.
Petuninʼs ellipses. Find a convex hull of X x1, y1 ,..., xn , yn and a diameter of this convex hull with ends xk , yk and xl , yl . Connect these point by a segment L . Find points xr , yr and xq , yq that are most distant from L . Find segments L1 and L2 passing through xr , yr and xq , yq parallel to L . Find segments L3 and L4 passing through xk , yk and xl , yl orthogonal to L and. Segments L1 , L2 , L3 and L4 are sides of a rectangle . Let us denote by a a short side and by b a long side.
Translate, rotate and shrink with a coefficient to obtain a square with a center a b x0 , y0 . The random points X x1, y1 ,..., xn , yn are mapped to points x1, y1 , x2 , y2 , ..., xn , yn . Find distances r1, r2 ,..., rn between x0 , y0 and x1, y1 , x2 , y2 , ..., xn , yn . Find
E containing points X x1, y1 ,..., xn , yn .
Petuninʼs ellipsoids. Find a convex hull of X x1,..., xn , xi d . Find a diameter of the convex hull with ends xk , yk and xl , yl . Rotate and translate the diameter aligning it along to Ox1 . Project x1, y1 , x2 , y2 , ..., xn , yn to the orthogonal complement of Ox1 . Find a convex hull of projections, rotate and translate it up to a two-dimensional rectangle . Construct a minimum volume axishull in problem.
aligned parallelogram in d-dimensional space containing the projections of input points. Shrink this parallelogram to hypercube. Find its center x0 and distances r1, r2 ,..., rn from x0 to x1,..., xn . Find R max r1, r2,..., rn . Consider a hypersphere with the center x0 and radius R . Perform inverse transformations. As a result, we obtain the Petuninʼs ellipsoid covering X x1,..., xn .
The Petuninʼs ellipsoids have remarkable properties. They uniquely arrange random multivariate points according their statistical depth due to the fact that the surface of Petuninʼs ellipsoid contains only one point from the initial set. Moreover, the probability that a Petuninʼs ellipsoid covers random points obeying the same distribution is equal to n 1. Therefore, we can find most and least probable n 1 points of a sample.
The most difficult problem of the construction of the Petuninʼs ellipses and ellipsoid in a space of
high dimension is the finding of two most distant points (ends of the convex hull diameter). at the
plane this is not a difficult problem because the computational complexity of the optimal algorithms
in this case is O(n ln n). When в d > 2, the computational complexity of the construction of a convex
d is O nn2 . However, using effective and fast procedures [
We provide empirical results of numerical experiments with samples of various sizes from Gaussian distribution with various parameters for estimation the sensitivity and specificity of the tests. We generated 100 pairs of samples that each contained 10, 20, 30, 40, and 100 random numbers with the same mean and different standard deviations, as well as different means and the same standard deviation. Then, we used both tests to empirically evaluate their specificity and sensitivity. The frequency of the occurrence when the p-value for various distributions is less than 0.05 is the Wilcoxon signed-rank test's sensitivity. The frequency of the occurrence where the upper confidence bound for the p-statistics is greater than 0.95 for identical distributions is the Klyushin-Petunin test's specificity. The frequency of the occurrence when the p-value is less than 0.05 for identical distributions is the specificity of the Wilcoxon signed-rank test. Table 1 presents the outcomes. validates the overall hypothesis of equivalence of the distribution functions whereas the Wilcoxon signed-rank test validates the claim concerning equality of means.
For estimation of specificity of the Klyushin–Petunin test and the Wilcoxon signed-rank test we again used 30 pairs of samples containing 10, 20, 30, 40 and 100 random numbers from the normal distribution N(0,1) (see Table 3). In contrasts to the previous experiments, now the Klyushin–Petunin test demonstrates stable results in every range of the samples sizes and its specificity is comparable with the specificity of the Wilcoxon signed-rank test in every range of the sample sizes.
Thus, we may conclude that the Klyushin–Petunin test has high sensitivity when n 40 and high specificity for all samples sizes and all possible variants of means and standard deviations. The Wilcoxon signed-rank test is valid for testing the hypothesis about distribution location shift but is invalid when different distributions have the same location and different standard deviations (hypothesis of distribution scale). Therefore, the results cited in Table 1 provide the technical validity of proposed test. It works better than the Wilcoxon signed rank test in all the hypothetical situations.
Evaluation of the criteria for explainable artificial intelligence according to existing methods is subjective. We propose an objective approach and a universal criterion of explainability for XAI. Artificial intelligence is explainable if the results of its application satisfy two statistical postulates: 1) objects can be represented by sample values of their parameters; 2) parameters of objects belonging to the same class have the same distributions, and the parameters of objects belonging to different classes have different distributions. In addition, XAI must allow for an estimate of the statistical depth of the results (for example, the individual risk in the healthycare context). As a mathematical tool for this formalism, we propose to use the Klyushin–Petunin test. It is universal due to high sensitivity and specificity when size of a sample is more than 40 (this is rational query that may be easily satisfied). Moreover, it robustly tests both hypotheses on location shift and scale shift of distribution.
The improvement of statistical techniques for evaluating the homogeneity of samples and their statistical depth, as well as the application of the suggested idea to other algorithms, are likely to be linked to the future development of the proposed methodology.