Explaining the behavior of AI-based tools, whose results may be unexpected even to experts, has become a major request from society and a major concern of AI practitioners and theoreticians. In this position paper we raise two points: (1) irrelevance is more amenable to a logical formalization than relevance; (2) since efective explanations must take into account both the context and the receiver of the explanations (called the explainee) so it should be also for the definition of irrelevance. We propose a general, logical framework characterizing context-aware and receiver-aware irrelevance, and provide a case study on an existing tool, based on Semantic Web, that prunes irrelevant parts of an explanation.
follows from (ii)+(iii). Observe that a complete explanation may involve all three statements, leading to what would be perceived as a redundant explanation—that is, an explanation full of details that may be considered truthful but irrelevant, since they are already known by the explainee. Observe that a simply redundant explanation contains both details that are known to the explainee, and details that are not known, while in a completely irrelevant explanation all details were already known.
In this position paper, we want to establish some objective criteria for defining knowledge
surely irrelevant, by elaborating on (and generalizing) ideas that we presented in a more restricted
context [
However a logic formalization of these widespread ideas is still missing, leading to direct implementations that, although justified by the above intuitions, are tailored to the specific application.
Stemming from the criteria above, we attempt a logical formalization of irrelevant explanation, in Section 2. In Section 3, we show the benefits of adopting such a formalization when explaining the similarity of groups of RDF resources. A final section concludes the paper.
To formalize irrelevance, we lay down several hypotheses about how to logically represent the setting in which an explanation arises. We are aware that some hypotheses may be questionable, but we consider them all necessary for a logical formalization of a relevant/irrelevant explanation. The elements we give a name to are: a deterministic system , an explainee , an input to . On input , outputs a result , which asks to explain. We suppose that: 1. both the input and the output (or, their descriptions) can be expressed as formulas , 2. the characteristics of can be represented by a logical theory such that ∪ {} |= ; observe that if were not deterministic, a more complex statement involving probabilities would be necessary 3. the knowledge possessed by —i.e., information consciously knows—can be represented as another logical theory 4. an explanation is formed by sentences (chunks), each sentence stating the truth of a logic formula , = 1, . . . , , so that the entire explanation can be represented as a .
conjunction = 1 ∧ · · · ∧ 5. an explanation—although possibly irrelevant—is always truthful with respect to the particular behavior of system it explains, that is, ∪ {} |= .
Assumption 1 seems rather straightforward: it is always possible to represent the inputs and the results of a system in some formal language.
Assumption 2 may seem too strong for numerical, nonlinear AI systems; however, it does not pretend to completely describe the inner functioning of ; only the fact that inputs and outputs can be logically related by .
Assumption 3 takes into account both general knowledge and specific knowledge that can be attributed to the explainee. For example, for a physician an ontology of medical knowledge—e.g., “Antibiotics cure bacterial infections”—can be added to general knowledge about the world.
Assumption 4 is necessary when an explanation is a complex argument, expressed as several sentences. The correspondence between sentences and formulas will be necessary for Point 2 below.
Finally, Assumption 5 is just a logical way to express a natural requirement: explanations should always be truthful with respect to how works on input . For example, if the counterfactual explanation given by for denying a loan was “If the monthly income raises by 25%, the loan could be granted”, one expects that just raising the monthly income (changing nothing else) the loan would really be granted. In formulas, if to explain the result on input , a counterfactual explanation “′ > ¬” is given to the explainee, then we expect the counterfactual to be true in —in formulas, ∪ {} |= (′ > ¬)—for some semantics of counterfactuals, which we do not want to delve into now.
Now, we consider the cases in which is irrelevant: Definition 1.
1. (Irrelevance for the general context) is irrelevant for Result if there exists another input ′, for which yields a diferent result ′, such that would explain also the result ′ 2. (Irrelevance for the specific explainee) is irrelevant for explainee if for all ∈ {1, . . . , } it holds |= , that is, no conjunct of was unknown to the explainee
Point 1 considers irrelevant those explanations that are too general—that is, not cogent for
the result to be explained. Consider for example a classification system, that outputs =Dolphin
when given as input the photo of some animal in the sea. The explanation =“Because it
swims.” is irrelevant in this context, and could raise the request of a contrastive explanation [
Point 2 takes into account the fact that explanations may be more than just one phrase, for
instance when a chain of reasoning is shown, that leads from the input to the result. Point 2
requires that at least one conjunct forming the explanation must be unknown to the user.
Observe that we do not exclude parts of the explanation that are already known, since if they
form an entire argument, excluding them would make the argument scattered. For instance, if
the explanation for denying the loan was =“Because you are not resident in this country, and
the risk-assessment threshold for non-residents is higher than the normal one”, then = 1 ∧ 2,
where
1
2
= “you are not resident in this country”
= “the risk-assessment threshold for non-residents is higher than the normal
Now if is not aware of 2 (that can be checked as ̸|= 2), then as a whole may be
considered relevant thanks of the presence of 2. The presence of 1 instead, although a little
redundant, may be considered part of the entire argument, so can be considered relevant
even if 1 is present. Note that in this brief discussion paper, we not tackle the question of
redundancy, which we consider diferent from (ir)relevance.
2.1. More Examples
Let us think about two popular examples in explanation research: the arthropods classification
by Miller [
Suppose that, given the classification of an image , a user asks the question "Why is image J labelled as a spider instead of a beetle?". In this case, it is irrelevant for the classification context an explanation like "Because it represents an arthropod"; this explanation chunk, although true, is obviously true for all images and thus irrelevant for understanding the classification reasons.
Imagine now that the explainee is a biologist, asking to the contrastive explanation agent "But an octopus can have eight legs too. Why did you not classify image J as an octopus?". An explanation like = 1 ∧ 2 ="Because my function is only to classify arthropods, and an octopus is not an arthropod" is relevant to a biologist only in its first part 1. Instead, the information 2 about the octopus category is well known by any biologist.
In the second example scenario, we focus on explanations of reasons for loan rejection. Any bank customer asking for a loan is not interested in rejection explanations like “The risk associated to your loan request is too high" (or, in a counterfactual fashion, “If the risk associated to your loan request was lower, then the loan would have been accepted” ). This rejection condition is true for all rejected loan requests (the input-output pairs ′–′ of Point 1), and then irrelevant for the context. Customers would be much more interested in knowing their own specific reasons for risk evaluation: age, health conditions, income level, and so on.
Analogously, explaining to the customer “You did not get the loan because you are over 40 years old" is irrelevant, because tells something he/she already knows (his/her age). A relevant explanation might have been = 1 ∧ 2 ∧ 3, where 1 2 3 = “You are over 40 years old” = “The risk evaluated for customers over 40 years old is high” = “Loans are denied to high-risk factor applicants” presuming that at least 2 was unknown to the explainee.
The ability of explaining why a recommender system suggested a user a particular item (or set of items) is now recognized as an important feature [8]. In particular, counterfactual explanations of recommended items [9] may suggest the user alternative items that could be recommended, provided that the user’s preferences change accordingly to the antecedent of the counterfactual. In symbols, a counterfactual explanation > can be communicated to the user as “I suggested you item , but if your preferences were changed to , I would suggest you item instead of ”.
In this application area, irrelevant explanations may hamper the user’s trust in the recommendation system, obtaining an opposite efect explanations were devised for. Imagine a smartphone recommender system, a user entering preferences , and being recommended a smartphone . An example of counterfactual explanation being irrelevant for the context (Point 1 above) would be “If you had no restrictions on budget, I would have suggested you an Apple iPhone 14 Pro 256GB.” While being true, such an explanation would fit any other preference setting ′ and subsequent recommendation ′, being the iPhone 14 Pro one of the possible obvious choices in case of unlimited budget. We note that researchers are implicitly aware of this kind of irrelevance, and usually, to avoid such explanations, explaining modules try to perturb as little as possible the initial input (e.g., raising the budget limits by a small amount only) in order to get a relevant counterfactual explanation, like for instance, “If you raised your budget by 10$, I would have recommended you this other smartphone.”
The formalization above is not just a theoretical speculation on irrelevance. We applied the above criteria in a tool proposed by Colucci et al. [10, 11] to provide a human-readable explanation of commonalities shared by groups of RDF [12] resources, somehow aggregated (e.g., by a clustering algorithm2).
The verbalization is based on the logic-based computation of the Least Common Subsumer(LCS) in RDF [14].
We apply the LCS-based verbalization tool to clustering results in two application scenarios: public procurement and drug comparison.
The first scenario is modelled in TheyBuyForYou (TBFY) dataset, a knowledge graph [ 15] that includes an ontology for procurement data, based on the Open Contracting Data Standard (OCDS)[16].
In particular, all contracting processes released on January, 30ℎ 2019 have been clustered with the K-means [17] algorithm3 and the smallest cluster has been explained in terms of commonalities (on the basis of the LCS = 1 of the set of items it contains).
The resulting explanation is given in Figure 1.
The reader may notice that the last explanation line—call it 1—is objectively irrelevant in this context (so, for any user): any contracting process in the original dataset has been 2Note that the tool does not explain a whole partition into clusters of a set of resources, as in [13]; it only describes the commonalities of two or more resources already clusterized. 3The implementation at https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html has been used released on January 30, 2019, causing this explanation chunk to be obvious in the addressed clustering scenario. In terms of the previous formalization, we can automatically exclude 1 in the following way: by computing the LCS of a wider set of resources ′—that is, adding to the cluster another random resource—we obtain an LCS ′ = 2 having, among others, the same release date already found in 1. Since the explanation 1 =“Released on January 30, 2019” is entailed by 2, we can conclude that 1 is irrelevant, and exclude it from the relevant explanations for 1.
In the second application scenario, the search for similarities between drugs modelled in RDF is addressed. In particular, the National Drug File - Reference Terminology hosted by Bioportal4 is used as a dataset.
Figure 2 shows an explanation for the similarity of two antibiotics: "cefepime" (http://purl.bioontology.org/ontology/NDFRT/N0000022054) and "ceftazidime" ( http://purl.bioontology.org/ontology/NDFRT/N0000145931) produced by the verbalization tool by Colucci et al. [10, 11] ).
If the explainee is a physician, some explanation chunks (blue arrows and lines in figure) are intuitively irrelevant: it is common knowledge (at least) for physicians that (i) any antibiotic may treat bacterial infections (and thus, infections) and that (ii) fever is a body temperature change.
The formalization we propose aims at pruning explanations of chunks which are irrelevant to the explainees if their knowledge is logically represented. In fact, by taking as (among others) the RDFS statements expressing (i)–(ii) as a medical ontology in Bioportal, it is possible to automatically check that entails5 both (i) and (ii), concluding that they are irrelevant for a physician, and exclude them from a concise explanation.
5In this case, entailment reduces to simple containment, but more elaborated examples involving blank nodes need entailment.
In this paper, we contribute to the discussion about explanation of AI-based tools, by formalizing a logical framework for identifying irrelevant chunks in a complex explanation.
Our proposal stems from the assumption that irrelevance is more amenable to a logical formalization than relevance, which is intrinsically subjective. In fact, we provide two definitions that may be implemented for pruning irrelevant portions of explanation: i) irrelevance for the general context: refers to knowledge true also for situations diferent from the one being explained; ii) irrelevance for the specific explainee : refers to knowledge already known to the explainee.
We demonstrate the practical applicability of these definitions, by implementing them in a tool that provides human-readable explanations of commonalities shared by group of RDF resources. Thanks to our formal definition, the use case shows how to prune complex similarity explanations by deleting irrelevant chunks.
We acknowledge partial support of the project "Contratto di Programma "Digital Enterprise", POR Puglia Grandi Imprese, Code NIL6S28. [7] R. Guidotti, Counterfactual explanations and how to find them: literature review and benchmarking, Data Mining and Knowledge Discovery Special Issue on Explainable and Interpretable Machine Learning and Data Mining (2022). doi:https://doi.org/10. 1007/s10618-022-00831-6. [8] Y. Zhang, X. Chen, Explainable recommendation: A survey and new perspectives, Found.
Trends Inf. Retr. 14 (2020) 1–101. URL: https://doi.org/10.1561/1500000066. doi:10.1561/ 1500000066. [9] J. Tan, S. Xu, Y. Ge, Y. Li, X. Chen, Y. Zhang, Counterfactual explainable recommendation, in: G. Demartini, G. Zuccon, J. S. Culpepper, Z. Huang, H. Tong (Eds.), CIKM ’21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021, ACM, 2021, pp. 1784–1793. URL: https://doi.org/10.1145/3459637.3482420. doi:10.1145/3459637.3482420. [10] S. Colucci, F. M. Donini, N. Iurilli, E. D. Sciascio, A business intelligence tool for explaining similarity, in: E. Babkin, J. Barjis, P. Malyzhenkov, V. Merunka (Eds.), Model-Driven Organizational and Business Agility - Second International Workshop, MOBA 2022, Leuven, Belgium, June 6-7, 2022, Revised Selected Papers, volume 457 of Lecture Notes in Business Information Processing, Springer, 2022, pp. 50–64. URL: https://doi.org/10.1007/978-3-031-17728-6_5. doi:10.1007/978-3-031-17728-6\_5. [11] S. Colucci, F. M. Donini, E. D. Sciascio, A human-readable explanation for the similarity of RDF resources, in: C. Musto, R. Guidotti, A. Monreale, G. Semeraro (Eds.), Proceedings of the 3rd Italian Workshop on Explainable Artificial Intelligence co-located with 21th International Conference of the Italian Association for Artificial Intelligence(AIxIA 2022), Udine, Italy, November 28 - December 3, 2022, volume 3277 of CEUR Workshop Proceedings, CEUR-WS.org, 2022, pp. 88–103. URL: https://ceur-ws.org/Vol-3277/paper7.pdf. [12] P. Patel-Schneider, D. Arndt, T. Haudebourg, RDF 1.2 semantics, https://www.w3.org/TR/rdf12-semantics/, 2023. [13] M. Moshkovitz, S. Dasgupta, C. Rashtchian, N. Frost, Explainable k-means and k-medians clustering, in: H. D. III, A. Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, PMLR, 2020, pp. 7055–7065. URL: https://proceedings.mlr.press/v119/moshkovitz20a.html. [14] S. Colucci, F. Donini, S. Giannini, E. Di Sciascio, Defining and computing least common subsumers in RDF, Web Semantics: Science, Services and Agents on the World Wide Web 39 (2016) 62 – 80. [15] A. Soylu, O. Corcho, B. Elvesater, C. Badenes-Olmedo, T. Blount, F. Yedro Martinez, M. Kovacic, M. Posinkovic, I. Makgill, C. Taggart, E. Simperl, T. C. Lech, D. Roman, TheyBuyForYou platform and knowledge graph: Expanding horizons in public procurement with open linked data, Semantic Web 13 (2022) 265–291. [16] A. Soylu, B. Elvesaeter, P. Turk, D. Roman, O. Corcho, E. Simperl, G. Konstantinidis, T. C.
Lech, Towards an ontology for public procurement based on the open contracting data standard, Springer-Verlag, Berlin, Heidelberg, 2019, p. 230–237. [17] X. Jin, J. Han, K-Means Clustering, Springer US, Boston, MA, 2010, pp. 563–564. URL: https: //doi.org/10.1007/978-0-387-30164-8_425. doi:10.1007/978-0-387-30164-8_425.