Quest: A Query-driven Explanation Framework for Black-Box Classifiers on Tabular Data

Keywords

• eol>XAI
• post-hoc explanation
• model-agnostic
• classification
• black box

-

Explainability eforts are well established in the ML query-driven post-hoc explanations of individual classiand AI communities by now, with local, model-agnostic ifer decisions on tabular data. approaches currently being the tool of choice in information retrieval [ 2 ] and search [3] as well as many other Query-driven explanation First, we introduce a areas. One major challenge in the field is the lack of more expressive representation of explanations consophisticated approaches for tabular/relational data, as sisting of query predicates. A custom set of common opposed to text or images. Generic approaches, e.g. fea- query predicates is extremely expressive while still comture importance, limit expressiveness and readability. pact. Queries such as capital-gain > capital-loss

LIME [4] still remains the basis of many approaches or rel == ’married’ AND children > 1 also have the for local, model-agnostic explanations. It was adapted to benefit of being easily converted into the WHERE clause tabular data to serve as a baseline for Anchors [1]. Both of a SQL statement, to be executed directly on any reapproaches use a white-box model (surrogate) to approx- lational database. Still more importantly, using queries imate a black-box model locally in order to explain its to explain black-box model behavior within local boundecisions. However, their resulting explanations are lim- daries has the advantage of explaining not only why a ited: LIME (as implemented by the authors) focuses on model produced an outcome but also why not! feature importance. Anchors are derived as simple predi- An explanation produced by Quest can be thought cates to create if-then rules. Figure 1 shows examples for of as boundaries and a decision surface that separates the UCI adult data set [5]. classes within the boundaries. Samples on one side of As an alternative we suggest Quest, a framework for the decision surface within the boundaries form the result set of a query . Alongside (“Why?”) stands (“Why not?”) with its result set covering samples on the opposite side of the decision surface. This consideration of queries as explanations gives the user an intuitive way of thinking about the local neighborhood, supports generalization on the user’s side and keeps the focus on the data instead of the surrogate model. The combination of DESIRES 2021 – 2nd International Conference on Design of Experimental Search & Information REtrieval Systems, September 15–18, 2021, Padua, Italy $ [e-mail] (N. Geisler) € https://www.dm.tu-darmstadt.de/ (N. Geisler) 0000-0002-5245-6718 (N. Geisler)

© 2021 Copyright for this paper by its authors. Use permitted under Creative CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCoEmmUoRns LWiceonsrekAstthribouptionP4r.0oIncteerenadtiionnagl s(CC(CBYE4U.0)R.-WS.org) and ensures an explanation from both sides of the decision surface, something established systems for this task lack. Query representation can be limited to a small number of operators without loss of expressiveness and normalized to facilitate comparison and elimination of duplicates. This is achieved through application-specific, user-defined functions for complex relationships and reducing logical operators while retaining functional completeness. An example would be the disjunctive normal form (DNF) using only AND, OR, and NOT operators.

We impose a complexity budget on / to ensure users are able to understand them well. This can be adapted to the target group. Complexity of queries can be thought of as the number and type of predicates making it easily computable and comparable.

Quest for a point near the decision boundary. Note that Quest boundaries (green, dashed) difer, left being distance-based, right linear. The decision boundary is linear with one attribute depending on the other in both cases. The explanation classes allow for endless extensibility, as quality metrics used for the selection process are defined on the query representation they all share.

Within explanation classes, we suggest imposing a hard complexity restraint (that could be adapted to context upfront) and then optimizing for accuracy.

We compare explanations (candidates produced by Quest and baselines such as LIME/Anchors) regarding accuracy, coverage (area and/or proportion of original samples) and class balance within an explanation.

Acknowledgments

References

of generating queries suitable to a wide range of scenar- (funding number 02L19C150) and managed by the Project

planation classes to minimize drawbacks of individual [1] M. T. Ribeiro , S. Singh , C. Guestrin , Anchors: High-

eliminated immediately while the further process is es- gence , volume 32 , 2018 , pp. 1527 - 1535 .

sentially a hyper-parameter search, covering the decision [2 ] M. Verma , D. Ganguly , Lirme: Locally inter-

parameters. Strategies like pruning or successive halving ings of the 42nd International ACM SIGIR Con-

applicable explanation classes for the given data point . mation Retrieval , SIGIR'19, Association for ComWe propose three exemplary classes that vary in ex- puting Machinery , New York, NY, USA, 2019 ,

pressiveness (i.e., complexity and form of the representa - p. 1281 - 1284 . URL: https://doi.org/10.1145/3331184.

tions they produce) as well as other properties: 3331377 . doi: 10 .1145/3331184.3331377. [3] J. Singh , A. Anand , Exs: Explainable search us-

Decision trees make robust candidates that can be ap- ing local model agnostic interpretability, in: Proplied to numerical and categorical attributes, can ceedings of the Twelfth ACM International Confercapture disjoint areas, and work with any condi- ence on Web Search and Data Mining, WSDM '19, tion type . Association for Computing Machinery , New York,

Adaptations of clustering techniques, using a suitable NY , USA, 2019 , p. 770 - 773 . URL: https://doi.org/ distance metric and constraints on labels intu- 10.1145/3289600 .3290620. doi: 10 .1145/3289600. itively fit the task of grouping instances . 3290620 . [4] M. T. Ribeiro , S. Singh , C. Guestrin , "Why should

A linear model on a reduced feature set could be used i trust you? Explaining the predictions of any clasfor local relations between attributes . sifier, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery

Each class could produce a diferent “form” of neigh- and data mining , 2016 , pp. 1135 - 1144 .

borhood, diferent from the rigidity of LIME/Anchors . [5] D. Dua , C. Graf, UCI machine learning repository,

Figure 2 shows a toy example for LIME, Anchors and 2017 . URL: http://archive.ics.uci.edu/ml.