The growing demand for transparency, interpretability, and explainability of machine learning models and AI systems has fueled the development of methods aimed at understanding the properties and behavior of such models (XAI). Since diferent methods answer diferent explainability questions, it is crucial to understand the kind of explanation the diferent XAI-methods provide, and in what situations they should be used. We introduce eXplego, an interactive tree-structured tool designed to assist users in selecting the most suitable XAI method for their use case. eXplego prompts users to answer questions regarding the type of explanation they seek, guiding them along the branches of the decision tree for further inquiries. After 2-5 questions, the tree reaches one of its leaves to suggest an XAI method aligned with the user's explainability need. The tool also provides helpful practical examples, simplified descriptions of the suggested method's functionality and interpretability, points to consider when using the method, and links to the paper introducing the method, additional resources, and software implementations. The tool is developed from an in-depth study to discern the characteristics of the most prominent methods and the nature of the explanations they provide. We believe eXplego will help streamline the process of XAI method selection and contribute to the practical implementation of XAI in various domains. The tool is available at explego.nr.no.
A plethora of XAI methods and variations thereof have been proposed in recent years, typically
grounded in formal and narrowly defined mathematical notions of interpretability [
In recent years, various XAI taxonomies have been proposed [
The recent taxonomy review paper [
We have restricted ourselves to post-hoc, model-agnostic explanation methods for tabular data models in eXplego. While explaining text and image-based models is important, their data formats require other types of questions to identify an appropriate explanation method. Moreover, we believe the need for a navigation tool is most pressing for post-hoc, model-agnostic methods, precisely because they can be widely applied – hence this additional restriction.
The eXplego tool prompts the users to answer questions regarding the type of explanation they seek, guiding them along the decision tree for further inquiries. Each question also comes with practical in-place examples. A section of the eXplego tool is shown in Figure 1. After 2-5 questions, the tool suggests an XAI method aligned with the user’s explainability needs.
The leaves also contain a short summary of its use and interpretability, a list of method usage considerations, and links to methodological papers, additional resources, and software implementations. Figure 1: A section of the eXplego tool.
eXplego is developed based on methodological and practical XAI experience, and an extensive study of the most prominent XAI methods and the kind of explanations they provide. The methods included in eXplego are listed below, along with brief justifications for their placement in the tree.
feature at a time Measures the value of observing the features in the whole model as it permutes features and measures the change in model performance. Since it permutes one feature at a time it does not consider joint eforts/dependence among features.
Decompose model loss onto features to explain the whole model in terms of the features. As Shapley values fix some subsets of features while imputing the others, it explains the value of observing the features considering their joint eforts .
Per-feature plots show changes in the ‘average’ prediction as one feature is altered. Thus, they explain how the whole model reacts to changes in the feature values.
Decomposes a performance score for the whole model on training observations. Similarly to Shapley values, subsets of observations are interchangeably fixed, while others imputed. Hence, it explains the value of observing the training observations.
eforts among features → Obey data distribution Explains specific predictions in terms of features, by decomposing them onto the features. As subsets of observations are interchangeably fixed/imputed, it explains the value of observing the features where joint eforts are considered. Properly estimated conditional expectations ensure feature dependence is accounted for.
among features → Ignore feature dependence Exactly like Conditional Shapley values, but estimates the conditional expectations with a simpler method ignoring the feature dependence.
a time Explains how specific predictions are afected by observing single features (assuming others known) by measuring how the prediction changes as they are replaced by conditional expectations.
decision → Same decision By providing feature space regions where a decision based on a prediction is unchanged, it explains specific predictions in terms of changes in features values for the same decision that was reached by the specific prediction.
values → Categorical decision → Diferent decision
score) by providing examples of (minimal) changes to the feature values that would give the desired decision.
prediction → Joint eforts among features By fitting a local surrogate model to a joint feature set sampled around a prediction, the method explains specific continuous predictions directly in terms of changes in the feature values, while accounting for joint eforts among features .
→ One feature at a time
prediction scores against single altered features.
ing the observations Uses Shapley values to explain the value of including (clusters of) training observations by decomposing specific predictions onto the diferent clusters.
change when perturbing features in the training observations.
To the best of our knowledge, the eXplego tool is unique in its form. That said, the structuring proposed in IBM’s Explainability 360 Toolkit [30] bears conceptual resemblance. eXplego difers in the following key points: eXplego is more comprehensive, covers a wider range of XAI methods, and is geared towards developers in that question prompts are more informed by the technicalities of the XAI methods. As explained above, eXplego is also interactive, and provides both in-place examples to help the user answer the questions, and detailed information beyond the method’s name in the leaves.
Since our tool is restricted to models for tabular data, we encourage other researchers to
apply our format to other scenarios and model types, such as text and images. Further, our tool
is limited to identifying quantitative XAI methods most befitting diferent use cases. Privacy,
contextual and normative dimensions [31], also need to be considered when providing adequate
and trustworthy explanations [
Finally, our tool has been built with the open-source diagramming application draw.io. The source code for our tool is available at github.com/NorskRegnesentral/explego. Feedback and suggestions for new methods are all welcome and can be submitted by opening an issue in the GitHub repository.
It is our impression that the practical dificulty of matching explainability needs with existing XAI methods is underestimated, and eXplego is a practical tool that can guide its users in selecting an appropriate explanation method.
The tool can inspire future research: As [32] puts it: "Despite the recent resurgence of explanation and interpretability in AI, most of the research and practice in this area seems to use the researchers’ intuitions of what constitutes a ‘good’ explanation." The tool can also be used to highlight explainability questions that no XAI method addresses. For instance, that our tree lacks a question addressing feature dependence for global explanations of feature observation with joint eforts, identifies that the SAGE method lacks a counterpart using conditional Shapley values. Finally, we believe eXplego will streamline XAI method selection and contribute to practical implementation of XAI in various domains. Greece, June 17–20, 2022, Proceedings, Part I, Springer, 2022, pp. 447–460. [16] P. Saleiro, B. Kuester, L. Hinkson, J. London, A. Stevens, A. Anisfeld, K. T. Rodolfa, R. Ghani,
Aequitas: A bias and fairness audit toolkit, arXiv e-prints (2018) arXiv–1811. [17] L. Breiman, Random forests, Machine learning 45 (2001) 5–32. [18] I. Covert, S. M. Lundberg, S.-I. Lee, Understanding global feature contributions with additive importance measures, Advances in Neural Information Processing Systems 33 (2020). [19] D. W. Apley, J. Zhu, Visualizing the efects of predictor variables in black box supervised learning models, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82 (2020) 1059–1086. [20] J. T. Wang, R. Jia, Data banzhaf: A robust data valuation framework for machine learning, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2023, pp. 6388–6421. [21] K. Aas, M. Jullum, A. Løland, Explaining individual predictions when features are dependent: More accurate approximations to shapley values, Artificial Intelligence 298 (2021) 103502. [22] S. M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, in:
Advances in neural information processing systems, 2017, pp. 4765–4774. [23] M. Robnik-Šikonja, I. Kononenko, Explaining classifications for individual instances, IEEE
Transactions on Knowledge and Data Engineering 20 (2008) 589–600. [24] M. T. Ribeiro, S. Singh, C. Guestrin, Anchors: High-precision model-agnostic explanations, in: Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018. [25] S. Wachter, B. Mittelstadt, C. Russell, Counterfactual explanations without opening the black box: Automated decisions and the GDPR, Harv. JL & Tech. 31 (2017) 841. [26] M. T. Ribeiro, S. Singh, C. Guestrin, "Why should I trust you?" Explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144. [27] A. Goldstein, A. Kapelner, J. Bleich, E. Pitkin, Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation, Journal of Computational and Graphical Statistics 24 (2015) 44–65. [28] A. Brandsaeter, I. K. Glad, Shapley values for cluster importance, Data Mining and
Knowledge Discovery (2022) 1–32. [29] P. W. Koh, P. Liang, Understanding black-box predictions via influence functions, in:
International conference on machine learning, PMLR, 2017, pp. 1885–1894. [30] V. Arya, R. K. Bellamy, P.-Y. Chen, A. Dhurandhar, M. Hind, S. C. Hofman, S. Houde, Q. V.
Liao, R. Luss, A. Mojsilović, et al., Ai explainability 360 toolkit, in: Proceedings of the 3rd ACM India Joint International Conference on Data Science & Management of Data (8th ACM IKDD CODS & 26th COMAD), 2021, pp. 376–379. [31] H. de Bruijn, M. Warnier, M. Janssen, The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making, Government Information Quarterly 39 (2022) 101666. [32] T. Miller, Explanation in artificial intelligence: Insights from the social sciences, Artificial Intelligence 267 (2019) 1–38.