Explanations are crucial for building trustworthy AI systems, but a gap often exists between the explanations provided by models and those needed by users. We build on prior research and direct feedback from clinicians, which reveal that users prefer interactive, question-driven, and diverse explanations. To bridge this gap, we present MetaExplainer, a neuro-symbolic framework that generates user-centered, multi-type explanations tailored to user questions. MetaExplainer follows a three-stage pipeline: (1) decompose user questions into machine-readable representations using state-of-the-art large language models (LLMs), (2) invoke appropriate model-specific explainer methods to generate recommendation rationales, and (3) synthesize coherent, user-friendly naturallanguage explanations that summarize and contextualize these outputs. We demonstrate MetaExplainer with an end-to-end example on the widely used PIMA Indian Diabetes dataset, highlighting how the framework addresses diverse clinical questions. Overall, MetaExplainer ofers a versatile, traceable approach to explanation generation, capable of addressing a broad spectrum of user queries and advancing AI explainability across domains. MetaExplainer's implementation, along with quantitative and qualitative evaluation results, is detailed in [1], and the open-source code is publicly available at https://github.com/tetherless-world/metaexplainer.
Explainable AI has been a cornerstone of research for decades and has evolved in tandem with
advancements in AI approaches. However, with the increase in complexity of AI methods, the results of
explainer methods alone have been found insuficient for end-user needs. Several researchers [
Hence, in the pursuit of supporting natural-language (NL) [
CEUR
ceur-ws.org
Our MetaExplainer is a framework that can generate natural-language explanations in response to user questions, along explanation types (Tab. 1) we currently support. In the design of the MetaExplainer (Fig. 1), we adopt a modular approach - in that we break down the design into three stages; decompose: convert the user question into a machine interpretation that can then be delegated: to explainer methods registered against explanation types in EO and finally, the explainer outputs are synthesized into NL explanations presented to the end-users. We implement the framework as a Python package, such that each of the stages can call methods and classes from one another , enabling the stages to build of of the intermediary stage outputs as seen in Fig. 1.
Decompose: In the Decompose stage, our objective is to generate a machine-actionable parse of a user question(s), including the explanation type that best addresses the question and the features being asked about. For example, in a question - “Why is a 60-year-old woman with a BMI of 28 more likely to have Diabetes?”, we would want to identify that this question is best addressed by facts in contrastive explanations and has filters applied on features - age, BMI. These attributes, if captured, help ensure that the explainer methods can reliably address the user’s question. We find that predicate logic can serve as a good intermediate to parse in further stages of the MetaExplainer. Hence, we build a question bank of : pairs (leveraging question type-explanation type mappings in EO) for each explanation type, expand this question bank using GPT, and fine-tune LLama2 and Llama3 LLMs on this question bank to generate from . Hence, in the Decompose stage, we set up a task to convert a user question into a reframed question , which is a tuple capturing the {question, explanation type, machine interpretation, action}.
Delegate: In the Delegate stage, the objective is to execute relevant explainer methods to address a user question ( ). Here, we further break it down into sub-tasks: parsing the , executing explainers ( ), and processing explainer outputs ( ). In the parsing task, we parse the machine interpretation ifeld of the for filters on features, the explanation type ( ) field to identify what explainer methods ( ) can be run, and the action field to identify if there are other actions the question asks about (e.g., preprocessing, accuracies, etc.) In the execution task, we run open-source explainer methods ( ) (Tab. 1), e.g. SHAP for contrastive explanations. We leverage the explanation type - explainer method mappings in the EO to identify what explainers to run. Finally, in the output processing step, we persist the explainer outputs as dataframes based on the explanation modality (i.e., features, instances or rules, again mappings captured in the EO), that the Synthesis stage can then use.
Synthesis: In the Synthesis stage, we combine and synthesize the explainer output(s) into
naturallanguage (NL) explanations aligned with the identified explanation type’s ( ) expected templates. We
further break down the Synthesis into three sub-tasks, including a retrieval step where we extract
relevant data points from the explainer outputs to include in the final explanation, an
augmentation/alignment step where we align the outputs to fit the identified explanation type’s (
) templates,
and a generation step where we output NL explanation (ℰ ) populated with the retrieved content in line
with the template for the . Hence, the synthesis task lends itself well to an application of the widely
popular Retrieval-Augmented-Generation (RAG) [
Algorithm 1 Pseudocode for MetaExplainer indicating the three stages - Decompose, Delegate, and Synthesis, and their inputs and outputs.
Require: Explanation Ontology (EO), Data Store (DS)
Explanation type - explainer graph, 1 = {( , ), ∀ ∈ and ∀ ∈ } Data type - explainer graph, 2 = {( , ), ∀ ∈ and ∀ ∈ }
= Decompose( ) = Delegate( ) = Synthesis( )
User Question list of questions reframed from list of explanations { } that answer where = {ℰ , , , , }
We currently support five explanation types in the MetaExplainer and explainer methods that can generate explanations along the same, as shown in Tab. 1. System developers could add support for more explanation types by including the explainer methods in the Delegate stage of the MetaExplaier and registering the expected outputs and question types in the EO.
We evaluated the MetaExplainer, by applying the five explainer methods (Tab. 1) we currently support in the Delegate stage on the best performing predictive model, a random forest (RF) model on the
Definition Provides solutions that are based on actual prior cases that can be presented to the user to provide compelling support for the system’s conclusions and may involve analogical reasoning, relying on similarities between features of the case and of the current situation.
Answers the question “Why this output instead of that output,” making a contrast between the given output and the facts that led to it (inputs and other considerations), and an alternate output of interest and the foil (facts that would have led to it).
Addresses the question of what solutions would have been obtained with a diferent set of inputs than those used.
Focuses on what the data is and how it has been used in a particular decision, as well as what data and how it has been used to train and test the ML model. This type of explanation can help users understand the influence of data on decisions.
About the “why” of an ML decision, and provides reasons that led to a decision, and is delivered in an accessible and understandable way, especially for lay users.
Explainer Method
Protodash
SHAP
DiCE
Protodash
RuleXAI
widely used tabular dataset, PIMA Indian Diabetes [
1 Question: How did the model justify predicting Diabetes for a 55-year-old male with a BMI of 18 and a
Diabetes Pedigree Function of 0.25?
As shown in Fig.s 2 and 3, our MetaExplainer framework processes a user question, : ”How did the model justify predicting diabetes for a 55-year-old male with a BMI of 18 and a diabetes pedigree function of 0.25?”
In Fig. 2, the user question, , is translated into its predicate logic equivalent, , and a rationale
explanation type is identified during the Decompose stage. The fields for machine interpretation
and explanation type are then passed to the Delegate stage. Here, the RuleXAI post-hoc explainer
method [
Finally, as depicted in Fig. 3, the rules and the user question, , are utilized by the Synthesis stage to produce NL summaries, , of the subset of data that matched the feature group restrictions in the question and the rules responsible for the classification model’s prediction of diabetes for a 55-year-old male with a BMI of 18 and a diabetes pedigree function of 0.25. This illustrates how each stage of the MetaExplainer builds upon the outputs of previous stages. Our modular, multi-stage framework efectively captures the restrictions in the question and traces back outputs that a purely LLM approach might miss.
1 Explanation of Matched subset: 2 There are no full matches in the dataset based on the specified feature group. However, the dataset has been summarized, showing the descriptive statistics for the variables in the dataset. 3 - The mean age is 68.1 years. 4 - The mean BMI is 29.49. 5 - The mean Diabetes Pedigree Function (DPF) is 0.4868. 6 - The outcome variable has a mean value of 0.3, indicating that the majority of the observations have an outcome of 0. 7 8 Explanation of explainer output: 9 The rules derived for classification based on the features 10 'Age' and 'BMI' are as follows: 11 1. IF BMI is less than or equal to 32.25, THEN label = 0 12 2. IF BMI is greater than 37.05, THEN label = 1 13 14 These rules provide a clear guideline for classifying data based on BMI values, with different labels assigned depending on the BMI range. This type of explanation helps users understand the rationale behind the decision-making process in machine learning models, allowing them to assess the reasoning and make informed judgments.
We provide an overview of the MetaExplainer, a multi-stage explanation generation framework that we have designed and implemented, that combines several AI and machine learning methods - LLMs, ontologies, predictive and explainer models- to generate user-centered explanations. We evaluate our framework both qualitatively and quantiatively (evaluating outputs at each stage) on an indicative healthcare use case, Diabetes, with plans and provisions to expand beyond. Since our framework is modular, system developers can add support for explanation types beyond those currently supported. We have open-sourced our implementation and invite the community to contribute. Overall, the MetaExplainer helps address a timely problem of tailoring explanations to end-user needs and leverages the strengths of a neuro-symbolic approach to do so.
During the preparation of this work, the authors used Grammarly in order to rephrase some of the sentences and also to fix grammar and spelling issues. After using these tools and services, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.