In the GEESE challenge, we present a pipeline to evaluate generated explanations for the task of Recognizing Textual Entailment (RTE) in Italian. The challenge focuses on evaluating the impact of generated explanations on the predictive performance of language models. Using a dataset enriched with human-written explanations, we employ two large language models (LLMs) to generate and utilize explanations for semantic relationships between sentence pairs. Our methodology assesses the quality of generated explanations by measuring changes in prediction accuracy when explanations are provided. Through reproducible experimentation, we establish benchmarks against various baseline approaches, demonstrating the potential of explanation injection to enhance model interpretability and performance.
The ability of a machine to justify its predictions and provide human-understandable explanations has been a key research objective of Machine Learning (ML) and Artificial Intelligence (AI) since their early stages [1,2,3]. In the past few years, the field of AI has experienced an unprecedented acceleration in most areas, such as computer vision [4], audio [5], video [6], and programming languages [7], and especially in Natural Language Processing (NLP), with the popularization of generative Large Language Models (LLMs) such as OpenAI's Chat-GPT [8], Google's Gemini [9], or Meta's Llama [10].
These models are currently able to produce naturalsounding and coherent language, often indistinguishable from natural language [11,12]. While these results open up new avenues for future applications and research, they also raise ethical issues considering the ubiquitous role of machines in our lives, and in sensitive fields like education, health, justice, and private life. In fact, the scarce transparency of neural architectures makes it hard to interpret their functioning (the so-called "black-box" problem). In addition, many of the currently available LLMs are not fully open-source, so the data they were trained on is not known to either researchers or the general public. Finally, these models have achieved such sizes that their results are difficult to replicate, making them a kind of "black box in a black box".
As a consequence, the need to develop methods to understand their reasoning is becoming central. Many recent efforts have been devoted to explaining such models [13], and the importance of interpretability and explainability in AI has become ever more urgent [14,15,16].
The role of explanations in NLP has been explored by a consistent body of research. Cambria et al. [17], for instance, provides a comprehensive survey of approaches for generating natural language explanations; Hartmann and Sonntag [18] examines the benefits of explanations for NLP models; Paranjape et al. [19] focuses on templatebased explanations, Lampinen et al. [20] and Ye and Durrett [21] demonstrate the benefits of in-context explanations for large models in challenging reasoning tasks.
Explanation generation quality has traditionally been evaluated through automated ovelap metrics like BLEU [22], ROUGE [23], or BERT-Score [24] against a gold reference explanation written by humans. This usually implies costly human-explanation collection campaigns; additionally, these measures may neither fully capture the informativity or the effectiveness of an explanation, nor faithfully reflect human judgments.
Recently, human simulatability scores have been proposed as an alternative method to understand the quality of explanations from the perspective of the "utility to an end-user" [25]. Rather than focusing on the overlap between explanations and ground-truth data, this approach assesses how explanations enhance predictive performance on a downstream task compared to the input alone. While humans have traditionally been the predictors [26], recent research has demonstrated that trained models can automate this process, showing moderate to strong correlations with human judgments [27]. Pruthi et al. [28], for instance, measures explanation quality based on downstream performance: their methodology involves training a student model on explanations generated by a teacher, using automatic explanation generation techniques and training the student for the end task.
However, current LLMs may also benefit from explanation injection even if they are not explicitly trained to do so, and some works suggest using the explanation to augment the input to condition predictions of future data points on both the input and the explanation [29,27]. In fact, LLMs are capable of understanding supplementary input content and including explanations in the input during inference without requiring additional supervision, which can indirectly demonstrate the role of explanations in the inference process.
These observations underline two crucial aspects:
β’ providing LLMs with quality explanations that allow them to infer relevant latent information, i.e. to provide additional background knowledge, improves performance compared to only using the input or to using spurious explanations; β’ the quality of a (human or machine-generated) explanation can be measured based on its helpfulness (or impairment) to the (model's or human's) performance on a downstream task.
To contribute to this line of research, we propose GEESE: Generating and Evaluating Explanations for Semantic Entailment at CALAMITA [30], a pipeline to indirectly assess the effectiveness of explanations through the evaluation of their impact on the task of Recognizing Textual Entailment (RTE) in Italian 1 .
Consider a pair of sentences < π 1 , π 2 >, like the ones in the following example:
(1) Il cielo Γ¨ grigio oggi.
(2) Faresti bene a prendere l'ombrello. 2 Consider a semantic relation π holding between π 1 and π 2 (e.g.,
Let πΈ be the set of possible explanations for π. GEESE's explanatory task consists in:
β’ generating an explanation π π β πΈ for the semantic relationship π for each < π 1 , π 2 > in the dataset; β’ predict the relation with and without the generated explanation π π ;
1 Code and data are made available at github.com/andreazaninello/ calamita-geese 2 (1) The sky is grey today. (2) You better take your umbrella with you.
β’ assess the quality of the generated explanations πΈ πππ by taking the delta between prediction accuracy with and without explanation as a proxy of explanations' quality.
Step 1: Generate Explanation: A first LLM (π 1 ) is prompted to produce explanations πΈ πππ = {π 1 , π 2 , β¦ π π } for a specific semantic relation π π holding between a given sentence pair, denoted as < π 1 , π 2 >. In the task, we focus on the entailment relationship, which can take three values: "YES" (sentence 1 is entailed by sentence 2), "NO" (sentence 1 is contradicted by sentence 2), "UNKNOWN" (sentence 1 is neither entailed nor contradicted by sentence 2). In our baselines, we focus on one explanation type (why-explanation), but other kinds of explanations or reasoning strategies (like counterfactual or examplebased ones) are possible. In our baselines, we use llama-3-3B-instruct [31] as π 1 .
Step 2: Use Explanation on Relation Prediction: A second LLM (π 2 ) is then provided with the generated explanations πΈ πππ to evaluate if the generated explanations improve the task of predicting the correct relations. In practice, this is achieved by appending the explanation as a "hint" to the prompt, and asking the model to make a prediction thereof. This process aims to discover how effectively π 2 leverages the explanations from π 1 to perform the target task. We use llama-3-8B as π 2 , but other combinations of π 1 and π 2 are possible.
Step 3: Evaluate Explanation Effectiveness Explanation effectiveness is evaluated by analyzing how providing different explanations generated in Step 1 affects the model π 2 prediction in Step 2. In practice, this is done by calculating the accuracy of the predictions of π 2 given the explanations and comparing them to the selected baselines (see Section 4).
The Recognizing Textual Entailment (RTE) task emerged in 2005 [32] as the problem of determining if two sentences stand in an entailment or not-entailment relationship. A common definition of "semantic entailment" (also referred to as presupposition in some studies) is that "A sentence S presupposes a proposition p if p must be true in order for S to have a truth-value (to be true or false)" [33]. A text t is said to entail another text (hypothesis, h) if h is true in every circumstance (possible world) in which t is true. RTE, however, suggests a more empirical definition, allowing for cases in which the truth of the hypothesis is highly plausible, for most practical purposes, rather than certain. According to [34], this "shallow" definition better accounts for the types of uncertain inferences that are typically expected from text-based applications.
Recognizing Textual Entailment was formalized through a series of successful challenges and workshops that began in 2005 [32] and lasted until 2012. Starting from the RTE-3 edition, the task was extended from two labels to a three-label classification, splitting the notentailment label into two classes, contradiction and neutrality. Given the interest in the task, an Italian version of the RTE-3 dataset was developed to explore language comprehension and textual entailment [35].
The dataset used in the challenge is the e-RTE-3-it dataset [36], which is an emended version enriched with human-written explanations of the RTE-3-it dataset [35].
The dataset contains 1600 text-hypothesis sentence pairs in Italian (text_t and text_h in the dataset) divided into an 800-example validation and an 800-example test split. Each example is annotated with an entailment label (label): "YES" (entailed), "NO" (contradicted), or "UNKNOWN" (neutral).
The e-RTE-3-it dataset presents human explanations written in Italian by native speakers. For each text-hypothesis pair, annotators provided a natural language explanation justifying the given label (explanation) for the entailment relation ("why does π 1 stand in an π relation with π 2 ?") 3 .
All annotations underwent quality control, involving two expert linguists who manually checked the explanations for grammaticality, fluency, and logical validity. This process ensured high quality of the final e-RTE-3-it explanations, informativeness, as well as minimal label leakage (see infra).
Label leakage [37] refers to the fact that the explanation may be directly suggesting the label without genuinely being informative. While the manual check of all original human explanations ensured minimal label leakage, to prevent this we automatically replace direct references to the label and to the task with placeholders in the human-written and generated explana-tions. In our implementation, this is done through regular expressions by substituting ("anonimize") the label strings ("YES", "NO", "UNKNOWN") and all words starting with entail.*, contradict.*, neutr.*, impl*, contradd.* (verbs and nouns directly stating the kind of relationship) with "XXX".
We therefore also provide the following "anonymized" additional explanations for each example, which we use in our prompts:
β’ anon_whyexp: the anonimized explanation generated by llama3 as π 1 ; β’ anon_human: the anonimized human-written explanation (from e-RTE-3-it).
The dataset is freely distributed in HuggingFace's Dataset format 4 . A snippet of the data is displayed in Table 1.
We conduct baseline experiments using Llama-3.1-8B-Instruct as π 1 with a custom implementation in Hugging-Face, and Llama-3-8B as π 2 , using the LLM-Evaluation-Harness library [38] in a zero-shot setting 5 . We provide baselines for the following settings:
1. no-exp: No explanations provided (baseline); 2. dummy: The hypothesis itself (text_t) provided as a "non-informative" explanation, controlling for input length and providing a second baseline. 3. human: Human-written explanations (from e-RTE-3-it) anonimized (anon_human) provided as additional input; 4. llama-3: The explanation generated using LLama-3-8B-Instruct as π 1 (anon_whyexp).
All experiments have been carried out in a zero-shot setting using the following prompts 6 .
Your task is to clarify the entailment relationship between a pair of sentences by explaining why a classifier predicted The XXX relationship between the two sentences is XXX because the relationship is not clearly established. Sentence 1 mentions the Inuit people, the Commission on Human Rights, and the United States, which are not directly related to the topic of the second sentence, which is about the threat of extinction of species due to hunting. The connection between the two sentences is not explicitly stated, and it is not clear how the information presented in Sentence 1 supports or XXX the information in Sentence 2. Therefore, the XXX label is XXX because the relationship between the two sentences is not clear. For example, if the XXX label was XXX, it would mean that Sentence 1 logically implies Sentence 2, but the information in Sentence 1 does not provide enough evidence to support this claim. Similarly, if the XXX label was XXX, it would mean that Sentence 1 logically XXX Sentence 2, but again, the information in Sentence 1 does not provide enough evidence to support this claim. Therefore, the XXX label is XXX because the relationship between the two sentences is not clear. anon_human Nella frase non vengono citate le conseguenze della caccia sulle foche perciΓ² non si puΓ² evincere il rischio per loro della caccia.
Example from the processed e-RTE-3-it dataset.
a specific entailment label. Sentence 1: text_t Sentence 2: text_h Entailment label: label. exp_type 7
(M2 -Prediction):
Your task is to predict the entailment label between two sentences, selecting one label among YES (entailment), NO (contradiction), or UNKNOWN (neutrality). Sentence 1: text_t Sentence 2: text_h Hint: anon_explanation. Entailment label: 8
Baseline results, reported in Table 2, demonstrate the impact of incorporating explanations on the performance of language models in the Recognizing Textual Entailment tasks. The accuracy scores indicate that models utilizing explanations generated by Llama-3 achieve the highest 7 Variables are indicated in color. In our experiments exp_type = "Explain how the two sentences are connected. " and the variables are read from each example. 8 Variables are indicated in color. In our experiments, anon_explanation can take the following values: "Not given. " (no-exp), text_h (dummy), anon_human (human), anon_whyexp (llama-3). accuracy at 78.12%. In comparison, using human-written explanations shows slightly lower accuracy compared to machine-generated, but higher scores compared to baselines, suggesting that explanations do enhance the models' understanding of semantic relationships.
Generated explanations, proving more effective than human-crafted ones, suggest that the quality and type of explanations provided can influence predictive performance, but also highlight the need for further research into optimizing explanation generation methods for improved outcomes in NLP tasks. In fact, note that generated explanations may be positively influenced by factors other than informativeness alone, such as the lengths of the explanations themselves, or may still be indirectly suggesting the right relationship despite the anonymization process described in 3.3.
For example, as reported by one of the anonymous reviewers, see "anon_whyexp" explanation in Table 1: "In other words, Sentence 2 provides enough information to infer the truth of Sentence 1". The generated explanation clearly (but not directly) hints at an "entail" label, potentially compromising the intended anonymity. The fairness of the comparison between human-and machinegenerated explanation is an aspect that deserves further investigation.
Results for the 0-shot baseline experiments on the full test set.
The findings from the GEESE challenge underscore the significance of effective explanation generation in enhancing the capabilities of language models in RTE tasks. Preliminary results show that models provided with explanations, whether human-written or generated by LLMs, exhibit improved predictive accuracy compared to those lacking such inputs. This supports the hypothesis that explanations can facilitate a deeper understanding of semantic relationships, thus aiding model inference.
The GEESE challenge establishes a framework for generating and evaluating explanations in the domain of semantic entailment. By demonstrating the utility of explanation injection, we contribute to the ongoing discourse on interpretability in AI, advocating for a balanced approach that enhances model transparency while maintaining robustness. Our findings encourage further exploration into the interplay between explanations and model performance, paving the way for more interpretable and user-friendly AI systems. As language models continue to evolve, integrating effective explanation mechanisms will be crucial for ensuring their responsible deployment in sensitive applications.
The study also highlights limitations, including potential biases in the generated explanations and the challenge of ensuring that explanations remain informative without directly revealing the answer. Future research could explore diverse explanation types and their varying impacts across different contexts and languages.
We would like to draw the readers' attention on the following. Firstly, the potential for bias in both the training data and the generated explanations can perpetuate stereotypes or misinformation, leading to harmful consequences, particularly in sensitive domains such as healthcare or legal applications. There is also the risk that users may place undue trust in machine-generated explanations, mistakenly believing them to be infallible. Finally, the collection and use of data for training these models must adhere to strict privacy standards to ensure that individuals' rights are respected. Addressing these ethical challenges is essential to foster trust and ensure that AI technologies are developed and used responsibly.
We release our original content under the MIT License.
Please refer to the original dataset's copyright and license regulations for information on the derived data.
This work has been partially funded by PNRR project FAIR -Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU and the ANTIDOTE project (CHIST-ERA grant of the Call XAI 2019 of the ANR with the grant number Project-ANR-21-CHR4-0002)