The rapid advance of Artificial Intelligence poses some fundamental ethical and social concerns, where providing the right explanations for instilling transparency in AI systems represents a main topic of discussion. An intuitive medium to provide explanations is through natural language, and with recent regulations comes an increasing need to provide evaluation methods for natural language explanations that will help us to assess the quality of those explanations in relation to the system they explain. This need for evaluating explanations has been further validated by studies from social sciences and psychology [2,8,11,17].
Particular questions arise around the amount of information needed to explain but not overload the user, the lexical choice for matching the user's understanding and expertise level, as well as the linguistic style adopted. Attributes such as informativeness, clarity, coherence, readability and effectiveness have been linked to human evaluation dimensions frequently used in the field of Natural Language Generation (NLG) [2,17]. Considering the strong focus of the NLG researchers on evaluating natural language, we propose that mapping existing NLG methods onto NL explanations can provide insights into the definition of a good explanation [3].
To get a better understanding of how we can define what makes an effective NL explanation, we designed and gathered the ExBAN Corpus (Explanations from Bayesian Networks). This corpus provides NL explanations for a set of Bayesian Networks, mainly motivated by the fact that Bayesian Networks are frequently used for the detection of anomalies in data and can approximate deep learning models. They also allow us to sense-check our explanation evaluation techniques as they are reasonably easy to understand for the non-expert user.
This paper presents current work into the evaluation of NL explanations , but also includes preliminary new linguistic analysis. The paper is thus structured into four parts: (1) we introduce the ExBAN corpus; (2) we present how automatic and human evaluation metrics from the field of NLG can be mapped onto NL explanations; (3) we present an analysis on how linguistic features correlate with human evaluation metrics; and (4) finally, we discuss how evaluation methods can capture the quality of NL explanations. Further details of this work can be found in [3].
Existing datasets of explanations have enabled significant progress in the way that explanations provide transparency of machine learning algorithms. However, less attention has been paid to methods to explain structured data, such as Bayesian Networks. Bayesian Networks have "the ability to cover any model with a probabilistic interpretation including supervised, unsupervised, and reinforcement learning (including deep learning)" [16]. Also, their graphical representation can be used for extracting information [7]. The ExBAN corpus is used here for evaluation, but it could also be used for training models for generating natural language explanations from graphical models such as Bayes Nets and other structured data more broadly.
Definition: ExBN: A Corpus of Natural Language Explanations for three Bayesian Networks's graphical representations (see Figure 1). Purpose: Possible application areas for the corpus: explainable AI, general artificial intelligence, academic linguistic research, natural language processing.
The ExBAN Corpus (Explanations for BAyesian Networks) consists of NL Explanations collected in a two-step process:
1. NL explanations were produced by human subjects (a total number of 84 participants) 2. In a separate study, these explanations were rated on a 7-point Likert scale, in terms of informativeness and clarity (a total number of 250 explanations, rated by 97 participants; each explanation was rated by minimum of 3 participants)
Models trained iteratively with large amounts of data are particularly hard to evaluate in a cost-effective and timely manner. Therefore, creating automatic methods for evaluating NLG systems that can capture the human-likeness of the generated output is essential. Human Evaluation. Explanations should be clear and easily understood by users, providing the right information in order to create better communication [6,9]. We focus on two dimensions: informativeness and clarity.
Automatic Evaluation. Here we describe automatic metrics used in the field of NLG evaluation and selected for this study, specifically: 1) word-based (untrained) metrics such as BLEU, METEOR and ROUGE, and 2) pre-trained metrics, such as BERTScore and BLEURT. Here, we briefly describe each in turn:
-BLEU [12] is a widely used metric in the field of NLG (borrowed from Machine Translation (MT)) that compares n-grams of a candidate text (e.g. that generated by algorithms) with the n-grams of a reference text. The number of matches defines the goodness of the candidate text. -SacreBLEU was proposed by [13] as a new version of BLEU that calculates scores on the detokenized text by applying its own metric-internal preprocessing. -METEOR was created in order to address the weaknesses of BLEU; ME-TEOR evaluates generated text by computing a score based on explicit wordto-word matches between a candidate and a reference. When using multiple references, the candidate text is scored against each reference, and the best score is reported. -ROUGE [10] evaluates n-gram overlap of the generated text (candidate) with a reference. -ROUGE-L computes the longest common subsequence (LCS) between a pair of sentences [14].
-BERTScore [18] is a token-level matching metric with pre-trained contextual embeddings using BERT [5] that matches words in candidate and reference sentences using cosine similarity.
-BLEURT [15] is a text generation metric based on BERT, pre-trained on synthetic data; it uses "random perturbations of Wikipedia sentences augmented with a diverse set of lexical and semantic-level supervision signals". BLEURT uses a collection of metrics and models from prior work, including BLEU and ROUGE.
In order to investigate the degree to which automatic metrics for NLG can capture the quality of NL explanations [3], we ran a correlation analysis with automatic metrics with human judgements. As shown in Figure 2, we can draw the following conclusions:
-Word-overlap metrics such as BLEU (n = 1,2,3,4), METEOR and ROUGE (n = 1,2) presented low correlation with human ratings. This might be due to certain limitations, such as the fact that they rely on word overlap and are not invariant to paraphrases. -BERTScore and BLEURT outperformed other metrics and produced higher correlation with human ratings than other metrics on all diagrams. These metrics might capture some relevant facts of explanations, as word representations are dynamically informed by the words around them.
We extracted a number of linguistic features presented in Table 1 to explore if there were any linguistic constructs that were found in our dataset and that mapped to good or bad explanations. For example, complex syntactic construction might lead to difficulty in understanding for the user, as reflected in the Height tree and Length tree features. Other features given in table 1 were motivated by similar studies correlating features with user ratings for surface realisation in NLG [4], and for social psychology [1]. We mapped the linguistic features in Table 1 to human evaluation metrics (informativeness and clarity) to see if there was any correlation between these features and the quality of the explanation, as rated by humans.
Our preliminary analysis shows some trends, but more investigation is needed to confirm these. We calculated the Spearman's correlation coefficient between the linquistic features and human evaluation ratings score, for both informativeness and clarity on a sample of 166 datapoints. With regards informativeness, the sentence length (r = 0.29) and the number of nouns (r = 0.36) presents weak correlatation with informativeness, as well as the number of coordinating conjunctions (r = 0.23).
Linguistic features do not seem to capture well the level of clarity of a sentence as no correlation was found in this regard. This is perhaps because clarity is multi-dimensional and implies more than lexical-syntactic relationships, including other factors such as causality, common sense and general knowledge.
Finding accurate automatic measures is challenging, particularly for explanations, as the pragmatic and cognitive processes underlying explanations, such as reasoning, causality, and common sense, might not be captured. In our study, the embedding-based metrics perform better than the word-overlap based ones, but we would recommend a larger study to show this empirically. Future work would involve examining the effectiveness of automatic metrics across a wider variety of explanation tasks and datasets. Finally, the next step is to use this work to automatically generate natural language explanations from structured data such as Bayes Nets, and this work contributes towards ensuring the quality of such explanations.
This work was supported by the EPSRC Centre for Doctoral Training in Robotics and Autonomous Systems at Heriot-Watt University and the University of Edinburgh. Clinciu's PhD is funded by Schlumberger Cambridge Research Limited (EP/L016834/1, 2018-2021). This work was also supported by the EPSRC ORCA Hub (EP/R026173/1, 2017-2021) and UKRI Trustworthy Autonomous Systems Node on Trust (EP/V026682/1, 2020-2024).