Argumentation, by which we here mean the ability to give reasons or arguments for a claim, plays a central role in society generally and in collaborative decision-making specifically. However, the role of argumentation in human-AI collaboration and AI-assisted decision-making has received limited attention, despite the widespread interest in "explainable" AI. This paper aims to bridge this gap. First, it is shown that many kinds of AI models are not argumentative in the sense that they do not enable human-AI interfaces to provide reasons or arguments for AI predictions. Second, it is shown that some interpretable AI models encode a knowledge structure that can be harvested for the purpose of supporting argumentative human-AI interaction. Third, a method for extracting such structures from an interpretable model is outlined. Finally, a prototype supporting argumentative dialogue between AI and human user is presented.
Argumentation plays a crucial role in society generally and in collaborative decision-making more specificially [1]. By requesting and providing support for claims, we justify our beliefs and actions, and evaluate claims made by others. In the context of artificial intelligence (AI) based on machine learning (ML), it is natural to treat predictions made by AI systems as claims. For example, if a statistical model predicts that a certain individual is introverted, we can intuitively understand this prediction as a claim. It is then natural to also ask whether the model can provide arguments for its claim? In current discourses revolving around AI, this question is typically approached as a matter of explainability or interpretability (see e.g. [2]). A distinction is often made between black-box models whose predictions and inner workings can only be explained by means of inherently unreliable explanation methods [3], and interpretable models whose logic can in principle be understood by humans [4]. However, from the perspective of human-AI collaboration, the notions of explainability and interpretability are not necessarily crucial in and of themselves. In this paper, we instead hypothesize that human-AI collaboration yield more value when the AI systems can engage in argumentation. The main aims of the paper are to briefly discuss the conditions that support argumentative dialogue between a machine learning model and human, and to demonstrate how such a capacity can be conceived theoretically and implemented technically.
The present work applies Toulmin's [5] theory of argumentation to ML-based AI. 1 According to this theory, argumentation is an interactive process through which presented claims are challenged and backed. Specifically, a claim (e.g. that Sam is introverted) can be backed with data (e.g. that Sam doesn't like danceable music). Data support claims by highlighting specific facts or circumstances. Furthermore, the backing of claims by data (often implicitly) rests on warrants (e.g. that people that like non-danceable music are generally introverted). While claims and data are specific, warrants are general; their argumentative function is to bridge data and claims by means of e.g. taxonomy (that instances of one category are always instances of another category) or statistics (that instances of one category also tend to be instances of another category). Warrants support conclusions with varying degrees of force, signalled linguistically with a qualifier (e.g., statistically, people that like non-danceable music are more introverted, and Sam doesn't like danceable music; so, presumably Sam is introverted).
The extent to which ML-based AI systems support argumentation differs across different kinds of ML models. Although a comprehensive assessment of this matter is beyond the scope of this paper, some brief remarks can be made. First, it can be noted that black-box models such as deep neural networks and random forests do not afford argumentation in any obvious manner. While claims (e.g. classifiying a person as introverted) can straightforwardly be qualified in terms of confidence (e.g. that the prediction is associated with a probability of 67%), data and warrants cannot readily be identified due to the complex inner workings of these models. To some extent, feature-importance based explanation methods such as LIME [7] and SHAP [8] can be conceived to identify data, since they highlight the features that were the most important for a particular prediction. For example, if a neural network predicts that a person is introverted based on the person's music preferences (measured as numerical values for features such as danceability and loudness), LIME may highlight danceability as the most important feature for the prediction at hand. From this information, one can construct the datum "On a scale from 0 to 1, Sam's preference for danceable music is 0.34" as a claim-backing datum. But what kind of warrant supports the conclusion from datum to claim? Has the model learned that people with a preference for danceable music of exactly 0.34 are generally introverted? Or has it learned something more general, e.g. that introverts prefer music with a danceability value below a certain threshold? These questions cannot be answered by methods such as LIME or SHAP. In reality, a black box may combine preference for danceable music with preference for loud music and other feature values in non-linear and complicated ways that may be difficult or impossible to express in words. In argumentative terms, no warrant can be generated.
For a more interpretable option, we can consider linear additive models such as logistic regression. In contrast to black boxes, these models force features to affect output independently of each other, without any interactions. 2 Furthermore, features affect output monotonically; for example, a stronger preference for danceable music always increases the predicted probability that the person is extraverted. Due to these formal properties, warrants such as "statistically, people that like danceable music are more extraverted" will faithfully reflect the actual knowledge learned by the model. Below, we will formally show how to extract data and warrants for claims obtained from linear additive models.
We assume a linear additive model on the form
where β 0 is the intercept/bias, β i are the coefficients (both of which are learned when fitting the model to training data), X is the instance (feature values), i denotes feature (e.g. 1 for energy, 2 for danceability, etc.), and y ˆis the output (predicted value). We also assume that output and features are standardized continuous variables so that 0 corresponds to mean. Sticking to the example domain above, we say that if the prediction is positive (y ˆ> 0), it is claimed that the person described by X is extraverted, or else introverted. Data supporting a positive claim can then be extracted by listing features with a positive value and a positive coefficient, and features with a negative value and a negative coefficient, since in both of these cases the feature contributes to a positive prediction. Conversely, a negative claim is supported by features with a positive value and a negative coefficient, and features with a negative value and a positive coefficient. A datum can be conveyed linguistically with reference to the feature and its polarity; for example, X 2 > 0 can be expressed as "The person likes danceable music".
As warrants, we construct support-relations between the polarity of a coefficient (e.g. β 2 < 0) and the polarity of the prediction (e.g. y ˆ< 0). This can be expressed linguistically as "Statistically, people that like non-danceable music are more likely to be introverted". The force of each combination of datum and warrant can be defined as the magnitude of the respective addend
In what follows, we will see how this extraction procedure can form the basis for an argumentative AI communicator.
We briefly describe MindTone 3 , a browser-based game featuring argumentative communication between AI and human. The task of the game is to estimate whether persons are extraverted or introverted based on their music preferences. In each round of the game, the player is shown specific tracks that a person has listened to frequently as well as audio statistics of music heard by the person, such as loudness and energy (see figure 1). The player is assisted by a chatbot that predicts whether the person is extraverted or introverted using a logistic regression model trained on audio features and personality traits [9]. The dialogue manager is implemented in a rule-based information-state update approach [10] inspired by conversation-oriented semantics [11]. Specifically, when a session starts, the system makes a claim, e.g. "I think this person is introverted" or "If I had to guess, I'd say that this person is extraverted" (where the qualification reflects the model's confidence). When the user challenges a claim, the system backs it up with its strongest datum, e.g. "The person likes high-energy music." When the user indicates not understanding how a datum supports a claim, the system provides a warrant, e.g. "Statistically, people that like high-energy music are more likely to be introverted". Importantly, such warrants faithfully reflect the actual reasoning process of the model and are not post-hoc approximations. The communicative capabilities are examplified by the following dialogue between user (U) and assistant (A):
A:
I think this person is introverted.
Why?
A:
The person likes high-energy music. Music heard by the person has a higher average score for energy than music in general.
OK, and does the fact that the person likes non-danceable music support the assessment that the person is introverted? A:
Yes.
For natural-language understanding, the system uses GPT-4 [12] to parse user utterances into a formal language for dialogue moves, while templates are used for natural-language generation. Note that while the prototype supports a particular domain (estimation of personality trait from music preferences), both the method for extracting argument structure and the dialogue system are domain-independent and can therefore be applied to any domain of choice.
The present work can be situated in the context of "conversational explainable AI", i.e. formalization, implementation and evaluation of systems that can explain AI predictions in a natural-language dialogue between system and human (see e.g. [13,14,15,16,17,18,19]). Typically, previous approaches do not support argumentation of the kind discussed here. For example, the system TalkToModel [18] uses explanation methods such as LIME and SHAP to explain specific predictions by listing features deemed important; for the current domain, it would amount to a phrase such as "the top 2 most important features are (1) energy (2) danceability". It can be noted that such an explanation provides neither data (whether the person likes how-or low-energy music, etc.) or warrants (how preferences for different aspects of music correlate with extraversion).
This person has listened to...
To Satellite
The B Foundation Bloo Bucket
The B Foundation Your Eyes (Realggae)
The B Foundation Be Alright >10 times ...and more generally:
The bars show properties of the music that the person has listened to, compared to the population at large (dotted blue lines).
I think this person is introverted.
You OK, why?
The person likes high-energy music.
Hmm I don't understand Assistant Statistically, people that like high-energy music are more likely to be introverted. As for dialogue systems with argumentative capabilities, Breitholtz [23] presented a formal account of how claims can be motivated with enthymemes, which, in Tolmin's framework, corresponds to backing claims with data; Maraev et al. [20] later implemented a working prototype on this basis. In contrast to previous work, the approach presented in this paper enables arguments for ML predictions, in the form of both data and warrants, to be communicated by the system.
We have argued that black-box AI systems cannot generate warrants to support their predictions, even in tandem with popular explainability methods such as LIME or SHAP. However, a human user may still identify or produce a warrant to fill the gap. In fact, implicit premises are ubiquitous in human communication and rarely need to be made explicit. 4 Someone says "It's cold in here so let's close the window" and you immediately understand the warrant: closing the window is likely to increase the indoor temperature (assuming that it's colder outside than inside). Since the listener can identify a warrant that makes the speaker's utterance comprehensible, no warrant needs to be verbalized. From this perspective, a lack of warrant-production in human-AI collaboration does not necessarily need to be a problem. But if the purpose of the AI is to enable improved human decision-making [24], one cannot assume that an AI always "reasons" in similar ways as humans. Arguably, potential differences in reasoning between AI and human is precisely the reason why explanations and arguments are needed.
As shown in previous sections, at least some interpretable models afford argumentation. However, to our best knowledge there is no empirical data that supports the hypothesis that argumentation of the kind discussed here benefits human-AI collaboration. (A survey study found that decision-makers prefer interactive explanations in the form of natural language dialogue [25], but did not specifically investigate argumentation.) In fact, previous work suggests that explanations can cause human over-reliance on AI [26] or have no effect on accuracy [27]. However, as far as we can tell, previously evaluated human-AI interactions have not involved argumentative AI systems. We propose two mechanisms through which argumentation might benefit hybrid human-machine decision-making. First, warrants may enable users to assess whether claims are supported by reasonable generalizations. If the system argues that liking music with high energy makes it more likely that one is introverted, and the user finds this correlation generally questionable, then the user can take this into account when assessing the reliablity of the claim that the generalization is intended to support. Second, warrants can potentially make it easier to combine an AI's assessment with the user's own judgement about the case at hand. If the system supports its claim with a statistical generalization, the user can assess to what extent the generalization seems relevant for the case at hand. Sure, you may reason, people that like music with high energy may be more introverted in general, but in this case you know exactly what kind of high-energy music the person listens to, and you don't associate this music with introversion. To the extent that users successfully assess the relevance of the system's generalizations, decision-making accuracy can be improved compared to a scenario without argumentation. In future work, it would be useful to empirically study how an argumentative AI communicator affects human decision patterns in comparison with a non-argumentative interface, and thereby generate data to either support or contradict our claim that argumentation is crucial not only in human communication, but also in communication between humans and AI.
This work was supported by the Swedish Research Council (VR) grant 2014-39 for the establishment of the Centre for Linguistic Theory and Studies in Probability (CLASP) at the University of Gothenburg.