Argumentation theory examines how conclusions are derived or refuted through logical reasoning, playing a crucial role in human interaction and decision-making. In artificial intelligence, computational argumentation leverages formal models to aid in decision-making processes. This paper investigates the influence of argument content (specifically the naturalness bias) and graphical representation on participants' adherence to the simple principles of reinstatement and void precedence principles. We conducted experiments testing three hypotheses related to participants' rationality in evaluating arguments with and without graphical aids and bias-provoking content. Contrary to our expectations, neither the presence of graphical representations nor the type of content significantly impacted participants' compliance with the reinstatement principle. Additionally, the graph did not enhance understanding, suggesting the need for instructional aids. Our findings challenge previous studies and highlight the complexity of factors influencing argument evaluation.
Argumentation theory is the interdisciplinary study of how conclusions can be supported or undermined by premises through logical reasoning. It is an essential part of our daily human reasoning, interaction and communication with one another. Argumentation is often used in decision-making problems, resolving conflicts of opinion through negotiation and deliberation processes, and influencing the thoughts of others through persuasion. The complex and dynamic nature of argumentation is studied in various disciplines.
It stands as an important domain of Artificial Intelligence (AI), especially in knowledge
representation and reasoning. Indeed, the use of AI allows for the development of computational
models for the exchange of arguments among various agents, facilitating the derivation of valid
conclusions from incomplete, inconsistent or conflicting information. Within AI, computational
argumentation focuses on creating formal models that support decision-making through the
construction and evaluation of arguments. Many works in computational arguments are based
on Dung’s seminal work [
On a given abstract argumentation framework, one can apply many diferent semantics to
compute sets of acceptable arguments (conflict-free, admissible, . . . ) [
A recent work by Vesic, Yun, and Teovanovic [
The latter result opens up several important questions that were not studied before. In what way does the content of the arguments (e.g., language, bias-provoking content) afect the principle compliance? Can graphical representations consistently serve as a prescriptive tool for enhancing rationality in individuals?
In this exploratory paper, we will focus on a restricted setting. We will study whether participants comply with the simple reinstatement principle and how this is afected by the naturalness bias and the graphical representation.
On the one hand, the simple reinstatement principle states that in a configuration with three arguments , , and such that attacks and attacks (see Figure 1), roughly speaking “both and must be stronger than ”. On the other hand, the naturalness bias (also known as appeal-to-nature) is a cognitive bias where people have a preference for things perceived as “natural" over those perceived as “unnatural" or artificial. This bias can influence attitudes and decisions in various domains, including food, medicine, or lifestyle choices, among others.
To experimentally check the impact of the content (e.g. bias-provoking text) and the graph, we set up an experiment featuring three hypotheses. Firstly, we expected participants to behave more rationally in the absence of the naturalness bias, regardless of graph presence. Next, we predicted that the graph would make participants more rational (as shown in previous studies) and we also expected that the efect of the graph would be stronger in the presence of naturalness bias than in its absence.
This paper is structured as follows. In Section 2, we recall the computational argumentation background on extension and ranking-based semantics needed to motivate the principles. In Section 3, we formulate our aims and hypothesis with respect to the principles, the graph, and the naturalness bias. In Section 4, we describe the design of our study, the sampling plan, and the instruments. Lastly, we present our results in Section 5 and conclude in Section 6.
We start this section by recalling the definition of an argumentation framework as defined by
Dung in his seminal paper [
Definition 1 (Argumentation framework). An argumentation framework is a pair ℱ = (, ), where is a finite set of arguments and ⊆ × is a set of binary attacks between arguments. The set of attackers of ∈ is () = { ∈ | (, ) ∈ }.
In the following sub-sections, we explain what is considered as rational with respect to extension-based semantics and ranking-based semantics.
Given an argumentation framework ℱ = (, ), we say that a set ⊆ is conflict-free if there is no , ∈ such that (, ) ∈ . A set ⊆ defends ∈ if for every ∈ such that (, ) ∈ , there exists ∈ such that (, ) ∈ . A set ⊆ is admissible if it is conflict-free and defends every argument in . A set is preferred if it is a maximal (for set inclusion) admissible set.
The preferred semantics is the function that computes all the preferred sets from an argumentation framework. There are many other acceptability semantics, which we do not introduce in this paper as the details are not of practical interest for the rest of our cognitive study1.
Example 1. In Figure 1, there are three admissible sets, which are ∅, {}, and {, }, but there is only one preferred set which is {, }.
In Example 1, the preferred extension shows that both and are better than (as and
are in the preferred set but is not). In this case, we can consider that and should
be equally acceptable. Thus, we define the extension-based simple reinstatement, inspired by
[
Definition 2 (Extension-based simple reinstatement). In a configuration with three arguments , , and such that attacks and attacks , the extension-based simple reinstatement is satisfied if is equally strong as , is stronger than and is stronger than .
While extension can extract sets of acceptable arguments, an orthogonal approach consists in ranking-arguments from the stronger to the weakest one.
Another family of argumentation semantics is called ranking-based. They do not calculate extensions; instead, they rank the arguments from the strongest to the weakest one. In this paper, we focus on a particular family of ranking-based semantics, called gradual semantics, which associates to each argument, a number from 0 (weakest) to 1 (strongest). Those scores naturally induce an order on the set of arguments.
Definition 3 (Gradual semantics). A gradual semantics is a function that takes as input
any ℱ = (, ) and returns a function ℱ : → [
For the rest of our study, we do not need particular semantics. However, just for the sake of
illustration, we introduce one semantics, called h-categorizer [
Definition 4 ( ℎ-categorizer). The ℎ-categorizer semantics is a ranking-based semantics such that for every ℱ = (, ), for every argument ∈ we have ℎ () = ℱ
1 1 + ∑︀∈() ℎ () ℱ
It was shown [
Many principles were defined for characterising the behaviour of ranking-based semantics. For the purpose of this paper, we do not need to recall them all. We just formalize the rankingbased simple reinstatement and the void precedence principles. Definition 5 (Ranking-based simple reinstatement). In a configuration with three arguments , , and such that attacks and attacks , the ranking-based simple reinstatement is satisfied if is stronger than and is stronger than .
We consider that both extension-based and ranking-based are valid and rational points of view. In the rest of the paper, we allow the rational agent to act based on either of them. Hence, we will say that the reinstatement is satisfied if either ranking-based simple reinstatement or extension-based simple reinstatement is satisfied.
Definition 6 (Simple reinstatement). In a configuration with three arguments , , and such that attacks and attacks , the simple reinstatement is satisfied if is equal or stronger than and is stronger than .
Void precedence states that a non-attacked argument is stronger than an attacked one.
Definition 7 (Void precedence [
Now that we have defined what is considered rational in abstract argumentation, we formally describe our aims and hypotheses.
In the current study, we examined if people comply with normative principles more readily when presented with arguments with neutral content in comparison to arguments that contain biasprovoking content. To do so, we explored people’s response patterns in a simple reinstatement scenario. As an example of bias-provoking content, we relied on the naturalness bias i.e., the tendency to prefer natural things over artificial ones, all other things being equal. Furthermore, we aimed to replicate previous findings that graphical representation of arguments leads to higher compliance with normative principles i.e., the presence of an argumentation graph facilitates rational inference.
We expected participants to comply more with normative principles in the neutral condition than in the naturalness bias condition, regardless of graph presence (H1). Moreover, we expected participants to comply more with normative principles in the graph condition relative to the no-graph condition, regardless of content (H2). Finally, we expected an interaction between the two factors, i.e., that the efect of the graph will be stronger in the naturalness bias condition relative to the neutral condition (H3).
On an exploratory level, we also checked whether people tend to follow extension-based or ranking-based argumentation semantics more, depending on the rating of diferent arguments (the non-attacked one and the reinstated one). The extension-based semantics predict that both arguments will have the same acceptability level, whereas most of the ranking-based semantics predict that the non-attacked one will be stronger.
The materials and data to reproduce the findings of this study can be found on the project OSF page: https://osf.io/kce7q/.
We employed a 2x2 design with two independent factors: content (neutral vs. bias-provoking) and graph (present vs. absent), resulting in four experimental groups of participants.
The sample size was based on an a priori power analysis. Since we considered several possible efects, we opted to base the power analysis on the smallest efect size of interest - i.e., a small interaction efect size (Cohen’s = .10). For power .80, the apriori analysis indicated we would need = 197 per group (788 in total) to detect this efect size. For power .90, for the same efect size, we would need = 263 per group (1052 in total). For further details on the power analysis, see preregistration - https://aspredicted.org/9T9_NG4.
As per preregistration, we included an attention check (“Please choose Completely agree to indicate you are paying attention”), and all participants who failed the attention check were automatically excluded from the sample and did not count towards the final sample size.
The final sample size consisted of a total of = 1048 UK and US Prolific participants. All respondents were compensated for their participation using the standard Prolific 9 £/hour rate. Most participants were from the UK (85.6%), and 61.8% were female, with an average age of = 41.45 ( = 13.71)2. Due to a technical error in the questionnaire setup, there was an unequal distribution between the four experimental groups (116 for neutral content without a graph, 398 for neutral content with a graph, 400 for bias-provoking content without a graph, and 133 for bias-provoking content with a graph).
The questionnaire was administered online via Prolific and hosted on the SoSci Survey
platform [
2Based on the data from = 1046 participants, since two participants’ demographic Prolific data could not be matched.
Consider the arguments below. • Argument A: It doesn’t matter if you drink natural or lab-produced water. • Argument B: Natural things are healthier than lab-produced ones, so you should drink natural water.
• Argument C: They are chemically identical, so they have the same impact on your health.
Consider the arguments below. • Argument A: It’s not going to rain today. • Argument B: But look at all these clouds, it will rain.
• Argument C: Those clouds are Cumulus, hence they do not produce rain.
As mentioned in Section 4.2, we had four experimental groups of participants (neutral vs. bias-provoking / graph vs. no-graph).
Each participant was first shown three arguments in their textual forms. Depending on their group (neutral vs. bias-provoking), the textual content was diferent (see Figures 2 and 3). We can observe that in both cases, argument B attacks argument A which consists only of a claim, argument C attacks argument B since C attacks B’ s premise.
The textual content of the arguments was inspired by structured argumentation which allows
for constructing arguments from a knowledge base using a formal language. In this context,
arguments are characterized as structured because they reveal their underlying premises and
conclusions clearly, the connection between them is formally established, and attacks among
them are formally defined. There exist several approaches in structured argumentation for
formalizing arguments such as the ASPIC+ framework [
Only for the participants in the graph groups, we displayed the graphical visualisation of the arguments and attacks (see Figure 1).
Lastly, the participants were asked to assess the strength of each argument using a 5-point Likert scale from 1 (Weak) to 5 (Strong), with 3 being “Neutral”.
Overall, participants complied with simple reinstatement to a low degree (see Figure 4), with the percentage of participants who followed these principles in either of the groups ranging from 10.3% to 16.5%.
We conducted a logistic regression analysis to investigate how well the two factors (content and graph) and their interaction predict compliance with simple reinstatement. The goodnessof-fit of the logistic regression model was assessed using a likelihood ratio test, which yielded a non-significant chi-square statistic, 2(3) = 2.96, = .398. Additionally, the 2 value indicated that the model accounts for only 0.3% of the variability in compliance with simple reinstatement. This suggests that the model with argument content and graphical representation did not provide a significantly better fit to the data than a model without them, as shown in Table 1.
Regarding void precedence, the participants in our sample followed this principle to a larger degree (see Figure 5) - the lowest percentage in either of the groups was 44.4% while the highest was 69.8%.
To explore if the two factors and their interaction predict compliance with void precedence, we conducted another logistic regression analysis. The goodness-of-fit of the model was assessed using a likelihood ratio test, which yielded a significant chi-square statistic, 2(3) = 53.66, < Predictor Intercept -3.78 Content 1.14
Graph 1.09 Content * Graph -0.60 .001, and 2 value indicated that the model accounts for 5% of the variability in compliance with void precedence. As shown in Table 2, content influenced rational behavior, while the graph and interaction terms were non-significant. Presenting arguments with naturalness bias-provoking content decreased compliance with void precedence by 71% (() = 0.29, 95% = [0.11, 0.77]).
Further analyses revealed that the efect of content is due to higher ratings of argument ((1057) = 2.66, = .008, = 0.16) and especially lower ratings of argument ((1057) = 7.69, < .001, = 0.47) in the group presented with bias-provoking content, regardless of the presence of graph (see Table 3). Ratings for argument did not difer significantly between bias-provoking and neutral content groups ( = .28).
-2.26 -1.24 -0.36 0.18
First, we expected participants to comply more with normative principles in the neutral condition than in the naturalness bias condition, regardless of graph presence (H1). For this hypothesis, our results are mixed. The experiment showed that the change in content (i.e. neutral vs biased) did not impact the degree of compliance with the simple reinstatement principle. However, the content influenced the degree to which participants complied with the void precedence principle. Namely, in the presence of arguments incorporating naturalness bias, the ratings of the non-attacked argument were very low while the ratings of the non-defended argument were high (probably due to the naturalness bias increasing participants’ ratings of ).
Second, we expected participants to comply more with normative principles in the graph
condition relative to the no-graph condition, regardless of content (H2). This hypothesis was
disconfirmed - our experiment showed that the graph did not provide a significantly better fit to
the data. This contrasts the recent results [
Third, we expected an interaction between the two factors, i.e., that the efect of the graph will be stronger in the naturalness bias condition relative to the neutral condition (H3). Contrary to our hypothesis, the experiment showed that the interaction efect was not significant in our model.
In our experiments, compliance with the reinstatement principle was very low (less than 20%).
Our results difer from the existing empirical work on reinstatement [
Note that according to the average scores of arguments in our experiment, is stronger than and is stronger than . One of the possible explanations for this is that the argument is just a statement and contains no justification. Argument has a general justification (e.g., there are clouds so it will rain) and might be seen as more acceptable for this reason. Finally the argument has the strongest hypothesis that fully justifies its conclusion and might seem dificult to attack (e.g., the cumulus clouds do not produce rain).
Another reason why compliance with simple reinstatement might be low is that participants’ intuition about the notion of defense diverges significantly from that of the researchers. More precisely, it might be that the participants misunderstood the meaning of attacks and associated attacks solely with a reduction in argument strength, ignoring the increase in strength brought by reinstatement for argument .
Importantly, several factors limit the generalizability of our findings. Firstly, due to a programming error, the four groups in our study were not equal in size, somewhat reducing statistical power. Next, since participants only responded to one presented scenario, it is dificult to say how they would respond to other scenarios, be it simple reinstatement scenarios with diferent content or more complex scenarios. Finally, we presented all arguments at the same time, which may have an impact on participants’ perception of attacks and, consequently, their ratings of arguments’ strength.
In future work, we plan to introduce the arguments step-by-step (as it was done by Rahwan
et al. [
This work benefited from the support of the project SATTORI in the framework of the Pavle Savic - Hubert Curien funding scheme, and from the support of the project AGGREEY ANR-22CE23-0005 of the French National Research Agency (ANR).