Argumentation Theory provides a very powerful set of principles, ideas and models. Yet, in this paper we will show that its fundamental principles unsatisfactorily explain real-world human argumentation and should be adapted. We will present an extensive empirical study on the incompatibility of abstract argumentation and human argumentative behavior, followed by practical expansion of existing models.
Argumentation Theory has developed rapidly
since Dung’s seminal work
All of these models and semantics try to
provide a normative approach to argumentation, i.e,
how argumentation should work from a logical
standard. From a descriptive point of view, the
study of
In order to further develop argumentative
applications and agents, we conducted a novel
empirical study, with hundreds of human subjects,
showing the incompatibility between some of the
fundamental ideas, stated above, and human
argumentation. In an attempt to mimic and understand
the human argumentative process, these
inconsistencies, which appear even in the weakest
argumentative requirements as conflict freedom, pose
a large concern for theoreticians and practitioners
alike. Our findings indicate that the fundamental
notions are not good predictive features of
people’s actions. A possible solution is also presented
which provided better results in explaining
people’s arguments than the existing theory. This
solution, which we call Relevance, captures a
perceptual distance between arguments. That is, how
one argument affects another and how this affect
is comprehended by a reasoner. Relevance also
holds a predicatory value as shown in recent work
This article’s main contribution is in showing that the Argumentation Theory has difficulties in explaining a big part of the human argumentative behavior, in an extensive human study. Secondly, the proposed notion of relevance could in turn provide the argumentation community with an additional tool to investigate the existing theory and 2
Argumentation is the process of supporting claims
with grounds and defending them against attacks.
Without explicitly specifying the underlying
language (natural language, first order logic. . . ),
argument structure or attack/support relations, Dung
has designed an abstract argumentation framework
Definition 1. A Dungian Argumentation Framework (AF) is a pair < A; R >, where A is a set of arguments and R is an attack relation over A A. Conflict-Free: A set of arguments S is conflictfree if there are no arguments a and b in S such that aRb holds.
Acceptable: An argument a 2 A is considered acceptable w.r.t a set of arguments S iff 8b:bRa ! 9c 2 S:cRb.
Admissible: A set S is considered admissible iff it is conflict-free, and each argument in S is acceptable with respect to S.
Dung also defined several semantics by which, given an AF , one can derive the sets of arguments that should be considered Justified (to some extent). These sets are called Extensions. The different extenstions capture different notions of justification where some are more strict than others. Definition 2. An extension S A is a set of arguments that satisfies some rules of reasoning. Complete Extension: E is a complete extension of A iff it is an admissible set and every acceptable argument with respect to E belongs to E. Preferred Extension: E is a preferred-extension in A iff it is a maximal (with respect to set inclusion) admissible set of arguments.
Stable Extension: E is a stable-extension in A iff it is a conflict-free set that attacks every argument that does not belong in E. Formally, 8a 2 AnE; 9b 2 S such that bRa.
Grounded Extension: E is the (unique) grounded extension of A iff it is the smallest element (with respect to the inclusion) among the complete extensions of A.
Definition 3. Similar to the attack relation R, one can consider a separate relation S which indicates
Support
Let us consider the following example;
During a discussion between reporters, R1 and R2, about the publication of information I concerning person X, the following arguments are presented: R1: I is important information, thus we must publish it.
R2: I concerns the person X, where X is a private person and we cannot publish information about a private person without his consent.
If you were R1, what would you say next? A. X is a minister, so X is a public person, not a private person.
B. X has resigned, so X is no longer a minister. C. His resignation has been refused by the chief of the government.
D. This piece is exclusive to us; If we publish it we can attain a great deal of appreciation from our readers.
See Figure 1 for a graphical representation.
In this example, all mentioned semantics agree on a single (unique) extension which consists of all arguments except ”Resigned” (option B) and ”Private Person” (R2’s argument). Thus, all arguments except ”Resigned” and ”Private person” should be considered Justified, regardless of the choice of semantics.
Argumentation Theory consists of many more ideas and notions, yet the very fundamental ones stated above are the focus of this work. 3
To get a deeper understanding of the relations
between people’s behaviour in argumentation and
the stated notions, we used real argumentative
conversations from Penn Treebank Corpus (1995)
Every conversation provided us with 2 argument sets A1 and A2, both subsets of A. We tested every Ai (i = 1; 2) such that jAij 3 in order to avoid almost completely trivial sets.
Participants were not expected to be aware of
all arguments in the BAF, as they were not
presented to them. Thus, in testing the Admissibility
of Ai and whether Ai is a part of some Extension,
we examined both the original BAF and the
restricted BAF induced by A1 [ A2. That is, the
argumentation framework in which A = A1 [ A2
and the attack and support relations are defined
over A1 [ A2 A1 [ A2, denoted as AF#A1[A2 .
3.1
The first property we tested was Conflict-Freedom,
which is probably the weakest requirement of a
set of arguments. We had anticipated that all Ai
would have this property, yet only 78% of the
deliberants used a conflict-free set Ai. Namely, that
22% of the deliberants used at least 2
conflicting arguments, i.e, one attacks the other. From
a purely logical point of view, the use of
conflicting arguments is very grating. Yet, we know
that some people try to portray themselves as
balanced and unbiased, and as such use
contradictory arguments to show that they can consider
both ends of the argument and can act as good
arbitrators. When we examined Acceptability, we
tested if every argument a 2 Ai is acceptable w.r.t
a . We found that 58% of the deliberants
Ai n f g
followed this rule. Admissibility was tested
according to both the original framework and the
restricted framework. Merely 28% of the Ais used
are considered admissible w.r.t the original
framework, while more than 49% qualify when
considering the restricted BAF. We can see that people
usually do not make the extra effort to ensure that
their argument-set is admissible. A possible
explanation can be values (norms and morals), as
described in
To strengthen our findings we performed yet another experiment. We tested the notions in a controlled and structured environment, where the participant is aware of all arguments in the framework. 4
We collected 6 fictional scenarios, based on known
argumentative examples from the literature
Two groups of subjects took part in this study; the first consisted of 64 US citizens, all of whom are workers of Amazon Mechanical Turk, ages ranging from 19 to 69 (mean=38, s.d=13.7) with varying demographics. The second consisted of 78 computer science B.Sc. students from Bar-Ilan University (Israel), ages ranging from 18 to 37 (mean=25, s.d=3.7) with similar demographics.
Each subject was presented with the 6 scenarios. Each scenario was presented in a short textual dialog between 2 participants, similar to the journalists’ example above. The subject was instructed to place himself in one of the deliberants’ roles, given the partial conversation, and to choose the next argument he would use from the four available arguments. We instructed the subject to consider only the arguments in the dialog and the proposed ones, and refrain from assuming any other information or possible arguments in the dialog’s context.
The following example, based on
A couple is discussing whether or not to buy an SUV.
Spouse number 1 (S1): ”We should buy an SUV; it’s the right choice for us”.
Spouse number 2 (S2): ”But we can’t afford an SUV, it’s too expensive”.
The participant was then asked to put himself in S1’s shoes and choose the next argument to use in the conversation. The options were: A. ”Good car loan programs are available from a bank”, B. ”The interest rates on car loans will be high”’, C. ”SUVs are very safe, safety is very important to us”, D. ”There are high taxes on SUVs”. See Figure 2 for a graphical representation of the aforementioned framework.
The distribution of selections in the above
example was as follows; A.35%, B.24%, C.8%, D.
33%. There is only one (unique) extension in this
scenario which includes ”High interest” and ”high
taxes”. Especially when considering ”Taking out a
loan”, it should be considered overruled
(unjustified/invalid), or at least very weak, as it is attacked
by an undisputed argument. As we can see, only
slightly over half of the subjects choose an
argument from the extension, i.e, a somewhat Justified
argument.
The distribution of selections, in all scenarios,
suggests that there could be different factors in play,
which differ from one subject to another. Thus,
there is no decisive answer to what a person would
say next. Unfortunately, testing Conflict Freedom
and Admissibility is inapplicable here. None of
the subjects was offered an argument that conflicts
with its previous one and could not choose more
than one argument to construct an admissible set.
When examining Extensions, all scenarios which
were presented to the subject are Well Founded
(that is to say, there exists no infinite sequence
a0; a1; : : : ; an; : : : such that 8i:(ai; ai+1) 2 R).
As such, all mentioned semantics coincide - only
one extension is Grounded, Stable and Preferred.
Of the 6 scenarios, 5 had suggested 2 justified
arguments and 2 overruled arguments (arguments
which are not part of any extension) to the
subject. In these 5 scenarios, 67.3% of the time a
justified argument was selected (on average). This
result is disappointing since 50% is achieved by
randomly selecting arguments. As for Support,
49.4% of the arguments selected were supporting
arguments, i.e, arguments which do not attack any
other argument in the framework. Even more
interesting is that 80% of the time people chose
(directly or indirectly) an argument supporting their
first argument. This phenomenon can be regarded
as a Confirmation Bias, which is recorded in many
fields
We had anticipated that in finite and simple
argumentative frameworks people would naturally
choose the ”right” arguments, yet we again see
that the argumentative principals unsatisfactorily
explain people’s argumentative selections. This is
not a complete surprise, since we have many
examples in the literature where people do not
adhere to the optimal, monolithic strategies that can
be derived analytically
We have shown here, in two separate experiments, that a similar phenomenon occurs in the context of argumentation - people do not choose ”ideal” arguments according to the Argumentation Theory. 5
It is well known that human cognition is limited, as
seen in many examples in
Definition 4. Let a,b be arguments in some AF . Rel : A ! P (A) is a personal relevance function which given argument a 2 A (for evaluation) returns a set of arguments A0 A which are, given the reasoner’s cognitive limitations and knowledge, relevant to a. Using Rel, we can distinguish between relevant and irrelevant arguments w.r.t a given argument, yet we gain additional strength in incorporating the reasoner’s limitation and biases.
We denote the restriction of AF to arguments relevant to a as AF #Rel(a) < A0; R0 > where A0 = Rel(a) and R0 = A0 A0 \ R.
On AF #Rel(a) one can deploy any semantic of choice.
The simplest way to instantiate the Rel is
Rel( ) = A, meaning that all arguments in the
AF are relevant to the given argument. This
instantiation is the way the classic frameworks
address the reasoner’s limitations, simply by saying
– there are none. As shown in
We suggest the following instantiation, which we examined empirically.
Definition 5. Let D(a; b) be a distance function, which given arguments a; b returns the directed distance from argument a to b in AF ’s graph.
Given a distance measurement D we can define an edge-relevance function as follows: Definition 6. RelD(a) = fbjD(b; a) k is a non-negative constant. kg where
Naturally, when setting k to 0, every argument a is considered justified in AF#RelD(a) (under any semantics). k can be thought of as a depth limitation for the search algorithm used by the reasoner. Of course, if k = 1, AF#RelD(a)= fAll affecting arguments on ag. 5.1
We used several D functions in our work on
predicting arguments given a partial conversation
Relevance provides a way to rationally justify
every argument within an AF to some extent.
Unlike VAF
We presented an empirical study, with over 400 human subjects and 250 annotated dialogs. Our results, based on both free-form human deliberations and structured experiments, show that the fundamental principles of Argumentation Theory cannot explain a large part of the human argumentative behavior. Thus, Argumentation Theory, as it stands, should not be assumed to have descriptive or predicatory qualities when it is implemented with people.
Our relevance notion provides a new way to rationalize arguments without prior knowledge about the reasoner. Relevance, as well as other psychological and social aspects, should be explored to better fit the Argumentation Theory to human behavior. This required step is crucial to the integration of argumentation in different human domains.