=Paper=
{{Paper
|id=Vol-1341/paper4
|storemode=property
|title=Argumentation Mining on the Web from Information Seeking Perspective
|pdfUrl=https://ceur-ws.org/Vol-1341/paper4.pdf
|volume=Vol-1341
|dblpUrl=https://dblp.org/rec/conf/argnlp/HabernalEG14
}}
==Argumentation Mining on the Web from Information Seeking Perspective==
https://ceur-ws.org/Vol-1341/paper4.pdf
Argumentation Mining on the Web from Information Seeking Perspective
Ivan Habernal†‡ , Judith Eckle-Kohler†‡ , Iryna Gurevych†‡
†
Ubiquitous Knowledge Processing Lab (UKP-TUDA)
Department of Computer Science, Technische Universität Darmstadt
‡
Ubiquitous Knowledge Processing Lab (UKP-DIPF)
German Institute for Educational Research
www.ukp.tu-darmstadt.de
Abstract cannot feasibly process such massive amounts of
data in order to reveal argumentation. Unfortu-
In this paper, we argue that an annota- nately, even current Web technologies (such as
tion scheme for argumentation mining is search engines or opinion mining services) are not
a function of the task requirements and the suitable for such a task. This drives the research
corpus properties. There is no one-size- field to the next challenge – argumentation min-
fits-all argumentation theory to be applied ing on the Web. The abundance of freely available
to realistic data on the Web. In two anno- (yet unstructured, textual) data and possible appli-
tation studies, we experiment with 80 Ger- cations of such tools makes this task very appeal-
man newspaper editorials from the Web ing.
and about one thousand English docu-
Our research into argumentation mining is mo-
ments from forums, comments, and blogs.
tivated by the information seeking perspective.
Our example topics are taken from the
The key sources are discussions (debates) about
educational domain.
controversies (contentions) targeted at a particular
To formalize the problem of annotating topic which is of the user’s interest. The scope is
arguments, in the first case, we apply a not limited to a particular media type as the source
Claim-Premise scheme, and in the second types can range from the on-line newspapers’ ed-
case, we modify Toulmin’s scheme. We itorials to user-generated discourse in social me-
find that the choice of the argument com- dia, such as blogs and forum posts, covering dif-
ponents to be annotated strongly depends ferent aspects of the issues. Understanding posi-
on the register, the length of the document, tions and argumentation in on-line debates helps
and inherently on the literary devices and users to form their opinions on controversial issues
structures used for expressing argumenta- and also fosters personal and group decision mak-
tion. We hope that these findings will fa- ing (Freeley and Steinberg, 2008, p. 9). The main
cilitate the creation of reliably annotated task would be to identify and extract the core ar-
argumentation corpora for a wide range of gumentation (its formal aspects will be discussed
tasks and corpus types and will help to later) and present this new knowledge to users.
bridge the gap between argumentation the- By utilizing argumentation mining methods, users
ories and actual application needs. can be provided with the most relevant informa-
tion (arguments) regarding the controversy under
1 Introduction investigation.
Argumentation mining apparently represents an Although argumentation mining on the Web
emerging field in Natural Language Processing has already been partly outlined in the literature
(NLP) with publications appearing at mainstream (Schneider et al., 2012; Sergeant, 2013), the re-
conferences, such as ACL (Cabrio and Villata, quirements and use-case scenarios differ substan-
2012; Feng and Hirst, 2011; Madnani et al., 2012) tially. Various tasks are being solved, most of them
or COLING (Stab and Gurevych, 2014; Levy et depending on the domain, e.g., product reviews or
al., 2014; Wachsmuth et al., 2014a). In particular, political contentions. As a result, different inter-
there is an increasing need for tools capable of un- pretations of arguments and argumentation have
derstanding argumentation on the large scale, be- been developed in NLP, and therefore, most of
cause in the current information overload, humans the existing researches are not directly adaptable.
News co Noroozi et al., 2013), pragmatics (Xu and Wu,
Comments nt
ro 2014), psychology (Larson et al., 2004), and many
ve
r sy others. Given so many different perspectives on
Forums
investigating argumentation, there is a plethora
of possible interpretations of argumentation min-
Blogs
ing. Thus, finding a common understanding of this
evolving field is a fundamental challenge.
For NLP research, this overwhelming amount
of related works brings many theoretical and prac-
tical issues. In particular, there is no one-size-
Argument
fits-all argumentation theory. Even argumentation
researchers disagree on any widely-accepted ulti-
Figure 1: Schematic overview of argumentation mate concept. For example, Luque (2011) criti-
mining on the Web cizes the major existing approaches in order to es-
tablish a new theory which is later again severely
Morover, not all of the related research works are criticized by other in-field researches (Andone,
tightly connected to argumentation theories (de 2012; Xie, 2012). Given this diversity of perspec-
Moor et al., 2004; Villalba and Saint-Dizier, 2012; tives, NLP research cannot simply adopt one par-
Cabrio et al., 2013b; Llewellyn et al., 2014). How- ticular approach without investigating its theoret-
ever, we feel that it is vital to ground NLP research ical background as well as its suitability for the
in argumentation mining in existing work on argu- particular task.
mentation.
2.1 What we do not tackle
In this article, we will particularly focus on
bridging the gap between argumentation theories Given the breath of argumentation mining just out-
and actual application needs that has not been tar- lined, we would also like to discuss aspects that do
geted in the relevant literature. We will support not fit into our approach to argumentation mining,
our findings by comprehensively surveying exist- namely macro argumentation and evaluation using
ing works and presenting results from two exten- formal frameworks.
sive annotation studies. First, we treat argumentation as a product (mi-
Our main findings and suggestions can be sum- cro argumentation or monological models), not
marized as follows: First, the use-case of any re- as a process (macro argumentation or dialogical
search in argumentation mining must be clearly models). While dialogical models highlight the
stated (i.e., in terms of expected outcomes). Sec- process of argumentation in a dialogue structure,
ond, properties of the data under investigation monological models emphasize the structure of
must be taken into account, given the variety of the argument itself (Bentahar et al., 2010, p. 215).
genres and registers (Biber and Conrad, 2009). Therefore, we examine the relationships between
Third, an appropriate argumentation model must the different components of a given argument,
be chosen according to the requirements. There- not a relationship that can exist between argu-
fore, we claim that it is not possible to formulate ments.1 Exploring how argumentation evolves be-
a single argumentation mining perspective that tween parties in time remains out of our scope.
would be applicable to the Web data in general. Second, we do not tackle any logical reason-
ing, defeasibility of reasoning, or evaluating argu-
2 Relation to Argumentation Theories mentation with formal frameworks in general. Al-
though this is an established field in informal logic
Research on argumentation is widely interdis- (Prakken, 2010; Hunter, 2013; Hunter, 2014),
ciplinary, as it spreads across philosophy and such an approach might not be suitable directly
rhetoric (Aristotle and Kennedy (translator), for Web data as it assumes that argumentation is
1991; Perelman and Olbrechts-Tyteca, 1991; Wal- logical (such a strong assumption cannot be guar-
ton et al., 2008), informal and formal logic
1
(Dung, 1995; Henkemans, 2000; Stoianovici, For further discussion see, e.g., (Blair, 2004; Johnson,
2000; Reed and Walton, 2003) or Micheli (2011) who sum-
2009; Schneider et al., 2013; Hunter, 2013), edu- marizes the distinction between the process (at a pragmatic
cational research (Weinberger and Fischer, 2006; level) and the product (at a more textual level).
anteed). Furthermore, acceptability of arguments pus. Appropriateness of such an approach remains
also touches the fundamental problem of the target questionable. On the one hand, Walton’s argumen-
audience of the argument, as different groups have tation schemes are claimed to be general and do-
different perceptions. Crosswhite et al. (2004) main independent. On the other hand, evidence
point out that “one of the key premises from which from the field shows that schemes might not be
the study of rhetoric proceeds is that influencing the best means for analyzing user-generated argu-
real audiences is not simply a matter of presenting mentation. In examining real-world political ar-
a set of rational, deductive arguments.” gumentation from (Walton, 2005), Walton (2012)
found out that 37.1% of the arguments collected
2.2 Common terminology did not fit any of the fourteen schemes they chose
Let us set up a common terminology. Claim is so they created new schemes ad-hoc. Cabrio et al.
“the conclusion we seek to establish by our argu- (2013a) select five argumentation schemes from
ments” (Freeley and Steinberg, 2008, p. 153) or Walton and map these patterns to discourse rela-
“the assertion put forward publicly for general ac- tion categories in the Penn Discourse TreeBank
ceptance” (Toulmin et al., 1984, p. 29). Premises (PDTB) (Prasad et al., 2008), but later they define
are “connected series of sentences, statements, or two new schemes that they discovered in PDTB.
propositions that are intended to give reasons of These findings confirm that the schemes lack cov-
some kind for the claim” (Freeley and Steinberg, erage for dealing with real argumentation in natu-
2008, p. 3). ral language texts.
3 Related Work 3.2 Previous works on annotation
Table 1 summarizes the previous research on an-
3.1 Opinion mining perspective
notating argumentation. Not only it covers re-
In existing works on argumentation mining of the lated work from the NLP community but also
Web data, the connection is often made to opin- studies from general discourse analysis (Newman
ion mining (Liu, 2012). From the users’ point and Marshall, 1991; Walton, 2012) and road-maps
of view, opinion mining applications reveal what or position papers (Schneider and Wyner, 2012;
people think about something. The key question Peldszus and Stede, 2013a; Sergeant, 2013). The
which brings argumentation on the scene is why heterogeneity of used argumentation models and
do they think so? – in other words, explaining the the domains under investigation demonstrates the
reasons behind opinions. breath of the argumentation mining field. We iden-
Villalba and Saint-Dizier (2012) approach tified the following research gaps.
aspect-based sentiment of product reviews by clas-
sifying discourse relations conveying arguments • Most studies dealing with Web data use
(such as justification, reformulation, illustration, some kind of proprietary model without re-
and others). They build upon Rhetorical Structure lation to any argumentation theory (Bal and
Theory (RST) (Mann and Thompson, 1987) and Saint-Dizier, 2010; Rosenthal and McKe-
argue that rhetorical elements related to explana- own, 2012; Conrad et al., 2012; Schneider
tion behave as argument supports. and Wyner, 2012; Villalba and Saint-Dizier,
For modeling argumentation in social media, 2012; Florou et al., 2013; Sergeant, 2013;
Schneider et al. (2012) suggest using Dung’s Wachsmuth et al., 2014b; Llewellyn et al.,
framework (Dung, 1995) with Walton schemes 2014).
(Walton et al., 2008), but do not provide evidence • Inter-annotation agreement (IAA) that re-
for such a decision. They admit that “It is far flects reliability of the annotated data is either
from clear how an argument [...] can be trans- not reported (Feng and Hirst, 2011; Mochales
formed into a formal argumentation scheme so and Moens, 2011; Walton, 2012; Florou et
that it can be reasoned in an argumentation frame- al., 2013; Villalba and Saint-Dizier, 2012), or
work” (Schneider et al., 2012, p. 22). is not based on a chance-corrected measure
Schneider and Wyner (2012) focus on the prod- (Llewellyn et al., 2014).
uct reviews domain and develops a number of ar-
gumentation schemes (inspired by (Walton et al., This motivates our research into annotating Web
2008)) based on manual inspection of their cor- data relying on a model based on a theoretical
Claim premise and a claim. The simplest way to rep-
restatement = {true, false}
resent the support and attack relations is to attach
labels to adjacent argument components, which in-
Pre-Support Pre-Attack
dicate their argumentative role. The span of argu-
Post-Support Post-Attack ment components is left unspecified, allowing for
premises
argument components spanning a clause or one to
several sentences. Using the six labels claim, re-
Figure 2: Claim-Premise scheme. Note that the re- statement, pre-claim support, post-claim support,
lations (arrows) are only illustrative; they are im- pre-claim attack and post-claim attack, a linear
plicitly encoded in the roles of the particular argu- sequence of non-nested arguments can be repre-
ment components. sented.
While graph structures where nodes stand for
background in argumentation and reporting IAA argument components, and edges for support or
that would confirm suitability of the model and re- attack relations are a more general way to repre-
liability of the annotated data. sent arguments (equivalent to, i.e., (Dung, 1995)
or (Freeman, 1991)), it is unclear which additional
4 Annotating argumentation in Web data
benefits such a more fine-grained annotation of ar-
Up until now, we have used the terms argumenta- guments brings for the annotation of Web docu-
tion and argument in their common meaning with- ments. In a pre-study performed by Kluge (2014),
out any particular formal definition. We will now the possibility to annotate nested arguments turned
elaborate on annotation schemes and discuss their out to be a drawback, rather than an advantage, be-
suitability and reliability for the Web data. cause the inter-annotator agreement dropped con-
siderably.
4.1 Annotation Schemes
Because of the lack of a single general-purpose
argumentation model (cf. discussion in §1), we Suitability of the scheme The main advantage
present here two different schemes.2 Both are built of the Claim-Premises scheme is its simplicity.
upon foundations in argumentation theories, but Therefore, it is particularly suited for annotating
they differ in their granularity, expression power, arguments in long Web documents, such as news
and other properties. articles, editorials or blog posts. Kluge (2014)
found that most documents of these text types con-
4.1.1 Claim-Premises scheme sist of three major parts: an introductory part,
The Claim-Premises scheme is widely used in pre- summarizing the document content in one or two
vious work on argumentation mining, e.g., (Palau paragraphs, the main part, presenting a linear se-
and Moens, 2009; Florou et al., 2013; Peldszus quence of arguments, and an optional concluding
and Stede, 2013b). It defines an argument as con- part summarizing the main arguments.
sisting of a (possibly empty) set of premises and a
The Claim-Premise scheme can be used to pro-
single claim; premises either support or attack the
vide an overview of the claims and their sup-
claim (Besnard and Hunter, 2008). We adopted
porting or attacking premises presented in a long
this general scheme for the purpose of annotating
Web document. From an information seeking per-
arguments in long Web documents (Kluge, 2014).
spective, arguments could be clustered by similar
According to this adopted version of the scheme,
claims or similar premises, and then ranked in the
claims, restatements and premises are subsumed
context of a specific information need by a user.
under the term argument component; a restate-
In a similar way, this scheme could be used for
ment of a claim is also considered as claim and is
automatic summarization.
part of the same argument. The scheme is depicted
in Figure 2. However, the Claim-Premises scheme does not
Premises either support or attack a claim, i.e., allow to distinguish between different kinds of
there is a support or attack relation between each premises supporting the claim. Hence, fine-
2
grained distinctions of premises into specific fac-
An exhaustive overview of various argumentation mod-
els, their taxonomy, and properties can be found in (Bentahar tual evidence versus any kind of common ground
et al., 2010). can not be captured.
Source Arg. Model Domain Size IAA
Newman and Marshall Toulmin legal domain (Peo- qualitative N/A
(1991) ple vs. Carney, U.S.
Supreme Court)
Bal and Saint-Dizier proprietary socio-political newspa- 56 documents Cohen’s κ
(2010) per editorials (0.80)
Feng and Hirst (2011) Walton legal domain (Aracu- ≈ 400 arguments not reported
(top 5 schemes) raria corpus, 61% sub- claimed to be small
set annotated with Wal-
ton scheme)
Georgila et al. (2011) proprietary general discussions 21 dialogues Krippendorf’s α
(negotiations between (0.37-0.56)
florists)
Mochales and Moens Claim-Premise legal domain (Aracu- 641 documents not reported
(2011) based on Freeman raria corpus, European w/ 641 arguments
Human Rights Council) (Aracuraria)
67 documents w/
257 arguments
(EHRC)
Walton (2012) Walton political argumentation 256 arguments not reported
(14 schemes)
Rosenthal and McKe- opinionated blogposts, Wikipedia 4000 sentences Cohen’s κ
own (2012) claim, sentence discussions (0.50-0.57)
level
Conrad et al. (2012) proprietary editorials and blogpost 84 documents Cohen’s κ
(spans of arguing about Obama Care (0.68)
subjectivity) on 10 documents
Schneider and Wyner proprietary, ar- camera reviews N/A N/A
(2012) gumentation (proposal/position
schemes paper)
Schneider et al. (2012) Dung + Walton unspecified social me- N/A N/A
dia (proposal/position
paper)
Villalba and Saint- proprietary, RST hotel reviews, hi-fi 50 documents not reported
Dizier (2012) products, political
campaign
Peldszus and Stede Freeman + RST Potsdam Commentary N/A N/A
(2013a) Corpus (proposal/position
paper)
Florou et al. (2013) none public policy making 69 argumentative not reported
segments / 322
non-argumentative
segments
Peldszus and Stede based on Freeman not reported, artificial 23 short documents Fleiss’ κ
(2013b) documents created for multiple results
the study
Sergeant (2013) N/A Car Review Corpus N/A N/A
(CRC) (proposal/position
paper)
Wachsmuth et al. none hotel reviews 2100 reviews Fleiss’ κ
(2014b) (0.67)
Llewellyn et al. (2014) proprietary, no ar- Riot Twitter Corpus 7729 tweets only percentage
gumentation the- agreement reported
ory
Stab and Gurevych Claim-Premise student essays 90 documents Krippendorf’s αU
(2014) based on Freeman (0.72)
Krippendorf’s α
(0.81)
Table 1: Previous works on annotating argumentation. IAA = Inter-annotation agreement; N/A = not
applicable.
Backing
Backing ponents (roles). “By identifying these roles, we
can present the arguments in a more readily un-
Grounds Claim
Grounds implicit = {true, false} derstandable fashion, and also identify the various
ways in which the argument may be accepted or
Rebuttal
attacked” (Bentahar et al., 2010, p. 216).
Refutation
Refutation Rebuttal
Toulmin’s model, as a general framework for
modeling static monological argumentation (Ben-
Figure 3: Extended Toulmin’s scheme. Note that
tahar et al., 2010), has been used in works on
the relations (arrows) are only illustrative; they are
annotating argumentative discourse (Newman and
implicitly encoded in the roles of the particular ar-
Marshall, 1991; Chambliss, 1995; Simosi, 2003;
gument components.
Weinberger and Fischer, 2006). However, its com-
plexity and the fact that the description of the com-
4.1.2 Toulmin’s scheme ponents is informal and sometimes ambiguous,
The Toulmin’s model (Toulmin, 1958) is a con- poses challenges for an application of the model
ceptual model of argumentation, in which differ- on real-world data, especially user-generated dis-
ent components play distinct roles. In the original course on the Web. Moreover, some of the com-
form, it consists of six components: claim, data ponents are usually left implicit in argumentation,
(grounds), warrant, backing, qualifier, and rebut- such as the warrant or even the claim (Newman
tal. and Marshall, 1991).
The roles of claim and grounds correspond
to the definitions introduced earlier (claim and 5 Preliminary results of annotation
premises, respectively). The role of warrant is to studies
justify a logical inference from grounds to claim.
In order to examine the proposed approaches, we
To assure the trustworthiness of the warrant, back-
conducted two extensive independent annotation
ing provides further set of information. Qualifier
studies. The central controversial topics were re-
limits the degree of certainty under which the ar-
lated to education. One distinguishing feature
gument should be accepted and rebuttal presents
of educational topics is their breadth, as they at-
a situation in which the claim might be defeated.
tract researchers, practitioners, parents, or policy-
For examples of arguments based on Toulmin’s
makers. Since the detailed studies are being pub-
original model see, e.g., (Freeley and Steinberg,
lished elsewhere, we summarize only the main re-
2008, Chap. 8).
sults and outcomes in this paper.
Based on our experiments during annotation
In the first study, we used the Claim-Premises
pre-studies, we propose an extension of the Toul-
scheme for annotating a dataset of web documents
min’s model by means of (1) omitting the qualifier
consisting of 80 documents from six current top-
for stating modality, as people usually do not state
ics related to the German educational system (e.g.,
the degree of cogency, (2) omitting the warrant as
mainstreaming, staying down at school), which is
reasoning for justifying the move from grounds to
described in (Kluge, 2014). The dataset contains
claims is not usually explained, (3) extending the
(newspaper) articles, blog posts, and interviews.
role of backing so it provides additional set of in-
It was created by Vovk (2013) who manually se-
formation to back-up the argument as a whole but
lected documents obtained from a focused crawler
is not directly bound to the claim as the grounds
and the top 100 search engine hits (per topic).
are, and (4) adding refutation which attacks the
In the second study, the annotation was split
rebuttal (attacking the attack). The scheme is de-
into two stages. In the first stage, we anno-
picted in Figure 3.
tated 990 English comments to articles and fo-
Suitability of the scheme As pointed out by rums posts with their argumentativeness (persua-
Bentahar et al. (2010), many argumentation sys- siveness). The source sites were identified using
tems make no distinction between their premises, a standard search engine and the content was ex-
despite the fact that in arguments expressed in nat- tracted manually; we chose the documents ran-
ural language we can typically observe premises domly without any pre-filtering. In the second
playing different roles. Toulmins’ scheme allows stage, we applied the extended Toulmin’s scheme
such a distinction using the set of different com- on 294 argumentative English comments to arti-
cles and forums posts and 57 English newspa- Argument Comments, Blogs Articles
per editorials and blog posts. The topics cover, Component Forums
e.g., mainstreaming,3 single-sex schools, or home- Claim 0.57 0.17 0.23
schooling, among others. Grounds 0.64 0.32 0.11
Backing 0.41 -0.16 0.28
Measuring inter-annotator agreement For
Rebuttal 0.33 -0.02 0.00
any real large-scale annotation attempt, measuring
Refutation 0.06 0.35 0.00
inter-annotator agreement (IAA) is crucial in
order to estimate the reliability of annotations Table 2: IAA scores (Krippendorf’s αU ) from an-
and the feasibility of the task itself. Both anno- notations using the Toulmin’s scheme.
tation approaches share one common sub-task:
labeling spans of tokens with their corresponding
argumentation concept, the boundaries of the whereas only in 11.6% of the arguments, the sup-
spans are not known beforehand. Therefore, the port precedes the claim. The corresponding pat-
most appropriate measure here is the unitized terns consisting of attack and claim are signifi-
Krippendorf’s αU as the annotators identify and cantly less frequent: only 3.4% of the arguments
label the units in the same text (Krippendorff, consist of a claim and an attack.
2013). Other measures, such as Cohen’s κ or Annotated examples can be found in §A.1.
Fleiss’ π, expect the units (boundaries of the
argument component) to be known beforehand, 5.2 Outcomes of annotating with Toulmin’s
which is not the case here. scheme
In the first stage, three independent annotators la-
5.1 Outcomes of annotating with
beled 524 out of 990 documents as argumenta-
Claim-Premises scheme
tive/persuasive on the given topic. Total size of
During an annotation study of 6 weeks, three this dataset was 130,085 tokens (mean 131, std.
annotators (one inexperienced annotator and two dev. 139) and 6,371 sentences (mean 6.44, std.
experts) annotated 80 documents belonging to dev. 6.53). Agreement on the first sub-set of
six topics. On average, each annotator needed this dataset of 300 documents was 0.51 (Fleiss’ π,
23 hours to annotate the 3863 sentences. The three annotators per document), the second sub-set
annotators marked 5126 argument components (690 documents) was then annotated by two anno-
(53% premises, 47% claims) and 2349 arguments, tators with agreement 0.59 (Cohen’s κ). This stage
which is 2.2 argument components per argument. took in total about 17 hours per annotator.
On average, 74% of the tokens in the dataset are In the second phase that took about 33 hours
covered by an argument component indicates that per annotator, a collection of comments and forum
the documents are in fact highly argumentative. posts (294 documents) was randomly chosen from
An average claim spans 1.1 sentences, whereas an the previously labeled argumentative documents
average premise spans 2.2 sentences. from the previous stage together with 49 blog
While the IAA scores appeared to be non- posts and 8 newspaper articles. The total size of
substantial, ranging from αU =34.6 (distin- this dataset was 345 documents, containing 87,286
guishing all 6 annotation classes and non- tokens (mean 253.00, std. dev. 262.90) and 3,996
argumentative) to αU =42.4 (distinguishing be- sentences (mean 11.58, std. dev. 11.72). Three in-
tween premises, claims and non-argumentative), dependent annotators annotated the whole dataset
they are in line with previous results: Peldszus and in multiple phases. After each phase, they dis-
Stede (2013b) report αU =42.5 for their sentence- cussed discrepancies, resolved issues and updated
level annotation study. the annotation guidelines. The inter-annotator
By analysing typical patterns of argument com- agreement was measured on the last phase con-
ponents used in arguments, Kluge (2014) found taining 93 comments and forum posts, 8 blogs,
that almost three quarters of arguments (72.4%) and 6 articles. During the annotations, 2 articles
consist of one claim and one premise. In 59.5% and 4 forum posts/comments were also discarded
of these arguments, the support follows the claim, as non-argumentative.
3
Discussion about benefits or disadvantages of including Agreement (Krippendorf’s αU ) varies signifi-
children with special needs into regular classes. cantly given different argumentation components
and registers, as shown in Table 2. Given these narratives, quotations from sources, or direct and
results, we formulate the following conclusions. indirect speech.
This scheme seems to fit well short documents
(forum posts and comments) as they tend to bring Well structured newspaper articles versus
up one central claim with a support (grounds). poorly structured user-generated content
Its suitability for longer documents (blogposts and Producing a well-understandable argument is
editorials) is doubtful. We examined the annota- actually a human skill that can be acquired by
tion errors and found that in well-structured doc- learning; many textbooks are available on that
uments, the annotators were able to identify the topic, e.g., (Sinnott-Armstrong and Fogelin, 2009;
concepts reliably. However, if the discussion of Weston, 2008; Schiappa and Nordin, 2013).
the controversy is complex (many sub-aspects are Thus, it is very likely that, for example, trained
discussed) or follows a dialogical manner, appli- journalists in editorials and lay people in social
cation the Toulmin’s scheme is all but straightfor- media will produce very different argumentation,
ward. in terms of structure, language, etc.
Furthermore, the distinction between grounds
and backing also allows to capture different kinds
6.2 Properties of argumentation in
of evidence. Authors purposely use grounds to ex-
user-generated discourse
plicitly support their claim, while backing mostly
serves as an additional information (i.e., author’s Non-argumentative texts Distinguishing argu-
personal experience, referring to studies, etc.) and mentative from non-argumentative discourse is a
the argument can be still acceptable without it. necessary step that has to be undertaken before an-
However, boundaries between these two compo- notating argument components. While in newspa-
nents are still fuzzy and caused many disagree- per editorials some parts (such as paragraphs) may
ments. be ignored during argument annotation (Kluge,
We show few annotation examples (as agreed 2014), in comments and forum posts we had to
by all annotators after the study) in §A.2. perform an additional step to filter documents that
do not convey any argumentation or persuasion
6 Observations (cf. §5.2 or Example 4 in §A.2).
In this section, we would like to summarize some
important findings from our annotation studies. Implicit argumentation components in Toul-
min’s model As already reported by Newman
6.1 Data heterogeneity and Marshall (1991), some argument components
are not explicitly expressed. This is mostly the
Variety or registers There exist many on-line
case of warrant in the original Toulmin’s model;
registers that carry argumentation to topics un-
we also discarded this component from our exten-
der investigation, such as newspaper reports (i.e.,
sion. However, even the claim is often not stated
events), editorials (opinions), interviews (single
explicitly, as seen in example 3 (§A.2). The claim
party, multiple parties), blogposts,4 comments to
reflects the author’s stance and can be understood
articles and blogs (threaded allowing explicit dis-
(inferred) by readers, but is left implicit.
cussion, linear with implicit discussion by quoting
and referencing), discussion forums, Twitter, etc.
Other rhetorical dimensions of argument All
Short versus long documents Different docu- the models for argumentation discussed so far fo-
ment lengths affect the style of argumentation. cus solely on the logos part of the argument. How-
Short documents (i.e., Tweets in the extreme case) ever, rhetorical power of argumentation also in-
have to focus on the core of the argument. By con- volves other dimensions, namely pathos, ethos,
trast, long documents, such as blog posts or edito- and kairos (Aristotle and Kennedy (translator),
rials, may elaborate various aspects of the topic 1991; Schiappa and Nordin, 2013). These have
and usually employ many literary devices, such as never been tackled in computational approaches to
4
modeling argumentation. Furthermore, figurative
In contrast to traditional publisher, bloggers do not have
to comply with strict guidelines or the use of formal language langauge, fallacies, or narratives (see example 3 in
(Santos et al., 2012). §A.2) are prevalent in argumentation on the Web.
6.3 Recommendations area includes, e.g., opinion-based summarization
Based on the experience from the annotation stud- of blogposts (a pilot task in TAC 20085 ). Carenini
ies, we would like to conclude with the follow- and Cheung (2008) compared extractive and
ing recommendations: (1) selection of argumen- abstractive summaries in controversial documents
tation model should be based on the data at hand and found out that a high degree of controver-
and the desired application; our experiments show siality improved performance of their system.
that Toulmin’s model is more expressive than the Similarly, presenting argumentation in a con-
Claim-Premise model but is not suitable for long densed form (the large concepts of the argument
documents, (2) annotating argumentation is time- are compressed or summarized) may improve
demanding and error-prone endeavor; annotators argument comprehension. This approach would
thus have to be provided with detailed and elab- mainly utilize tools for document compression
orated annotation guidelines and be extensively (Qian and Liu, 2013).
trained (our experiments with crowdsourcing were
8 Conclusions
not successful).
In this article, we formulated our view on argu-
7 Follow-up use cases mentation mining on the Web and identified var-
ious use-case scenarios and expected outcomes.
Understanding argumentation in user-generated
We thoroughly reviewed related work with focus
content can foster future research in many areas.
on Web data and annotation approaches. We pro-
Here we present two concrete applications.
posed two different annotation schemes based on
7.1 Understanding argumentative discourse their theoretical counterparts in argumentation re-
in education search and evaluated their suitability and reliabil-
ity for Web data in two extensive independent an-
Computer-supported argumentation has been a
notation studies. Finally, we outlined challenges
very active research field, as shown by Scheuer
and gaps in current argumentation mining on the
et al. (2010) in their recent survey of vari-
Web.
ous models and argumentation formalisms from
the educational perspective. Many studies on Acknowledgments
computer-supported collaboration and argumenta-
tion (Noroozi et al., 2013; Weinberger and Fischer, This work has been supported by the Volks-
2006; Stegmann et al., 2007) can directly bene- wagen Foundation as part of the Lichtenberg-
fit from NLP techniques for automatic argument Professorship Program under grant No. I/82806,
detection, classification, and summarization. In- and by the German Institute for Educational Re-
stead of relying on scripts (Dillenbourg and Hong, search (DIPF).
2008; Scheuer et al., 2010; Fischer et al., 2013)
A Annotated examples
or explicit argument diagramming (Scheuer et al.,
2014), collaborative platforms can further provide A.1 News articles using Claim-Premises
scholars with a summary of the whole argumen- scheme
tation to the topic, reveal the main argumenta- Example 1
tive patterns, provide the weaknesses of other’s
[claim: ,,Die Umstellung zu G8 war schwierig“,
arguments, as well as identify shortcomings that
sagt Diana. ] [support: In den Sommerferien nach
need to be improved in the argumentative knowl-
dem Sitzenbleiben holte sie das nach, was ihr die
edge construction. Automatic analysis of micro-
G8er voraus hatten: Lateinvokabeln, Stochastik,
arguments can also help to overcome the existing
Grammatik. ,,Den Vorteil, durch das Wiederholen
trade-off between freedom (free-text option) and
den Stoff noch mal zu machen, hatte ich nicht.“ ]
guidance (scripts) (Dillenbourg and Hong, 2008).
[claim: “The change [to G8] was difficult,” says
7.2 Automatic summarization of Diana. ] [support: (Since) After staying down,
argumentative discourse she had to catch up with the G8 students during
When summarizing argumentative discourse, her summer holiday, studying Latin vocabulary,
knowledge of the underlying structure of the ar- 5
http://www.nist.gov/tac/publications/
gument is a valuable source. Previous work in this 2008/papers.html
stochastics, and grammar. “I did not have the A.2 Forum posts using extended Toulmin’s
advantage of reviewing previous material.” ] scheme
Example 1
Example 2
[backing:
. . . . . . . . . . . . . I’m
. . . . .a. . .regular
. . . . . . . .education
. . . . . . . . . . .teacher.
. . . . . . . . . . .I
[claim: Lehrer wird man, weil das ein have
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .every
students mainstreamed into my class .....
sicherer Beruf ist. ] [support: So denken year.]
. . . . . . [grounds: My opinion is that it needs to be
noch immer viele junge Leute, die sich für done far more judiciously than it is done now- if
eine Pädagogenlaufbahn entscheiden. Gut acht six exceptional children are put in my class, that
von zehn Erstsemestern, die 2009 mit einem is the equivalent of putting an entire special ed
Lehramtsstudium anfingen, war dieser Aspekt classroom into my regular class.] [grounds: I
ihres künftigen Berufs wichtig oder sogar sehr personally feel like these kids are shortchanged-
wichtig. Keine andere Studentengruppe, die some of them are good kids who need an adult
die Hochschul-Informations-System GmbH HIS close by and able to give more focused attention.
befragte, legt so viel Wert auf Sicherheit. ] In a class of 30+, this isn’t going to happen
consistently.] [grounds: And some of the
[claim: People become teachers because it is a ones who come to me have legally imposed
safe job. ] [support: This is what more and more modifications, some of which have little or no
young people who decide to become a teacher bearing on what I teach, so I am not allowed to
think. Well over eight of 10 freshman students handle my class in a way I think it should be
who started to study to become teachers in 2009 done. That impairs my efficiency as an educator.]
considered this an important or very important [grounds: Also, some have so many modifications
aspect. No other group of students interviewed by that for all intents and purposes they are merely
the HIS set that much value on safeness. ] taking a special ed class whose physical location
just happens to be in a regular classroom.] [claim:
Example 3 From my point of view, mainstreaming is not a
terrible idea, but it is lamentable in its execution,
[claim: Für die Unis sind Doktoranden günstige
and because of that, damaging in its results.]
Arbeitskräfte. ] [support: Eine Bekannte hatte
mit ihrem Doktorvater zu kämpfen, der versuchte, Comments Quite a good argument with an ex-
sie noch am Institut zu halten, als ihre Arbeit plicit claim, few grounds and some backing.
längst fertig war. Er hatte immer neue Ausreden,
Example 2
weshalb er noch keine Note geben konnte. Als
sie dann auch ohne Note einen guten Job bekam, tara mommy:
auerhalb der Uni, spielte sich eine Art Rosenkrieg I agree with you too, which is why I said:
zwischen den beiden ab. Bis heute verlangt er von [rebuttal: There are obviously cases where this
::::::::::::::::::::::::::::::::::::::::::::
ihr noch Nacharbeiten an der Dissertation. Sie isn’t going to work. Extreme behavioral trouble,
::::::::::::::::::::::::::::::::::::::::::::
schuftet jetzt spätabends und am Wochenende für kids that just aren’t able to keep up with what
::::::::::::::::::::::::::::::::::::::::::::
ihren Ex-Prof, der natürlich immer nur an ihrem they’re learning in average classes, etc.] [claim:
::::::::::::::::::::::::::::::::::::
Fortkommen interessiert war. ] But on the whole, I like mainstreaming.]
[claim: At university, graduate students are Comments Only claim and rebuttal; no support-
cheap employees. ] [support: An acquaintance ing grounds.
struggled with her Ph.D. supervisor, who tried to
keep her in his group at any rate, even though Example 3
she had already completed her thesis. He pled l think as parents of the child you have to be
more and more excuses for not yet grading her certain and confident that your child is ready
work. When she finally found a good job outside to mainstream. lf not, it can backfire on the
university even without a final grade a martial child. [backing:. . . . . . . . . . . . .My. . . . .child
. . . . . .was
. . . . . in
. . . ”preschool
...........
strife arose. Still today, he asks her to rework handicapped”
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .tried
from age 2-5. We . . . . . . .to.
her dissertation. Now, she is drudging for her mainstream
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . a. .hard
him in kindergarten, but he had ....
ex-supervisor, who always only wanted the best time
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .one
adjusting. So the school got him a one on ...
for her, late in the evening or on the weekend. ] para
. . . . . and
. . . . .it. .helped
. . . . . . . a. .bit.
. . . . 2. . grades
. . . . . . . .later,
. . . . . .he
. . .still
. . . .has
...
a. .one
. . . . on
. . . .one
. . . .aide
. . . . .but
. . . .doing
. . . . . .EXCELLENT.]
................ Philippe Besnard and Anthony Hunter. 2008. El-
Our goal is for him to not have a one on one by ements of argumentation, volume 47. MIT press
Cambridge.
middle school. We took him off meds and we have
a strong behavior plan, he sees therapists, and it is Douglas Biber and Susan Conrad. 2009. Register,
hourly teaching and redirecting with him. Truth be Genre, and Style. Cambridge Textbooks in Linguis-
told College may not be in his future, but we will tics. Cambridge University Press.
do everything in our power to try to get him there. J. Anthony Blair. 2004. Argument and its uses. Infor-
mal Logic, 24:137151.
Comments The claim is implicit, the author is
slightly against mainstreaming. Mainly story- Elena Cabrio and Serena Villata. 2012. Combin-
telling, which is not considered as grounds but as ing textual entailment and argumentation theory for
supporting online debates interactions. In Proceed-
backing. The typos (using ‘l’ instead of ‘I’) are ings of the 50th Annual Meeting of the Association
kept uncorrected. for Computational Linguistics (Volume 2: Short Pa-
pers), pages 208–212, Jeju Island, Korea, July. As-
Example 4 sociation for Computational Linguistics.
My lo has mild autism, he has only just been di- Elena Cabrio, Sara Tonelli, and Serena Villata.
agnosed, he is delayed in some areas (but not oth- 2013a. From Discourse Analysis to Argumen-
ers), he goes to ms school, and has some one to tation Schemes and Back: Relations and Differ-
ences. In João Leite, Tran Cao Son, Paolo Torroni,
one (this should increase now, I hope). There is
Leon Torre, and Stefan Woltran, editors, Proceed-
one TA and a full time TA who supports another ings of 14th International Workshop on Computa-
child with autism. It’s a smallish school. tional Logic in Multi-Agent Systems, volume 8143
He isn’t disruptive (well he sometimes doesn’t do of Lecture Notes in Computer Science, pages 1–17.
as asked and can be a little akward), he has never Springer Berlin Heidelberg.
been aggressive in anyway, he is very happy. Elena Cabrio, Serena Villata, and Fabien Gandon.
I am worried about his future (high school)after 2013b. A support framework for argumentative
reading this. discussions management in the web. In Philipp
Cimiano, Oscar Corcho, Valentina Presutti, Laura
Sarah x
Hollink, and Sebastian Rudolph, editors, The Se-
mantic Web: Semantics and Big Data, volume 7882
Comments Not an argumentative/persuasive of Lecture Notes in Computer Science, pages 412–
text. 426. Springer Berlin Heidelberg.
Giuseppe Carenini and Jackie Chi Kit Cheung. 2008.
Extractive vs. NLG-based abstractive summariza-
References tion of evaluative text: The effect of corpus contro-
Corina Andone. 2012. Bermejo-Luque, Lilian. Giving versiality. In Proceedings of the Fifth International
Reasons. A Linguistic-Pragmatic Approach to Argu- Natural Language Generation Conference, INLG
mentation Theory. Argumentation, 26(2):291–296. ’08, pages 33–41, Stroudsburg, PA, USA. Associ-
ation for Computational Linguistics.
Aristotle and George Kennedy (translator). 1991. On Marilyn J. Chambliss. 1995. Text cues and strate-
Rhetoric: A Theory of Civil Discourse. Oxford Uni- gies successful readers use to construct the gist of
versity Press. lengthy written arguments. Reading Research Quar-
terly, 30(4):778–807.
Bal Krishna Bal and Patrick Saint-Dizier. 2010. To-
wards Building Annotated Resources for Analyz- Alexander Conrad, Janyce Wiebe, and Rebecca Hwa.
ing Opinions and Argumentation in News Editorials. 2012. Recognizing arguing subjectivity and ar-
In Nicoletta Calzolari, Khalid Choukri, Bente Mae- gument tags. In Roser Morante and Caroline
gaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Sporleder, editors, Proceedings of the Workshop on
Mike Rosner, and Daniel Tapias, editors, Proceed- Extra-Propositional Aspects of Meaning in Compu-
ings of the Seventh International Conference on tational Linguistics, pages 80–88, Jeju Island, Ko-
Language Resources and Evaluation (LREC’10), rea. Association for Computational Linguistics.
pages 1152–1158. European Language Resources
Association (ELRA). Jim Crosswhite, John Fox, Chris Reed, Theodore Scalt-
sas, and Simone Stumpf. 2004. Computational
Jamal Bentahar, Bernard Moulin, and Micheline models of rhetorical argument. In Chris Reed and
Bélanger. 2010. A taxonomy of argumentation Timothy J. Norman, editors, Argumentation Ma-
models used for knowledge representation. Artifi- chines, volume 9 of Argumentation Library, pages
cial Intelligence Review, 33:211–259. 175–209. Springer Netherlands.
Aldo de Moor, Lilia Efimova, and Aldo De Moor. Anthony Hunter. 2014. Probabilistic qualification
2004. An Argumentation Analysis of Weblog Con- of attack in abstract argumentation. International
versations. In Proceedings of the 9th International Journal of Approximate Reasoning, 55(2):607–638,
Working Conference on the Language-Action Per- January.
spective on Communication Modelling (LAP 2004),
volume 197, pages 1–16. Ralph H Johnson. 2000. Manifest rationality: A prag-
matic theory of argument. Routledge.
Pierre Dillenbourg and Fabrice Hong. 2008. The me-
chanics of CSCL macro scripts. International Jour- Roland Kluge. 2014. Automatic Analysis of Ar-
nal of Computer-Supported Collaborative Learning, guments about Controversial Educational Topics
3(1):5–23. in Web Documents, Master Thesis, Ubiquitious
Knowledge Processing Lab, TU Darmstadt.
Phan Minh Dung. 1995. On the acceptability of ar-
Klaus Krippendorff. 2013. Content Analysis: An In-
guments and its fundamental role in nonmonotonic
troduction to Its Methodology. Thousand Oaks, CA:
reasoning, logic programming and n-person games.
Sage Publications, 3rd edition.
Artificial Intelligence, 77(2):321 – 357.
Meredith Larson, M. Annae Britt, and Aaron Larson.
Vanessa Wei Feng and Graeme Hirst. 2011. Classi- 2004. Disfluencies in comprehending argumentative
fying arguments by scheme. In Proceedings of the texts. Reading Psychology, 25:205–224.
49th Annual Meeting of the Association for Com-
putational Linguistics: Human Language Technolo- Ran Levy, Yonatan Bilu, Daniel Hershcovich, Ehud
gies - Volume 1, HLT ’11, pages 987–996, Strouds- Aharoni, and Noam Slonim. 2014. Context depen-
burg, PA, USA. Association for Computational Lin- dent claim detection. In Proceedings of the 25th In-
guistics. ternational Conference on Computational Linguis-
tics (COLING 2014), August. To appear.
Frank Fischer, Ingo Kollar, Karsten Stegmann, and
Christof Wecker. 2013. Toward a script theory of Bing Liu. 2012. Sentiment analysis and opinion min-
guidance in computer-supported collaborative learn- ing. Synthesis Lectures on Human Language Tech-
ing. Educational Psychologist, 48(1):56–66. nologies, 5(1):1–167.
Eirini Florou, Stasinos Konstantopoulos, Antonis Clare Llewellyn, Claire Grover, Jon Oberlander, and
Koukourikos, and Pythagoras Karampiperis. 2013. Ewan Klein. 2014. Re-using an Argument Cor-
Argument extraction for supporting public policy pus to Aid in the Curation of Social Media Col-
formulation. In Proceedings of the 7th Workshop lections. In Proceedings of the Ninth International
on Language Technology for Cultural Heritage, So- Conference on Language Resources and Evaluation
cial Sciences, and Humanities, pages 49–54, Sofia, (LREC’14), pages 462–468.
Bulgaria. ACL.
Lilian Bermejo Luque. 2011. Giving Reasons:
Austin J. Freeley and David L. Steinberg. 2008. Ar- A Linguistic-Pragmatic Approach to Argumenta-
gumentation and Debate. Cengage Learning, Stam- tion Theory, volume 20 of Argumentation Library.
ford, CT, USA, 12th edition. Springer Netherlands.
Nitin Madnani, Michael Heilman, Joel Tetreault, and
James B Freeman. 1991. Dialectics and the
Martin Chodorow. 2012. Identifying high-level
macrostructure of arguments: A theory of argument
organizational elements in argumentative discourse.
structure, volume 10 of Trends in Linguistics. De
In Proceedings of the 2012 Conference of the North
Gruyter.
American Chapter of the Association for Computa-
tional Linguistics: Human Language Technologies,
Kallirroi Georgila, Ron Artstein, Angela Nazarian,
NAACL HLT ’12, pages 20–28, Stroudsburg, PA,
Michael Rushforth, David Traum, and Katia Sycara.
USA. Association for Computational Linguistics.
2011. An annotation scheme for cross-cultural ar-
gumentation and persuasion dialogues. In Proceed- William C. Mann and Sandra A. Thompson. 1987.
ings of the SIGDIAL 2011 Conference: the 12th An- Rhetorical structure theory: A theory of text organi-
nual Meeting of the Special Interest Group on Dis- zation. Technical report, Information Sciences Insti-
course and Dialogue, pages 272–278, Portland, Ore- tute, University of Southern California, Marina del
gon. Association for Computational Linguistics. Rey, CA, USA.
A. Francisca Snoeck Henkemans. 2000. State-of-the- Raphaël Micheli. 2011. Arguing Without Trying to
art: The structure of argumentation. Argumentation, Persuade? Elements for a Non-Persuasive Definition
14(4):447–473. of Argumentation. Argumentation, 26(1):115–126,
September.
Anthony Hunter. 2013. A probabilistic approach to
modelling uncertain logical arguments. Interna- Raquel Mochales and Marie-Francine Moens. 2011.
tional Journal of Approximate Reasoning, 54(1):47– Argumentation mining. Artificial Intelligence and
81, January. Law, 19(1):1–22, April.
S. Newman and C. Marshall. 1991. Pushing Toulmin Rodrygo LT Santos, Craig Macdonald, Richard MC
Too Far: Learning From an Argument Representa- McCreadie, Iadh Ounis, Ian Soboroff, et al. 2012.
tion Scheme. Technical report, Xerox Palo Alto Re- Information retrieval on the blogosphere. Founda-
search Center 3333 Coyote Hill Road, Palo Alto, CA tions and Trends in Information Retrieval, 6(1):1–
94034. 125.
Omid Noroozi, Armin Weinberger, Harm J.a. Biemans, Oliver Scheuer, Frank Loll, Niels Pinkwart, and
Martin Mulder, and Mohammad Chizari. 2013. Bruce M. McLaren. 2010. Computer-supported ar-
Facilitating argumentative knowledge construction gumentation: A review of the state of the art. In-
through a transactive discussion script in CSCL. ternational Journal of Computer-Supported Collab-
Computers & Education, 61:59–76, February. orative Learning, 5(1):43–102.
Raquel Mochales Palau and Marie-Francine Moens. Oliver Scheuer, BruceM. McLaren, Armin Weinberger,
2009. Argumentation mining: The detection, clas- and Sabine Niebuhr. 2014. Promoting critical, elab-
sification and structure of arguments in text. In orative discussions through a collaboration script
Proceedings of the 12th international conference on and argument diagrams. Instructional Science,
artificial intelligence and law, pages 98–107, New 42(2):127–157.
York, NY, USA. ACM.
Edward Schiappa and John P. Nordin. 2013. Argumen-
Andreas Peldszus and Manfred Stede. 2013a. From tation: Keeping Faith with Reason. Pearson UK, 1st
Argument Diagrams to Argumentation Mining in edition.
Texts:. International Journal of Cognitive Informat-
ics and Natural Intelligence, 7(1):1–31, January. Jodi Schneider and Adam Wyner. 2012. Identifying
Consumers’ Arguments in Text. In Diana Maynard,
Andreas Peldszus and Manfred Stede. 2013b. Rank- Marieke van Erp, and Brian Davis, editors, Semantic
ing the annotators : An agreement study on argu- Web and Information Extraction SWAIE 2012, pages
mentation structure. In Proceedings of the 7th Lin- 31–42.
guistic Annotation Workshop & Interoperabilty with
Discourse, pages 196–204. Association for Compu- Jodi Schneider, B Davis, and Adam Wyner. 2012. Di-
tational Linguistics. mensions of argumentation in social media. In Lec-
ture Notes in Computer Science, volume 7603, pages
Chaim Perelman and Lucie Olbrechts-Tyteca. 1991. 21–25. Springer Berlin Heidelberg.
The New Rhetoric. University of Notre Dame Press.
Jodi Schneider, Tudor Groza, and Alexandre Passant.
2013. A review of argumentation for the social se-
Henry Prakken. 2010. An abstract framework for ar-
mantic web. Semantic Web, 4(2):159–218.
gumentation with structured arguments. Argument
& Computation, 1(2):93–124, June. Alan Sergeant. 2013. Automatic Argumentation Ex-
traction. In ESWC 2013, pages 656–660. Springer-
Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Milt- Verlag Berlin Heidelberg.
sakaki, Livio Robaldo, Aravind Joshi, and Bonnie
Webber. 2008. The Penn Discourse TreeBank 2.0. Maria Simosi. 2003. Using Toulmin’s frame-
In Nicoletta Calzolari, Khalid Choukri, Bente Mae- work for the analysis of everyday argumentation:
gaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Some methodological considerations. Argumenta-
and Daniel Tapias, editors, Proceedings of the Sixth tion, 17:185–202.
International Conference on Language Resources
and Evaluation (LREC’08), pages 1–4. European Walter Sinnott-Armstrong and Robert J. Fogelin. 2009.
Language Resources Association (ELRA). Understanding Arguments: An Introduction to In-
formal Logic. Cengage Learning, 8 edition.
Xian Qian and Yang Liu. 2013. Fast joint compres-
sion and summarization via graph cuts. In Proceed- Christian Stab and Iryna Gurevych. 2014. Annotat-
ings of the 2013 Conference on Empirical Methods ing argument components and relations in persua-
in Natural Language Processing, pages 1492–1502, sive essays. In Proceedings of the 25th International
Seattle, Washington, USA, October. Association for Conference on Computational Linguistics (COLING
Computational Linguistics. 2014), August. To appear.
Chris Reed and Douglas Walton. 2003. Argumenta- Karsten Stegmann, Armin Weinberger, and Frank Fis-
tion schemes in argument-as-process and argument- cher. 2007. Facilitating argumentative knowl-
as-product. In Proceedings of the conference cele- edge construction with computer-supported collab-
brating informal Logic, volume 25. oration scripts. International Journal of Computer-
Supported Collaborative Learning, 2(4):421–447.
Sara Rosenthal and Kathleen McKeown. 2012. De-
tecting Opinionated Claims in Online Discussions. Dragan Stoianovici. 2009. Formal Logic vs. Philo-
In 2012 IEEE Sixth International Conference on Se- sophical Argument. Argumentation, 24(1):125–133,
mantic Computing, pages 30–37. IEEE, September. January.
Stephen Toulmin, Richard Rieke, and Allan Janik.
1984. An Introduction to Reasoning. Macmillan,
2nd edition.
Stephen E. Toulmin. 1958. The Uses of Argument.
Cambridge University Press.
Maria Paz Garcia Villalba and Patrick Saint-Dizier.
2012. Some Facets of Argument Mining for Opin-
ion Analysis. In Bart Verheij, Stefan Szeider, and
Stefan Woltran, editors, Proceedings of Fourth In-
ternational Conference on Computational Models of
Argument, COMMA 2012.
Artem Vovk. 2013. Discovery and Analysis of Public
Opinions on Controversial Topics in the Educatio-
nal Domain, Master Thesis, Ubiquitious Knowledge
Processing Lab, TU Darmstadt.
Henning Wachsmuth, Martin Trenkmann, Benno Stein,
and Gregor Engels. 2014a. Modeling Review Argu-
mentation for Robust Sentiment Analysis. In Pro-
ceedings of the 25th International Conference on
Computational Linguistics COLING 2014, page To
appear, Dublin, Ireland.
Henning Wachsmuth, Martin Trenkmann, Benno Stein,
Gregor Engels, and Tsvetomira Palakarska. 2014b.
A Review Corpus for Argumentation Analysis. In
Alexander Gelbukh, editor, 15th International Con-
ference on Intelligent Text Processing and Compu-
tational Linguistics (CICLing 14), pages 115–127.
Springer.
Douglas Walton, Christopher Reed, and Fabrizio
Macagno. 2008. Argumentation Schemes. Cam-
bridge University Press.
Douglas Walton. 2005. Fundamentals of Critical Ar-
gumentation. Critical Reasoning and Argumenta-
tion. Cambridge University Press, 1 edition.
Douglas Walton. 2012. Using Argumentation
Schemes for Argument Extraction: A Bottom-Up
Method. International Journal of Cognitive Infor-
matics and Natural Intelligence, 6(3):33–61.
Armin Weinberger and Frank Fischer. 2006. A frame-
work to analyze argumentative knowledge construc-
tion in computer-supported collaborative learning.
Computers & Education, 46(1):71–95, January.
Anthony Weston. 2008. A Rulebook for Arguments.
Hackett Pub Co., 4 edition.
Yun Xie. 2012. Review of Giving Reasons. Informal
Logic, 32(4).
Cihua Xu and Yicheng Wu. 2014. Metaphors in the
perspective of argumentation. Journal of Pragmat-
ics, 62:68–76, February.