In existing literature, little attention has been paid to the problems of how the uncertainty reflected by natural language text (e.g. verbal and linguistic uncertainty) can be explicitly formulated in argumentation schemes, and how argumentation schemes enriched with various types of uncertainty can be exploited to support argumentation mining and evaluation. In this paper, we focus on the first problem, and introduce some preliminary ideas about how to classify and encompass uncertainty in argumentation schemes.
Mining and evaluating arguments from natural
language text
Corresponding author 1Due to space limitations, we assume knowledge of Dung’s theory in the following.
When dealing with natural language sources, one of the challenging problems is to handle the uncertainty of arguments. In fact, natural language statements typically include several kinds of uncertainty. This calls for the need to encompass uncertainty in the formalisms which are meant to provide a representation of natural arguments, first of all in argumentation schemes, in order to avoid that some useful information carried by the text source is lost in the first modelling step.
To illustrate this problem, let us consider a simple example concerning two conflicting natural language excerpts E1 and E2, possibly taken from some medical publications:
E1: According to [Smith 98], drug X often causes the side effect Y.
E2: According to recent experimental trials, it is highly likely that drug X does not increase the probability of the side effect Y.
In order to identify argument structures in these
texts, one may resort to specific argumentation
schemes. Referring to the classification proposed
in
After A1 and A2 are identified, it may be noted
that (though expressed with different linguistic
nuances) their conclusions are in conflict: briefly, A1
leads to the claim that X causes Y, while A2 to the
claim that X does not cause Y. As a consequence,
a mutual attack relation between A1 and A2 can
be identified. Then, the arguments and their
attacks can be formalized as an abstract
argumentation framework AF = (fA1; A2g, f(A1, A2),
(A2; A1)g) and the status of arguments in AF can
be evaluated according to a given argumentation
semantics. For instance, under grounded
semantics, both A1 and A2 are not accepted. It must
be noted however that such a modelling approach
2) How argumentation schemes enriched with various types of uncertainty can be exploited to support argument mining and evaluation? 3) Which is (are) the most appropriate abstract et al., 2013)). In this case, we would get a differformalism(s) for the evaluation of arguments with f(A2; A1)g).
A1 is rejected, while A2 is accepted.
Then, under grounded semantics, ent argumentation framework AF 0 = (fA1; A2g, uncertainty?
Texts with uncertainty (e.g. verbal / linguistic uncertainty) Argument
Mining Argumentation schemes with uncertainty
Semi-formal argumentation with uncertainty
B B NB
Formal argumentation with uncertainty
A A
UA Argument Formalization
Argument Evaluation
However, a static preference relation on the adopted scheme appears too rigid: in most cases the preference for an argument over another one is not simply based on their structure but, rather, on their content. To exemplify, in this case, one may have different opinions on the reliability of the source [Smith 98], mentioned in E1, and of the experimental trials mentioned in E2. Moreover, the two excerpts include several terms expressing vagueness and/or uncertainty, like often, highly likely, the probability of, that may be taken into account in the preference ranking of arguments.
However, this is not possible in the approach sketched above, since the argument schemes adopted in the formalization do not encompass these forms of uncertainty and the relevant information carried by the text is lost in the first modelling step.
Given the pervasiveness of vagueness and uncertainty in natural language this appears to be a severe limitation for the use of argumentation schemes in argument mining from texts. To overcome this problem we envisage the study of argumentation schemes extended with uncertainty in the context of the process sketched in Figure 1. Here argumentation schemes with uncertainty are used to extract arguments from texts, keeping explicit the relevant uncertainties that can then be used in the step of argument evaluation using suitIn natural language texts different types of uncerable abstract formalisms and semantics with untainty can be identified. To give a brief account of certainty. As to the latter step, the study of exthe richness and complexity of this topic and of the tensions of Dung’s framework with explicit unresearch activities that are being carried out in this certainty representation is receiving increasing atarea, we quickly recall some examples of uncertention in recent years (Li et al., 2011a; Thimm, tainty classifications considered in the literature.
In the context of scientific discourse, de Waard
and
For instance, according to de Waard and
1) Explicit source of knowledge: the knowledge evaluation can be explicitly owned by the author (‘We therefore conclude that . . . ’) or by a named referent (‘Vijh et al. [28] demonstrated that . . . ’).
2) Implicit source of knowledge: if there is no explicit source named, knowledge can implicitly still be attributed to the author (‘ these results suggest . . . ’) or an external source (‘It is generally believed that . . . ’).
3) No source of knowledge: the source of knowledge can be absent entirely, e.g. in factual statements, such as ‘transcription factors are the final common pathway driving differentiation’.
Since different sources may have different degrees of credibility, this leads to identify a first type of uncertainty, namely the (possibly implicit) source uncertainty.
As to knowledge evaluation, de Waard and
This kind of evaluation, called uncertainty about statements in the following, is typically expressed through suitable linguistic modifiers.
Actually linguistic modifiers have a quite
generic nature and have been the subject of
specific stu
Regan et al. (2002) distinguish between epistemic uncertainty (uncertainty in determinate facts) and linguistic uncertainty (uncertainty in language) and claims that the latter has received by far less attention in uncertainty classifications in the fields of ecology and biology. Linguistic uncertainty is in turn classified into five distinct types: vagueness, context dependence, ambiguity, indeterminacy of theoretical terms, and underspecificity, with vagueness being claimed to be the most important for practical purposes. In fact, all of them refer in some way to the problem that some natural language expressions admit alternative interpretations. Hence this classification is focused on a specific form of uncertainty and the use of the term linguistic uncertainty here is rather restricted with respect to other works.
Taking into account the discussion above, in this paper we consider, as a starting point, three uncertainty types:
1) Source uncertainty, denoted in the following as U1, concerning the fact that to evaluate the credibility of different statements one may take into account the credibility of their sources;
2) Uncertainty about a statement, denoted as U2, arising in situations where a subject making a statement expresses a partial degree of commitment to the statement itself;
3)Linguistic uncertainty or uncertainty inside a statement, denoted as U3, namely uncertainty generically present in natural language statements, with no further more precise meaning specified.
For instance in the sentences “According to [Smith 98], Drug X causes headache” and “According to recent experimental trials, Drug X causes headache”, one may identify U1 since they refer the statement “Drug X causes headache” to a source (a paper and clinical trials, respectively).
On the other hand, the sentence “It is likely that Drug X causes headache” provides an example of U2 since the statement “Drug X causes headache” is not regarded as certain.
Finally, a sentence like “Drug X sometimes causes severe headache” provides an example of U3.
For a more articulated example including
several uncertainty types, let us consider the
following text, taken from
Here, some expressions (likely and suggest that)
indicate a partial commitment of authors to the
corresponding statements (U2), and the
knowledge source is made explicit by the citation of
Given that uncertainty pervades natural language
texts and argumentation schemes appear as
suitable formal tool for argumentation mining from
texts, the question of how to capture uncertainty in
argumentation schemes naturally arises. This
appears to be an open research question, as the
stateof-the-art formulation of argumentation schemes
The first scheme we consider, called Argument from Position to Know (APK), is defined as follows:
Major Premise: Source a is in a position to know about things in a certain subject domain S containing proposition A.
Minor Premise: a asserts that A (in domain S) is true (false).
Conclusion: A is true (false).
CQ1: Is a in a position to know whether A is 2Recall that an argument scheme basically consists of a set of premises, a conclusion defeasibly derivable from the premises according to the scheme, and a set of critical questions (CQs) that can be used to challenge arguments built on the basis of the scheme.
true (false)? CQ2: Is a an honest (trustworthy, reliable) source? CQ3: Did a assert that A is true?
In this scheme, no explicit uncertainty is included, but the critical questions correspond to several forms of uncertainty that may affect it.
The second scheme, called Argument from Cause to Effect (ACE), is defined as follows: Major Premise: Generally, if A occurs, then B will (might) occur.
Minor Premise: In this case, A occurs (might occur).
Conclusion: Therefore, in this case, B will (might) occur.
CQ1: How strong is the causal generalization? CQ2: Is the evidence cited (if there is any) strong enough to warrant the casual generalization? CQ3: Are there other causal factors that could interfere with the production of the effect in the given case?
In this case, in addition to the implicit uncertainty corresponding to critical questions, explicit expressions of uncertainty are included, namely the modifier Generally and the might specifications in the parentheses.
Clearly the representation of uncertainty in the two schemes is not uniform (since the second scheme encompasses explicit uncertainty in the premises and the conclusion, while the first does not) but it is not clear whether this non-uniformity is based on some underlying difference between the schemes or is just accidental in the natural language formulation of the schemes. Indeed, it seems possible to reformulate these schemes in a dual manner (adding explicit uncertainty mentions to the first one, removing them from the second one) while not affecting their meaning, as follows:
Major Premise: Source a is (possibly) in a position to know about things in a certain subject domain S containing proposition A. Minor Premise: a asserts that A (in domain S) is (might be) true (false).
Conclusion: A is (might be) true (false).
Major Premise: If A occurs, then B will occur. Minor Premise: In this case, A occurs.
Conclusion: Therefore, in this case, B will
The above-mentioned non-uniformity suggests that a more systematic treatment of uncertainty in argument schemes is needed in order to face the challenges posed by the representation of natural language arguments.
Indeed, a recent work
Devising a systematic approach to encompass natural language uncertainty in argumentation schemes is a long term research goal, posing many conceptual and technical questions and challenges, partly evidenced in the previous sections. We suggest that such an approach should include the following “ingredients”: 1) a classification of uncertainty types; 2) a characterization of the uncertainty types relevant to each argumentation scheme;
3) a formalism for the representation of uncertainty evaluations (of various types) in actual arguments, i.e. in instances of argument schemes; 4) a mechanism to derive an uncertainty evaluation for the conclusion of an argument from the evaluations concerning the premises and the applied scheme.
While each of the items listed above is, by itself, a large and open research question, we provide here some preliminary examples of point 2, using for point 1 the simple classification introduced in Section 2. In particular we suggest that the scheme specification should be accompanied by an explicit account of the types of uncertainty it may involve, while the use of linguistic uncertainty expressions in the scheme (like in ACE above) should be avoided within the natural language description of the scheme itself. This approach prevents the non-uniformities pointed out in Section 3 and enforces the adoption of clear modelling choices about uncertainty at the moment of definition of the scheme. In particular, as evidenced below, it may point out some ambiguities in the definition of the scheme itself.
In the following examples, we explicitly associate uncertainty types with the premises of the considered schemes (that may affected by them) and with the critical questions (that point out the potential uncertainty affecting the premises). Analysing the uncertainty possibly affecting the scheme itself or its applicability (that may also be expressed by some critical questions) is left to future work (and requires a richer classification of uncertainty types), while, according to point 4 above, the uncertainty about the conclusion is regarded as a derived notion and, for the sake of the present analysis, is considered as derived uncertainty, denoted as DU. The syntax we use to associate uncertainty types with parts of argument schemes is as follows: f: : :g[Ux; : : :], where the part of the scheme (possibly) affected by uncertainty is enclosed in braces and is followed by the relevant uncertainty type(s) enclosed in brackets.
First, let us consider the APK scheme. Here, the major premise explicitly refers to a source a, so it can be associated with U1 (as evidenced by the critical questions CQ1 and CQ2). Further one may consider that the inclusion of proposition A in domain S and the proposition A itself can be specified with some linguistic uncertainty (U3). As to the minor premise, since it refers explicitly to a given assertion, it can be associated with uncertainty about assertions (U2). Actually, the critical question CQ3 refers to the minor premise and its statement “Did a assert that A is true?” is, in fact, ambiguous as far as the type of uncertainty is concerned. On the one hand it might raise a doubt about the fact that a did actually make any assertion about A, on the other hand it might raise a doubt about the contents of the assertion made by a. For instance, a might have made a weaker assertion, like “A is probably true”, or a completely different assertion like “A is false”. The three alternatives mentioned above are rather different and involve different uncertainty types. The possibility that a made a weaker assertion is a case of U2, while if a made a completely different assertion (or no assertion at all) about A, the entire minor premise is challenged, and this amounts to be uncertain about the credibility of the (implicit) source from which we learned that “a asserted that A is true”, hence a case of U1. As this ambiguity is present in the current formulation of the scheme, we leave it unresolved and indicate both types of uncertainty for the minor premise and CQ3.
This leads to reformulate APK as follows: Major Premise: fSource a is in a position to know about things in a certain subject domain Sg[U1] fcontaining proposition
Ag[U3].
Minor Premise: fa asserts that A (in domain S) is true (false)g[U1; U2].
Conclusion: fA is true (false)g[DU].
CQ1: fIs a in a position to know whether A is true (false)?g[U1] CQ2: fIs a an honest (trustworthy, reliable) source?g[U1] CQ3: fDid a assert that A is true?g[U2; U1].
Let us now consider the ACE scheme. Its first premise is a causal generalization, which, as suggested by the use of (might) in its original formulation, is not always valid. In our simple classification this can be regarded as a form of linguistic uncertainty inside the statement (U3). This kind of uncertainty may also affect the actual formulation of the statements A and B in the instantiations of the scheme. The major premise is challenged by CQ1 and CQ2. While their interpretation allows some overlap, CQ1 seems to concerns the “strength” of the causal generalization as it is formulated, while CQ2 refers to the implicit evidential source of knowledge supporting the causal generalization. Accordingly, CQ1 may be referred to U3, while CQ2 to U1.
The minor premise concerns the observation of a fact (the occurrence of A), that might involve linguistic uncertainty U3. Indeed, also the observation of the occurrence of A might have a source, so that, in principle, the second premise might be affected by U1, and one might have an additional critical question CQ+ like “Does A actually occur?”, which would turn out very similar in nature to CQ3 in the APK scheme. The fact that a question like CQ+ is not considered in this scheme, points out a further non-uniformity in the formulation of argument schemes: one may wonder why a sort of explicit confirmation of the minor premise is required by a critical question in the APK scheme, while the same kind of confirmation is not required in the ACE scheme. While one might answer that similar questions may have a different importance in different schemes, we suggest that a further analysis is needed to address these issues in a systematic way and that a classification of uncertainty types can be very useful in this respect. To point out this, we add CQ+ in the revised version of the ACE scheme, with the relevant uncertainty type U1 associated with the minor premise. Finally, CQ3 raises the question about possible other factors interfering with the causal relation between A and B, i.e. suggests the presence of possible exceptions in the application of the scheme. This kind of uncertainty is not encompassed in our simplistic preliminary classification, hence we let it unspecified (denoted as [??]), as a pointer to future developments. This leads to reformulate ACE as follows:
Major Premise:fIf A occurs, then B will occurg [U1; U3].
Minor Premise: fIn this case, A occursg [U1; U3].
Conclusion:fTherefore, in this case, B will occurg [DU ].
CQ1: fHow strong is the causal
generalization?g[U3] CQ2: fIs the evidence cited (if there is any) strong enough to warrant the casual generalization?g[U1] CQ+: fDoes A actually occur?g[U1] CQ3: fAre there other causal factors that could interfere with the production of the effect in the given case?g[??] 5
In recent years, the issue of combining
explicit uncertainty representation and
argumentation has received increasing attention, with
several works dealing in particular with probabilistic
argumentation
The research reported in this paper was partially supported by the National Science Foundation of China (No.61175058), and Zhejiang Provincial Natural Science Foundation of China (No. LY14F030014).