There is a driving need to interrogate large bodies of text for pragmatic meaning, e.g., to detect sentiment, diagnose genre, plot chains of reasoning, and so forth. But this type of meaning is often implicit, 'hidden' meaning, evoked by linguistic cues, stylistic arrangement, or argumentation structurefeatures that have hitherto been difficult for Natural Language Processing (NLP) systems to recognize and use. Pragmatic concerns were historically the province of rhetorical studies, and we have turned to rhetoric in order to find new solutions to computational pragmatics. This paper highlights a form of rhetorical device that encodes deep levels of pragmatic meaning and yet lends itself to automated detection. These devices are the linguistic configurations known as rhetorical figures, which have been poorly understood and vastly underutilized in Computational Linguistics and Computational Argumentation. We present an annotation scheme using XML for rhetorical figures to make figuration more tractable for NLP, enhancing applications for argument mining, along with a range of other tasks. We also discuss the intellectual and technical challenges involved in figure annotation and the implications for Machine Learning.
Rhetorical figures are cognitively governed linguistic devices that serve functional, mnemonic, and aesthetic purposes. Take the famous maxim from Kennedy's inaugural address: 1.!Ask not what your country can do for you. Ask what you can do for your country. [Kennedy (and
Sorensen) 1961] This expression quickly became proverbial in the American consciousness for the way it captures the spirit of a particular historical moment, the ethos of a particular administration, and the aspirations of a particular generation. Countless more prosaic formulations, by Kennedy and others, expressed that confluence too, but they left a distinctly less memorable impression. Why? Two reasons. Firstly, the formal structure and the functional structure are virtually isomorphic: Kennedy (and speechwriter Ted Sorensen) expressed the rejection of one civic attitude and its replacement by the opposite one, in the iconicity of reversing the terms of reference. Secondly, that very snug form/function coupling inhabits a material structure that is, on its own, cognitively very sticky. The Kennedy-Sorensen phrase has become so widely known, that is, so easily shared, so frequently invoked and quoted and recited because of (1) the schematic congruence with which the form matches the Rejection-Replacement function its arrangement serves, and (2) the cognitive affinities humans have for its structural properties (opposition, repetition, and symmetry).
The cognitive affinities explain its mnemonic and aesthetic effects, but, an interest in Computation Argumentation scholars focuses attention on its tight form-functional correlation, in an approach known as figural logic. The form makes it tractable for automated detection, while the function gives us its rhetorical purpose. In terms of argument mining, an application that accessed this correlation could epitomize Kennedy's inaugural address (which argued for the rejection of an ethos of entitlement and its replacement by an ethos of duty) virtually on the basis of this expression alone.
We are developing an approach to computational pragmatics that combines the insights for argumentation that rhetorical figures provide, together with argument mining, corpus linguistics, and machine learning, with payoffs for both computer science and for rhetoric. There has to this point been success at detecting some rhetorical figures, but little sense of what to do with them once they have been detected.
There has been a growing interest in the convergence of
rhetoric, argumentation, and NLP, sparked by such works as
Our approach is a more sophisticated use of rhetorical figures than has been attempted, operating at layers of formal and functional abstraction. It depends fundamentally on an annotation format for rhetorical figures.
In this paper we argue for the importance of rhetorical figures for NLP generally and argument mining specifically; we identify the challenges and opportunities of integrating a knowledge of figures into NLP; and, most specifically, we offer an XML annotation scheme for rhetorical figures that meets some of these challenges and therefore opens up new opportunities for NLP. 2
Computationally, figures are important for four central reasons. First, they are endemic to human language. This is very well established for a few tropes, such as metaphor, which is the central focus of Cognitive Linguistics and deeply entrenched in ontologies like FrameNet and WordNet. But it is equally true of literally (a word we don't use lightly) hundreds of other figures. If we want languageperceptive algorithms, they must have knowledge of figure structure. Secondly, figures epitomize argument structure, increasingly a prime concern for NLP. Again, this is well understood for metaphor (and simile, though it gets much less overt attention), which epitomize analogic argumentation. Thirdly, many figures (especially the ones called schemes) work in terms of formal patterns that algorithms can detect through surface analysis; our Sentence 1 illustrates this aspect clearly. Fourthly, they correlate with rhetorical functions (pragmatic and argumentative meaning). We will illustrate this shortly. For now, the rejectionreplacement function of Sentence 1 will have to stand.
The contemporary scholar most responsible for the
position that rhetorical figures are constructions with
especially tight couplings of form and function is Jeanne
Fahnestock, whose figural logic is brilliantly articluated in
Rhetorical Figures in Scientific Argumentation [1999; see
also
But figures are not without their challenges for Natural
Language Processing. Metaphor remains elusive, for
instance, despite all the attention it has attracted in cognitive
science, AI, and linguistics, including Computational
Linguistics, in the last two decades. Metaphor is a type of figure
known as a trope, which depends on semantic deviation. We
are not yet successful enough with straight-laced semantics
to support forays into semantic distortions. Some tropes
(such as oxymoron, which is a juxtaposition of antonymic
terms, such as square circle or deafening silence) can be
reliably detected [Gawryjolek 2009]. We believe antithesis
(juxtaposed opposite predications, as in Sentence 2, a
double antithesis) has a similar potential for reliable detection.
(We adopt the convention of identifying the defining
figurative elements parenthetically.)
2.! The young would choose an exciting life; the old a
happy death. (young, old; life, death) [
Another type of figure, schemes, are formal deviations,
shifts of expected structure, as in Sentence 1, an
antimetabole (reverse lexical repetition; in this case you and your
country). The computational detection of figures, including
antimetabole, is finding success [Gawryjolek 2009;
Gawryjolek, Harris, and DiMarco 2009; Hromada 2011;
The work of these researchers is sometimes only
loosely connected to the rhetorical traditions. Many of them, too,
only concerned detection—an essential first step but one
that doesn't get us very close to argument mining. They did
not attempt to find meaning in the figures they detected.
Gawryjolek [2009], Hromada [2011], Dubremetz and Nivre
2 As
Antimetabole has a small set of rhetorical functions, keyed to the iconicity of its formal structure (which evokes balance and opposition, as well as sequence or priority). We have very limited space in this paper to demonstrate these rhetorical functions, so a few examples will have to suffice.
One function of antimetabole is to convey Reciprocal
Force, illustrated by Sentence 3, Newton's third law of
motion. (We adopt the convention of identifying the defining
figurative elements parenthetically.)
3.!If you press a stone with your finger, the finger is
also pressed by the stone. (stone / finger) [
A very similar rhetorical function of antimetabole is to
convey Reciprocal Specification, a kind of mutual
definition, illustrated by Sentence 4:
4.!Gay rights are human rights, and human rights are
gay rights. (human rights / gay rights) [
Another rhetorical function of the antimetabole is to convey Comprehensiveness, illustrated by the ordinarylanguage example, Sentence 5: 5.!A place for everything, and everything in its place.
(place / everything) [Traditional] The reverse repetition in Sentence 5 shifts from reciprocal force to a reciprocal coverage, largely because it has prepositional predication rather than the transitive predication of Newton's Sentence 3. We call this function comprehensiveness because the sequential iconicity means a back-andforth, alpha-to-omega, omega-to-alpha coverage of some domain—in this case, the domain of tidiness. All things have assigned places; all places have their assigned things.
A fourth rhetorical function of the antimetabole is to convey Irrelevance-Of-Order, well known from algebra and predicate calculus: 6.!m + n = n + m (m / n) [Traditional; commutative principle] There are other ways to express the principle of commutation, but none as natural and iconic as formulae like 6. Opposite sequences of the same variables, on either side of the same operator, pivoted by a predication of identity, equivalence, or equality inescapably means that neither sequence
We have built a curated list of over 400 antimetaboles illustrating these functions, but only have space for a few more representative examples:
7.! A corollary of PHC [the Principle of Hierarchical
Coincidence] is that resources flow toward political
power, and political power flows toward resources;
or, the power of state and of capital typically appear
in conjunction and are mutually reinforcing.
(resources / political power) [
9.!The negation of a conjunction is the disjunction of the negations and the negation of a disjunction is the conjunction of the negations. (negation of a conjunction / disjunction of the negations) [De Morgan's law; traditional] 10.! Anger and depression, the pop-psych books tell us, are two sides of the same coin: depression is anger suppressed, anger is depression liberated. (depression / anger) [Hertzberg 2008]
11.! I meant what I said and I said what I meant.
(meant / said) [
13.! With a similar qualification, in the Cambridge
Grammar of the English Language, a head 'plays
the primary role' in 'determining the distribution of
the phrase' (introductory chapter signed by Pullum
and Huddleston, in Huddleston and Pullum
2002:24) (Pullum / Huddleston) [
Again, however, there are challenges. They are not as thorny as the challenges of most tropes because they concern surface analysis, not semantic plumbing. But they exist. In particular, figures rarely come in isolation. The KennedySorenson maxim, for instance (Sentence 1), is an antimetabole (you / your country). But it is also an antithesis (ask not X / ask X). It is, thirdly, a mesodiplosis (clause-medial repetition; here, can do occurs in the middle of both clauses).
We call this phenomenon, when figures co-occur and mutually reinforce each other, stacking. It presents both a challenge and an opportunity. It is a challenge because rather than detecting a single figure or multiple independent figures, we need to detect overlapping figures. It is an opportunity because the functions are enhanced and stabilized under stacking. When two or more figures coincide in the same utterance, the functions they convey are highly consistent. Formal stacking breeds a functional conspiracy.
For instance, when antimetabole stacks with antithesis
(conjoined or highly proximal opposite predications), the
joint function is primarily to reject the negated predication
utterly and replace it with the positive predication. Again,
Sentence 1 is our paradigm, but here are two more:
Reject-Replace
15.! We don't build services to make money; we make
money to build better services. (services / money)
[Mark Zuckerberg, qtd in
(plain statement / metaphor) [
We do not pretend to have a full and complete mapping of form to function, however. This work is still in the very early stages, but we believe it holds considerable promise, and we believe machine-learning corpus studies can be extremely helpful, especially for the challenges and opportunities of stacking.
Figural stacking, as we come to understand the functional combinatorics better, is perhaps the greatest promise of rhetorical figures for computational understanding of natural language. Our paradigm example, which stacks the schemes antimetabole, mesodiplosis (both entailing ploche), and the trope antithesis provides a pitch-perfect example of the rhetorical function, Reject-Replace. A computational analysis of Kennedy's inaugural address tuned to the workings of rhetorical figures could tell us what the address was about—namely, the rejection of an ethos of entitlement and its replacement with an ethos of responsibility—virtually on the basis of this particular stacking (along with, of course, the lexical semantics of you, your country, and so on)
We can, and should, rely on rhetoricians to tell us what the functions of certain figures and certain figure-stacks are, at least in these early stages. But the rhetorical tradition is haphazard, and sometimes conflicting. The terminology alone is forbidding. As much as computational argument studies can benefit from a better understanding of rhetorical figures, rhetorical figures can benefit from computational studies of form and meaning. (And, yes, that sentence was an antimetabole, stacked with mesodiplosis; the rhetorical function is Reciprocal Force, modulated by the possibility modality of can.)
The path forward is to bootstrap rhetoricians' knowledge by way of annotation, marked-up text corpora, and machine learning, so that computationally mined data can start to tell them what functions figures have, through confirmation, through refinement, and through new discoveries, all of which we have good reason to anticipate.
We can discover the proportionality of certain stackings
(anecdotally, both antithesis and mesodiplosis strongly
cooccur with antimetabole), the correlation of the stackings
with the rhetorical functions (as specified above, on the
basis of limited and anecdotal research). At its best, this work
can revolutionize Computation Argumentation studies and
rhetoric in the way corpus linguistics revolutionized
lexicography and established ontologies like WordNet and
Framenet. But even at its least productive, we are very
confident of finding important form/function correlations that
can importantly inform Computation Argumentation and
discourse studies, in novel ways.
3
There have been limited successes in figure detection over
the past several years due to strict figure mappings and
some unreported data [Gawryjolek 2009; Gawryjolek,
Harris, and DiMarco 2009; Hromada 2011;
Hromada's [2011] work, for instance, was very
successful at the detection of antimetabole, but he defined
antimetabole in an overdetermined way. Using the Waterloo Figure
Representation Notation [Harris and DiMarco 2009]3
(where W stands for Word, the subscripts indicate identity,
and "…" represents other linguistic matter, extraneous to the
figure, possibly null), Hromada defines antimetabole as
<WA WB WC … WC WB WA >, whereas a more accurate
definition
Most of these researchers did not look for stacked
figures, except accidentally. Hromada [2011] looked for other
figures (anadiplosis, epanaphora, and epiphora), but only in
isolation.4 Conversely, he 'searched' for mesodiplosis
unwit3 Hromada [2011] calls this notation, Rhetoric Figure
Representation Formalism or RFRF, which he adapts from Harris and
DiMarco [2009]. Harris and DiMarco did not label their formalism
in their paper, but we use their term for it here. The WFRN is a
formalism for the general structure of rhetorical schemes, but it
does not represent functions at all. For this we need a richer
system, which may be provided by Construction Grammar (e.g.,
Hoffmann and Trousdale 2013). For an argument to this effect, see
[
And, of course, detecting rhetorical figures is the beginning of the story. We know, from millennia of humanistic research, that linguistic forms correlate with rhetorical functions—that figures do communicative work beyond 'mere aesthetics'—and we can thank Fahnestock for collating and expanding this research so clearly in the contemporary era. On the basis of this research, we can use the detected figures to help chart meanings—sometimes very fundamental meanings, like the Reject-Replace antithetical antimetabole of Example 1, which diagnoses the exact tenure of Kennedy's inaugural address.
But how well do the form-function couplings that humanists have found stand up beyond the small sampling of discourse that humanists have been able to explore—in the conversations, news stories, opinion pieces, blogs, review articles, short stories, tweets, scientific arguments, and so on, that populate the vast sea of everyday and specialist human discourse? We don't know, but corpus studies should tell us. Do Reciprocal Force antimetaboles collate with transitive verbs, for instance? Do Reciprocal Specification and Subcategorization antimetaboles collate with copulas? Do Irrelevance-of-Order antimetaboles collate with conjunctions and disjunctions? How frequently does mesodiplosis collate with antimetabole? What other stackings are there, with what functional implications? We have intuitions, and much particularized research (that is, specific works of rhetorical criticism), but intuitions and particularized research need to be tested on copora.
How do figures cluster in terms of genres? Do individual authors have identifiable figure proclivities? Is sentiment a trigger for certain figures? Do certain argument types favour certain figures? Are there author-genre figural effects? Argument-sentiment figural effects? Author-sentiment? Again, intuitions and particularized research suggest answers; again, these need to be tested.
When multiple figures co-occur, as they almost always
do, which functions stack, which remain independent, which
configurations corresponding to the same label, and with some
linguistic activity that really isn't figurative labeled as figures. The
taxonomy of figures is, in short, a mess. We have developed a
much more rigorous, consistent, and principled taxonomy of
figures at Waterloo. See
This work can undoubtedly be strengthened by machine learning. We have developed a format for annotating rhetorical figures, in parallel to the annotation formalisms developed for part-of-speech tagging, speech-act annotation, and so on. Corpora annotated with rhetorical figures can be used to train systems on new and more sophisticated detection tasks, especially for stackings and functional correlations. 4
We want to come at the detection problem for rhetorical
figures from the other end. There is a "serious bottleneck …
[from] the lack of annotated data" [
The Extensible Markup Language (XML) is widely used for annotations and we apply it here to rhetorical figures. The main challenges of using such an annotation scheme is in the intricacies that figure-rich texts present. These intricacies include stacking figures and interpenetrating figures. The annotation methods developed in this paper address these two issues. The desire is to develop an annotation scheme that will highlight the structure of rhetorical figures allowing them to be more easily understood by computational learning-based algorithms while keeping figures intact. Now, using XML we analyze the development process of a suitable markup.
We have used HTML in the past for annotating figures—specifically, JANTOR (Java ANnotation Tool Of Rhetoric) allowed for "manual and automated annotation of files in HTML format" [Gawryjolek 2009; Gawryjolek, Harris, and DiMarco 2009]—but XML presents such obvious advantages that we have adopted it in our recent work. It is especially valuable for the flexibility it provides in creating one's own tags and attributes.
Our original markup focused on the names of tags and did not include attributes. This is adequate, using a general markup template like the one in 19, for simple instances of isolated ploche, such as 20a (annotated as 20b): b. <antithesis/A/ 1><ploche1>...<antimetabole>! <ploche2>...<mesodiplosis>...</antithesis/A/ 1>! <antithesis/B/ 1>...</ploche1>...</mesodiplosis>! ...</antithesis/B/ 1></ploche2></antimetabole> <example> <antithesis> <antithesis-A-1> <ploche1> <ploche1-A-1>Ask</ploche1-A-1> not what <antimetabole> <ploche2> <antimetabole-A-1>
<ploche2-A-1>your country</ploche2-A-1> </antimetabole-A-1> <mesodiplosis> <mesodiplosis-A-1>can do for</mesodiplosis-A-1> <antimetabole-B-1>you</antimetabole-B-1> </antithesis-A-1> <antithesis-B-1> <ploche1-A2>Ask</ploche1-A2> </ploche1> what <antimetabole-B-2>you</antimetabole-B-2> <mesodiplosis-A-2>can do for</mesodiplosis-A-2>
</mesodiplosis> <antimetabole-A-2>
<ploche2-A-2>your country</ploche2-A-2> </antimetabole-A-2> </antithesis-B-1> </ploche2> </antimetabole> </antithesis> </example>
A syntax issue arises in Example 21b where multiple figure tags close in the incorrect parent tag. For example, we have <antithesis/A/ 1>…<antimetabole>…</antithesis/A/ !>…! <antithe/ sis/A/ B>…</antimetabole>…</antithesis/A/ B>. Figure 3 below shows the other figures that also fall to this error.
The syntax of XML does not allow the interpenetration of tags. When considering this problem, it becomes apparent that the tags marking off the beginning and endings of figures are causing the most trouble. Further analysis reveals that these tags are unnecessary. The key components of a figure are their defining elements such as repeating or contrasting elements (words, sounds).
The semantic complication has to do with nesting XML tags. Arbitrary hierarchies can form when some figures happen to appear inside others. Rhetorical figures may, however, contain other rhetorical figures which do observe hierarchical properties. Thus we require a method that is more explicit about creating hierarchies. This is achievable with the introduction of attributes and thus the creation of a new annotation scheme. <antithesis> </ploche2> </antimetabole> <antithesis-A-1> <antithesis-B-1> <antimetabole> <ploche1> <ploche2> <mesodiplosis> </ploche1> </mesodiplosis> <parent tag> <parent tag> <child tag> </incorrectly_positioned_closing_tag>
Figure 3 displays the complexity of this version of the annotation scheme. The dashed arrows represent the consequences of tagging when you need to mark the end of the antithesis before the end of the antimetabole; there is no hierarchy, or perhaps only a partial and .fragmentary hierarchy, but it creates havoc. The nesting, if we can even call it that, is incomplete, falling outside XML's basic capacities. Hierarchy problems also become apparent as <antimetabole/ element/number>!tags are sub-tags of <ploche>.!
The improved annotation scheme recognizes the above problems and attempts to resolve them. It focuses on highlighting the defining elements of figures. A general markup is shown in number 22 (a fully formatted example for this markup is provided in Figure 5, given between the Conclusion and the Acknowledgements for purposes of layout): 22.! <element! figure='figure1! [figure2]'! lettergroup=! '[figure1/(A! to! Z)...]'! position='[figure1/(1! to! n)...]'>...! text...</element>! If one wanted to create a hierarchy, say in the instance that figure1 always accompanies figure2 meaning figure1 is a subpart of figure2, this is still possible. The XML from the example would look like: <element figure='figure2'> … <element figure='figure1'> … </element> … </element>. This way we now have the option to create or avoid a hierarchy.
As Figure 4 reveals, the improved markup focuses on tagging parts of strings and providing them with more information. The figure focusses on antithesis, antimetabole and ploche, where ploche referes to ploche1. Notice how we are able to combine the antimetabole and ploche tags into one attribute and avoid a hierarchy
Using attributes also helps to separate information about a tag providing algorithms with easier access. The lettergroup attribute grants information on which tags surround the same word or, as the names suggests, groupings of letters. If the letters inside the tag are the same as inside another tag the attribute will end in the same character. The position attribute clarifies the location of the letter group in the figure. For example, antimetabole has two A's in its ABBA structure. To differentiate between them we write the position attribute of the first A as Antimetabole-1 and the second as Antimetabole-2. Using these tags and attributes to annotate rhetorical figures in text would create the required computational structure for figure analysis. 5
The computational uses of rhetorical figures are indisputable. We can clearly see their ability to enhance fields such as author and genre detection, NLP systems, and argumentation mining. We also know how intricate they can become. Stacking and intersecting with one another, many figures can be overlooked as observed in the previous works mentioned here. To exploit their uses, yet overcome their intricacy, a rhetorical figure markup becomes imperative and should be thought of as such.
Our annotation scheme represents the first move in what we hope will be a line of research that others will find profitable to join. The outline of the annotation scheme has been developed, and now the flexibility of XML allows others to improve and customize the mechanism for their own uses. The eventual goal is to develop a markup scheme that provides computationally accessible information for all rhetorical figures.
We would like to thank Cliff O'Reilly for valuable XML advice, as well as our colleagues at the University of Waterloo, including Elena Afros, Adam Bradley, Ashley Kelly, Isabel Li, Ricky Rong, and Terry Stewart; our international colleagues, including Cliff (again), Marie Dubremetz, Jelena Mitrovic, Chris Reed, and James Wynn; and the Social Sciences and Humanities Research Council of Canada for financial assistance. We also thank three anonymous reviewers for CMNA, for their helpful queries and suggestions. Our figure-annotation research is part of an overall project of Computational Rhetoric at the University of Waterloo, organized around a comprehensive OWLbased ontology of rhetorical figures. ing rhetorical figures. Proceedings, CMNA IX (Computational Models of Natural Argument), held with IJCAI-09, Pasadena, CA, July 13.
Gladkova, Olga, Chrysanne DiMarc
Gladkova, Olga, Randy Allen Harris and Chrysanne DiMarco. 2011. Schematic organization of clinical decisionmaking: Findings from qualitative corpus analysis. Proceedings, CMNA XI (Computational Models of Natural Argument), 7 August 11, San Francisco, CA.
Grasso, Floriana. 2002a. Towards a framework for rhetorical argumentation. EDILOG 2002 - Proceedings of the 6th Workshop on the Semantics and Pragmatics of Dialogue, J. Bos, M.E. Foster and C. Matheson (eds), Edinburgh, UK, 46 September 2002, p. 53-60.
Grasso, Floriana. 2002b. Towards computational rhetoric. Informal Logic 29.3: 195-229.
Green, Nancy. 2010. Representation of argumentation in text with Rhetorical Structure Theory. Argumentation 24.2:181-196.
Green, Nancy. 2015. Identifying argumentation schemes in genetics research articles. In Proceedings of the Second Workshop on Argumentation mining, North American Conference of the Association for Computational Linguistics (NAACL), 12-21, Denver, CO, 2015.
Greer, Germaine. 1988. The proper study of womankind. Times Literary Supplement (3-9 June).
Harris, Randy Allen. 2013. Figural logic in Mendel's Experiments on plant hybrids. Philosophy and Rhetoric 46.4: 570-602.
Harris, Randy Allen, and Chrysanne DiMarco. 2009. Constructing a rhetorical figuration ontology. Symposium on Persuasive Technology and Digital Behaviour Intervention, Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB), Edinburgh, Scotland, April.
Hertzberg, Hendrik. 2008. The spat. New Yorker (February 11).
Hoffmann, Thomas, and Graeme Trousdale, eds. 2013. The Oxford handbook of Construction Grammar. New York: Oxford University Press.
Hromada, Daniel Devatman. 2011. Initial experiments with multilingual extraction of rhetoric figures by means of PERL-compatible regular expressions. Proceedings of the Second Student Research Workshop associated with RANLP 2011, Hissar, Bulgaria.
Kanoksilapatham, Budsaba. 2003. A corpus-based investigation of scientific research articles: Linking move analysis with multidimensional analysis. PhD dissertation, Department of Linguistics, Georgetown University.
Unknown author. Unknown date. Western Spanglish language: The United States unofficial language. Western women in leadership and innovation: Discovering the wellsprings of metaphorical voices. http://westernwomenleadershipinnovation.net/westernspanglish-language.html