English. In this paper we propose an approach to irony detection based on Appraisal Theory(Martin and White(2005)) in Shakespeare's Sonnets, a well-known data set that is statistically valuable. In order to produce meaningful experiments, we created a gold standard by collecting opinions from famous literary critics on Shakespeare's Sonnets focusing on irony. We started by manually annotating the data using Appraisal Theory as a reference theory. This choice is motivated by the fact that Appraisal annotation schemes allow smooth evaluation of highly elaborated texts like political commentaries. The annotation is then automatically compiles and checked against the gold standard in order to verify the persistence of certain schemes that can be identified as ironic, satiric or sarcastic. Upon observation, irony detection reaches a final match of 80%1.
Italiano. In questo articolo si propone un approccio basato sulla Appraisal Theory per l’individuazione dell’ironia nei Sonetti di Shakespeare, un dataset che è statisticamente valido. Allo scopo di produrre esperimenti significativi, abbiamo creato un gold standard raccogliendo le opinioni di famosi critici letterari sullo stesso corpus, con l’ironia come tema. Abbiamo poi annotato manualmente i sonetti utilizzando gli strumenti e i tratti della Appraisal Theory che permettono di ottenere una valutazione di testi altamente elaborati come gli articoli di politica. L’annotazione è 1Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) stata poi raccolta automaticamente e confrontata con il gold standard per verificare la persistenza di certi schemi che possono essere identificati come ironici, satirici o sarcastici, raggiungendo una corrispondenza finale del 80%.
Shakespeare’s Sonnets are a collection of 154 poems which is renowned for being full of ironic content (Weiser(1983)), (Weiser(1987)) and for its ambiguity thus sometimes reverting the overall interpretation of the sonnet. Lexical mbiguity, i.e. a word with several meanings, emanates from the way in which the author uses words that can be interpreted in more ways not only because inherently polysemous, but because sometimes the additional meaning meaning they evoke can sometimes be derived on the basis of the sound, i.e. homophone (see “eye”, “I” in sonnet 152). The sonnets are also full of metaphors which many times requires contextualising the content to the historical Elizabethan life and society. Furthermore, there is an abundance of words related to specific language domains in the sonnets. For instance, there are words related to the language of economy, war, nature and to the discoveries of the modern age, and each of these words may be used as a metaphor of love. Many of the sonnets are organized around a conceptual contrast, an opposition that runs parallel and then diverges, sometimes with the use of the rhetorical figure of the chiasmus. It is just this contrast that generates irony, sometimes satire, sarcasm, and even parody. Irony may be considered in turn as: what one means using language that normally signifies the opposite, typically for humorous or emphatic effect; a state of affairs or an event that seems contrary to what one expects and is amusing as a result. As to sarcasm this may be regarded the use of irony to mock or convey contempt. Parody is obtained by using the words or thoughts of a person but adapting them to a ridiculously inappropriate subject. There are several types of irony, though we select verbal irony which, in the strict sense, is saying the opposite of what you mean for outcome, and it depends on the extralinguistics context(Attardo(1994)). As a result, Satire and Irony are slightly overlapping but constitute two separate techniques; eventually Sarcasm can be regarded as a specialization or a subset of Irony. It is important to remark that in many cases, these linguistic structures may require the use of nonliteral or figurative language, i.e. the use of metaphors. This has been carefully taken into account when annotating the sonnets by means of Appraisal Theory Framework (hence ATF). In our approach we will follow the so-called incongruity presumption or incongruity-resolution presumption. Theories connected to the incongruity presumption are mostly cognitive-based and related to concepts highlighted for instance, in (Attardo(2000)). The focus of theorization under this presumption is that in humorous texts, or broadly speaking in any humorous situation, there is an opposition between two alternative dimensions. As a result, we will look for contrast in our study of the sonnets, produced by the contents of manual classification. The purpose of this study is to show how ATF can be useful for detecting irony, considering its ambiguity and its elusive traits. 2
In order to produce a gold standard that may encompass strong hints to classification in terms of humour as explained above, we collected literary critics’ reviews of the sonnets. We used criticism from a set of authors including (Frye(1957)) (Calimani(2009)) (Melchiori(1971)) (Eagle(1916)) (Marelli(2015)) (Schoenfeldt(2010)) (Weiser(1987)) (Serpieri(2002)) all listed in the reference section. The gold standard classification has been produced by second author and checked by first author. It is organized into a number of separate fields in a sequence to allow the reader to get a better picture of the sonnet in the collection. All classifications are reported in a supplementary file in the Appendix. Here below are the classifications for two sonnets:
SEQUENCE:
man urged to reproduce METAPHOR: Through progeny the young man will not be alone NEG.EVAL: The young man seems to be disinterested POS.EVAL: Young man positive aesthetic evaluation CONTRAST: Between one and many
SEQUENCE: 18-86 Time and Immortality MAIN THEME: Love ACTION: The Young man must understand the sincerity of poet’s love METAPHOR: True love is sincere NEG.EVAL: The young man listens the false praise made by others POS.EVAL: Young Man positive aesthetic evaluation CONTRAST: Between true and fictitious love As can be seen, we indicate SEQUENCE for the thematic sequence into which the sonnet is included; this is followed by MAIN THEME which is the theme the sonnet deals with; ACTION reports the possible action proposed by the poet to the protagonist of the poem; METAPHOR is the main metaphor introduced in the poem sometimes using words from a specialized domain; NEG.EVAL and POS.EVAL stand for Negative Evaluation and Positive Evaluation contained in the poem in relation to the theme and the protagonist(s); finally, CONTRAST is the key to signal presence of opposing concrete or abstract concepts used by Shakespeare to reinforce the arguments purported in the poem. Many sonnets have received more than one possible pragmatic category. This is due to the difficulty in choosing one category over another. In particular, it has been particular hard to distinguish Irony from Satire, and Irony from Sarcasm. Overall, we ended up with 54 sonnets receiving a double marking over 98, representing the total number of sonnets with some kind of pragmatic label by the literary critics, with a ratio of 98/154, corresponding to a percentage of 63.64%. We ended up with the count of annotated sonnets reported above in Table 1.
Eventually, as commented in the section below, the introduction of annotations based on Appraisal Theory has helped in choosing best pragmatic classification. In fact, literary critics were simply hinting at "irony" or "satire", but the annotation gave us a precise measure of the level of contrast present in each of the sonnets regarded generically as "ironic". The experiment we have been working on is an attempt to describe irony, parody and sarcasm in terms of a strict scientifically viable linguistic theory, the Appraisal Framework Theory (Martin and White(2005)), as has already been done in the past by other authors (see (Taboada and Grieve(2004)) (Read and Carrol(2012)) but also (Stingo and Delmonte(2016)) (Delmonte and Marchesini(2017)) . The idea is as follows: produce a complete annotation of the sonnets using the tools made available by the theory and then verify how well it fits into the gold standard produced. The primary purpose of the Appraisal Framework Theory(hence AFT) is to delineate the interpersonal dimension of communication, supplying schemes by which it is possible to recognize evaluative sequences within texts and information about the positioning of the author in relation to evaluated targets.2
The annotation has been organized around only one category, Attitude, and its direct subcategories, in order to keep the annotation at a more workable level, and to optimize time and space in the XML annotation. Attitude includes different options for expressing positive or negative evaluation, and expresses the author’s feelings. The main category is divided into three primary fields with their relative positive or negative polarity, namely: Affect is every emotional evaluation of things, processes or states of affairs, (e.g. like/dislike), it describes proper feelings and any emotional reaction within the text aimed towards human behaviour/process and phenomena.
2Further information can be found on the dedicated website dedicated to the Appraisal Framework Theory: http://www.languageofevaluation.info/appraisal/
evaluation of things, processes and state of affairs (e.g. beautiful/ugly; useful/useless), and represent any aesthetic evaluation of things, both man-made and natural phenomena.
Eventually, we end up with six different classes: Affect positive, Affect Negative, Judgement Positive, Judgement Negative, Appreciation Positive, Appreciation Negative. Overall in the annotation there is a total majority of positive polarities with a ratio of 0.511, in comparison to negative annotations with a ratio of 0.488. In short, the whole of the positive poles is 607, and the totality of the negative poles is 579 for a total number of 1186 annotations. Judgement is the more interesting category because it allows social moral sanction, in that it refers to two subfields, Social Esteem and Social Sanction - which however we decided not to mark. In particular, whereas the positive polarity annotation of Judgement extends to Admiration and Praise, the negative polarity annotation deals with Criticism and Condemnation or Social Esteem and Social Sanction (see (Martin and White(2005)), p.52). In particular, Judgement is found mainly in the final couplet of the sonnets.
The annotation work on the texts has been accomplished by first author and checked by second author. Given the level of objective difficulty in understanding the semantic content of the sonnets, we have decided not to resort to additional annotators - second author produced the annotation as part of his Master thesis work. So far, we have not been able to produce a measure for interannotator agreement: however, since I was obliged to correct 35% of all annotations that measure could be approximated by 65% of agreement. The tags we used for the annotation include a tag for <text> contains the whole text of the sonnet; <p> to mark stanzas, and <s> to mark lines. Focusing on the annotation of the evaluative sequences instead, every time we found an evaluative word (or sequence of words), we delimited the item/phrase within the tags <apprsl></apprsl>. Subsequently, following the general indications mentioned above provided by (Martin and White(2005)), we assigned one of the three subcategories – affect, judgement and appreciation – as an attribute of the tag <apprsl>, also providing the positive/negative sentiment orientation as a value of the attribute. Here below we show the annotation for Sonnet 40 which is highly contrasted: <?xml version="1.0" encoding="ISO-88591"?> <text> <p> <s> Take all my loves, my love, yea take them all, </s> <s> What hast thou then more than thou hadst before? </s> <s> No love, my<apprsl affect="positive">love,</apprsl>that thou mayst<apprsl appreciation="positive"> true</apprsl>love call, </s> <s> All mine was thine, before thou hadst this more: </s> </p> <p> <s> Then if for my<apprsl affect="positive">love,</apprsl>thou my<apprsl affect="positive">love</apprsl>receivest, </s> <s> I cannot<apprsl judgment="negative">blame</apprsl>thee, for my<apprsl affect="positive">love</apprsl>thou usest, </s> <s> But yet<apprsl judgement="negative">be blamed,</apprsl>if thou thy self<apprsl judgement="negative">deceivest</apprsl> </s> <s> By<apprsl appreciation="negative">wilful</apprsl>taste of what thy self<apprsl appreciation="negative">refusest</apprsl> </s> </p> <p> <s> <apprsl judgement="positive">I do forgive</apprsl><apprsl judgement="negative">thy robbery</apprsl> <apprsl appreciation="positive">gentle</apprsl> thief </s> <s> textbfAlthough<apprsl judgement="negative">thou steal thee all my poverty:</apprsl> </s> <s> And yet love knows it is a<apprsl affect="negative">greater grief</apprsl> </s> <s> To<apprsl appreciation="negative">bear love’s wrong,</apprsl>than<apprsl appreciation="negative">hate’s known injury</apprsl>. </s> </p> <p> <s> <apprsl appreciation="negative">Lascivious</apprsl>grace, in whom <apprsl appreciation="negative">all ill</apprsl> well shows, </s> <s> Kill me with<apprsl affect="negative">spites</apprsl>yet <apprsl judgement="positive">we must not be foes</apprsl>. </s> </p> </text> In the choice of which and how many items to annotate, we adopted the following linguistic criteria to enhance the notational analysis.
Anytime one or more verb/noun modifiers are found, when they do not represent meaningful evaluation by themselves, they are annotated together with the part of speech that they contribute to modify. Any instance of evaluation of a multiword expression, is annotated as a single appraisal unit. Any instance of evaluation of rhetorical or figurative language, is annotated as a single appraisal unit. When possible, the evaluations are embedded so as to include appraisal units into a bigger evaluative unit, in order to fully capture figures of speech such as oxymora, apagoges, rhetorical questions, interjections and the like.
Without exceeding the length of the proposition, it is allowed to annotate phrases as single appraisal unit up until a clause-level, whenever they express opinions or evaluations. Additionally, for those cases where complex phrasal structures were found, we limited ourselves to the annotation of the most evaluative part within the overall sequence, so as to avoid overproduction of long annotation. Again, when possible, the clauses have been de-structured so that through embedding we were able to capture the evaluation on a clause-level in greater detail. It is allowed to annotate evaluative sequences on a clause level even beyond the punctuation marks limits. However, these annotations are very rare. In case of dyad/triad of items, whenever they share the same attribute and the same polarity orientation, they are annotated as single evaluative units. In case of more than three items in a row that share the same attribute and the same polarity orientation, they were annotated separately.
As to interpretation criteria, we assumed that sonnets with the highest contrast could belong to the category of Sarcasm. The reason for this is justified by the fact that a high level of Negative Judgements accompanied by Positive Appreciations or Affect is by itself interpretable as the intention to provoke a sarcastic mood. As a final result, there are 44 sonnets that present the highest contrast and are specifically classified according to the six classes above (see Figure 1 in the Appendix). There is also a group that contains ambiguous sonnets which have been classified with a double class, mainly by Irony and Sarcasm. As a first remark, in all these sonnets, negative polarity is higher than positive polarity with the exception of sonnet 106. In other words, if we consider this annotation as the one containing the highest levels of Judgement, we come to the conclusion that possible Sarcasm reading is mostly associated with presence of Judgement Negative and in general with high Negative polarity annotations (see table 2 below). As a first result, we may notice a very high convergence existing between critics’ opinions as classified by us with the label highest contrast and the output of manual annotation by Appraisal classes.
In the group of 50 sonnets classified, mainly or exclusively, with Irony, the presence of Judgement Negative is much lower than in the previous table for Sarcasm (see Figure 2 in the Appendix). In fact only half of them – 25 – has annotation for that class, the remaining half introduces two other negative classes: mainly Affect Negative, but also Appreciation Negative - see table 3 below. As to the main Positive class, we can see that it is no longer Judgement Positive, but Appreciation Positive which is present in 33 sonnets. This is followed by Affect Positive which is better distributed.
In other words we can now consider that Sarcasm is characterized by a majority of negative evaluations 224 over 141; while Irony is characterized by a majority of Positive evaluations 262 over 183 and that the values are sparse and unequally distributed. The final table concerns the number of sonnets with blank evaluation by critics which amount to 60. As a rule, this group of sonnets look different from the two groups we already analysed. The prevailing trait is Affect Negative; Judgement Negative is only occasionally present; the second preminent trait is Affect Positive. In order to know how much the difference is, we can judge from the quantities shown in table 3 above (but see also Figure 3 in the Appendix).
In particular, in this case the ratio Negative/Positive is more balanced 226 over 176 with a majority of Positive annotations as happened with Irony but with a lower gap. The appraisal category with highest number of annotations is now Affect, whereas in the case of Irony it was Appreciation, and in Sarcasm it was Judgement. So eventually we have been able to differentiate the three main and more frequent pragmatic categories by means of Appraisal Framework features: they are characterized by a different distribution of positive vs. negative evaluations and also by a prominent presence of one of the three main subcategories into which Appraisal has been subdivided that is Appreciation for Irony, Judgement for Sarcasm and Affect where no evaluation has been expressed. 3
In this paper we have presented work carried out to annotate and experiment with the theme of irony in Shakespeare’s Sonnets. The gold standard for the experiment has been created by collecting comments produced by literary critics on the presence of some kind of thematic, semantic and syntactic opposition in the sonnets as to produce some sort of irony. At first the sonnets have been annotated using the framework of Appraisal Theory and then we checked the results: we obtained a very high level of matching with the critics’ opinions at 80%. Eventually, Appraisal framework has shown its ability to classify and diversify different levels of irony effectively. Northrop Frye. 1957. Anatomy of Criticism: Four Essays. Princeton University Press.
Maria Antonietta Marelli. 2015. William Shakespeare,
I Sonetti – con testo a fronte. Garzanti.
J. Martin and P.R. White. 2005. Language of Evaluation, Appraisal in English. Palgrave Macmillan, London and New York.
Giorgio Melchiori. 1971.
Adriatica Editrice, Bari.
Shakespeare s Sonnets.
J. Read and J. Carrol. 2012. Annotating expressions of appraisal in english. Language Resources and Evaluation, 46:421–447.
Michael Schoenfeldt. 2010. Cambridge introduction to Shakespeare’s poetry. Cambridge University Press, Cambridge.
Alessandro Serpieri. 2002. Polifonia Shakespeariana.
Bulzoni, Roma.
Michele Stingo and Rodolfo Delmonte. 2016. Annotating satire in italian political commentaries with appraisal theory. In Natural Language Processing meets Journalism - Proceedings of the Workshop, NLPMJ-2016, pages 74–79, Stroudsburg, PA, USA. ACL.
David K. Weiser. 1987. Mind in Character – Shakespeare s Speaker in the Sonnets. The University of Missouri Press.