=Paper= {{Paper |id=Vol-2481/paper26 |storemode=property |title=Detecting Irony in Shakespeare’s Sonnets with SPARSAR |pdfUrl=https://ceur-ws.org/Vol-2481/paper26.pdf |volume=Vol-2481 |authors=Rodolfo Delmonte,Nicolò Busetto |dblpUrl=https://dblp.org/rec/conf/clic-it/DelmonteB19 }} ==Detecting Irony in Shakespeare’s Sonnets with SPARSAR== https://ceur-ws.org/Vol-2481/paper26.pdf
               Detecting Irony in Shakespeare’s Sonnets with SPARSAR

                    Rodolfo Delmonte                                   Nicolò Busetto
              Department of Linguistic Studies                 Department of Linguistic Studies
                   Ca Foscari University                            Ca Foscari University
                   Ca Bembo - Venezia                               Ca Bembo - Venezia
                 delmont@unive.it                               830070@stud.unive.it


                        Abstract                                recitazione della poesia inglese con TTS.
                                                                Il sistema produce una rappresentazione
       English. In this paper we propose a novel                linguistica profonda a livello fonetico, sin-
       approach to irony detection in Shake-                    tattico e semantico. E’ stata usata per in-
       speare’s Sonnets, a well-known data set                  dividuare l’ironia sulla base dell’analisi
       that is statistically valuable. In order to              fonetica e del sentiment. All’inizio la va-
       produce a meaningful experiment, we cre-                 lutazione è stata molto deludente, solo il
       ated a gold standard by collecting opin-                 50% di tutti i sonetti erano inclusi nel gold
       ions from famous literary critics on the                 standard. Poi sulla base della rappre-
       same data focusing on irony. In the ex-                  sentazione semantica prodotta dal sistema
       periment, we use SPARSAR a system for                    a livello proposizionale, è stata messa in
       English poetry analysis and reciting by                  luce la struttura logica del sonetto cal-
       TTS. The system produces a deep linguis-                 colando le relazioni del discorso del dis-
       tically based representation at phonetic,                tico e/o della quartina finale. In questo
       syntactic and semantic level. It has been                modo abbiamo ottenuto un miglioramento
       used to detect irony with a novel approach               dell’accuracy del 17% raggiungendo il
       based on phonetic processing and senti-                  66.88%.
       ment analysis. At first the evaluation was
       very disappointing, only 50% of the son-
       nets matched the gold standard. Even-               1    Introduction
       tually, taking advantage of the semantic
       representation produced by the system at            Shakespeare’s Sonnets are a collection of 154 po-
       propositional level, the logical structure of       ems which is renowned for being full of ironic
       the sonnet has been highlighted by com-             content (Weiser, 1983), (Weiser, 1987) and for its
       puting the discourse relations of the cou-          ambiguity thus sometimes reverting the overall in-
       plet and/or the final quatrain. In this way         terpretation of the sonnet. Lexical ambiguity, i.e.
       we managed to improve accuracy by 17%               a word with several meanings, emanates from the
       up to 66.88%1 .                                     way in which the author uses words that can be
                                                           interpreted in more ways not only because inher-
       Italiano. In questo articolo si propone             ently polysemous, but because sometimes the ad-
       un nuovo approccio per l’individuazione             ditional meaning it evokes is derived on the ba-
       dell’ironia nei Sonetti di Shakespeare, un          sis of the sound, i.e. by homophones (see “eye”,
       dataset che è statisticamente valido. Allo          “I” in sonnet 152). The sonnets are also full of
       scopo di produrre esperimenti significa-            metaphors which many times require contextual-
       tivi, abbiamo creato un gold standard rac-          ising the content to the historical Elizabethan life
       cogliendo le opinioni di famosi critici let-        and society. Furthermore, the sonnets are full of
       terari sullo stesso corpus, con l’ironia            words related to specific language domains. For
       come tema. Nell’esperimento abbiamo us-             instance, there are words related to the language
       ato SPARSAR un sistema per l’analisi e la           of economy, war, nature and to the discoveries of
   1
                                                           the modern age, and each of these words may be
     Copyright c 2019 for this paper by its authors. Use
permitted under Creative Commons License Attribution 4.0   used as a metaphor of love. Many of the son-
International (CC BY 4.0)                                  nets are organized around a conceptual contrast,
an opposition that runs parallel and then diverges,      con basis or on a supervised feature basis where
sometimes with the use of the rhetorical figure of       in both cases, it is just a binary - ternary or graded
the chiasmus. It is just this contrast that generates    - decision that has to be taken. This is certainly not
irony, sometimes satire, sarcasm, and even par-          explanatory of the phenomenon and will not help
ody. Irony may be considered in turn as: what            in understanding what it is that causes humorous
one means using language that normally signifies         reactions to the reading of an ironic piece of text.
the opposite, typically for humorous or emphatic         It certainly is of no help in deciding which phrases,
effect; a state of affairs or an event that seems con-   clauses or just multiwords or simply words, con-
trary to what one expects and is amusing as a re-        tribute to create the ironic meaning (see (Reyes et
sult. As to sarcasm this may be regarded the use of      al., 2012); (Reyes and Rosso, 2013)).
irony to mock or convey contempt.(Attardo, 1994)         We will not comment here on the work done to
Parody is obtained by using the words or thoughts        produce the gold standard which has already been
of a person but adapting them to a ridiculously          described in a separate paper (Busetto & Del-
inappropriate subject. There are several types of        monte, 2019 - To appear) but see all the file in the
irony, though we select verbal irony which, in the       Supplementary materials). We simply say that we
strict sense, is saying the opposite of what you         considered as ironic or sarcastic all sonnets that
mean for outcome, and it depends on the extra-           have been so defined by at least one of the many
linguistics context. It is important to remark that      literary critics’ comments we looked into2 .
in many cases, the linguistic structures on which
irony is based, may require the use of nonliteral or     2    The Architecture of SPARSAR:
figurative language, i.e. the use of metaphors.               Syntax and Semantics
In our approach we will follow the so-called in-
                                                         SPARSAR3 (Delmonte, 2016) builds three rep-
congruity presumption or incongruity-resolution
                                                         resentations of the properties and features of
presumption. Theories connected to the incon-
                                                         each poem: a Phonetic Relational View from the
gruity presumption are mostly cognitive-based
                                                         phonological and the phonetic content of each
and related to concepts highlighted for instance,
                                                         word; a Poetic Relational View where the main
in (Attardo, 2000). The focus of theorization un-
                                                         poetic devices are addressed, related to rhythm
der this presumption is that in humorous texts, or
                                                         and rhyme, and the overall metrical structure; then
broadly speaking in any humorous situation, there
                                                         a Semantic Relational View where the syntac-
is an opposition between two alternative dimen-
                                                         tic, semantic and pragmatic content of the poem
sions. As a result, in our study of the sonnets,
                                                         is represented, at the lexical semantic level, at
produced by the contents of manual classification,
                                                         the anaphoric level and at the predicate-argument
we have been looking for contrasting situations;
                                                         structure. At this level, also the sentiment or over-
while in the sentiment analysis experiment, we
                                                         all mood of the poem is computed on the basis
have been concerned with a quantitative count of
                                                         of a lean lexically based sentiment analysis. The
polarity related items.
                                                         system uses a modified version of VENSES, a se-
Computational research on sentiment analysis has
                                                         mantically oriented NLP pipeline (Delmonte et al.,
been based on the use of shallow features with a
                                                         2005). It is accompanied by a module that works
binary choice to train statistical model (Carvalho
                                                         at sentence level and produces a whole set of anal-
et al., 2009) that, when optimized for a particular
                                                         ysis both at quantitative, syntactic and semantic
task, will produce acceptable performance. How-
                                                         level. As regards syntax, the system makes avail-
ever generalizing the model has proven to be a
                                                         able chunks and dependency structures. Then the
hard task. In addition, the text addressed by re-
                                                         system introduces semantics both in the version
cent research has been limited to tweets, which
                                                         of a classifier and by isolating verbal complex in
are in no way comparable to the sonnets contain
                                                         order to verify propositional properties, like pres-
a lot of nonliteral language. The other common
                                                         ence of negation, to compute factuality from a
approach used to detect irony, in the majority of
the cases, is based on polarity detection(Van Hee            2
                                                               We used criticism from a set of authors including (Frye,
et al., 2018). Sentiment Analysis(Kim and Hovy,          1957) (Calimani, 2009) (Melchiori, 1971) (Eagle, 1916)
                                                         (Marelli, 2015) (Schoenfeldt, 2010) (Weiser, 1987) (Serpieri,
2004) and (Kao and Jurafsky, 2012) is in fact an         2002) all listed in the reference section.
indiscriminate labeling of texts either on a lexi-           3
                                                               the system is freely downloadable from its website
                                                         https://sparsar.wordpress.com/
crosscheck with modality, aspectuality – that is de-     and coherence. The output of this mapping is a
rived from the lexica – and tense. On the other          rich dependency structure, which contains infor-
hand, the classifier has two different tasks: sep-       mation related also to implicit arguments, i.e. sub-
arating concrete from abstract nouns, identifying        jects of infinitivals, participials and gerundives.
highly ambiguous from singleton concepts (from           LFG representation also has a semantic role as-
number of possible meanings from WordNet and             sociated to each grammatical function, which is
other similar repositories). Eventually, the sys-        used to identify the syntactic head lemma uniquely
tem carries out a sentiment analysis of the poem,        in the sentence. Finally it takes care of long dis-
thus contributing a three-way classification: neu-       tance dependencies for relative and interrogative
tral, negative, positive that can be used as a pow-      clauses. When fully coherent and complete predi-
erful tool for prosodically related purposes.            cate argument structures have been built, pronom-
State of the art semantic systems are based on           inal binding and anaphora resolution algorithms
different theories and representations, but the fi-      are fired. Coreferential processed are activated at
nal aim of the workshop was reaching a consen-           the semantic level: they include a centering algo-
sus on what constituted a reasonably complete se-        rithm for topic instantiation and memorization that
mantic representation. Semantics in our case not         we do using a three-place stack containing a Main
only refers to predicate-argument structure, nega-       Topic, a Secondary Topic and a Potential Topic.
tion scope, quantified structures, anaphora resolu-      Main Topics are chosen as best candidates for free
tion and other similar items. It is referred essen-      pronominals - as long as morphological features
tially to a propositional level analysis, which is the   are matching. In order to become a Main Topic,
basis for discourse structure and discourse seman-       a Potential Topic must be reiterated. Discourse
tics contained in discourse relations. It also paves     Level computation is done at propositional level
the way for a deep sentiment or affective analy-         by building a vector of features associated to the
sis of every utterance, which alone can take into        main verb of each clause. They include informa-
account the various contributions that may come          tion about tense, aspect, negation, adverbial mod-
from syntactic structures like NPs and APs, where        ifiers, modality. These features are then filtered
affectively marked words may be contained. Their         through a set of rules which have the task to clas-
contribution needs to be computed in a strictly          sify a proposition as either objective/subjective,
compositional manner with respect to the meaning         factual/nonfactual, foreground/background. In ad-
associated to the main verb, where negation may          dition, every lexical predicate is evaluated with re-
be lexically expressed or simply lexically incorpo-      spect to a class of discourse relations. Eventually,
rated in the verb meaning itself. The system does        discourse structure is built, according to criteria of
low level analyses before semantic modules are           clause dependency where a clause can be classi-
activated, that is tokenization, sentence splitting,     fied either as coordinate or subordinate. Factuality
multiword creation from a large lexical database.        is used to set apart opinions from facts and sub-
Then chunking and syntactic constituency parsing         jectivity is also used to contribute positively to the
which is done using a rule-based recursive tran-         choice of expressing ironic content.
sition network: the parser works in a cascaded
recursive way to include higher syntactic struc-
tures up to sentence and complex sentence level.
These structures are then passed to the first se-        3   The Architecture of SPARSAR:
mantic mapping algorithm that looks for subcate-             Phonetics and Poetic Devices
gorization frames in the lexica made available for
English, including VerbNet, FrameNet, WordNet
and a proprietor lexicon of some 10K entires, with       The second module is a rule-based system
most frequent verbs, adjectives and nouns, con-          that converts graphemes of each poem into
taining also a detailed classification of all gram-      phonetic characters, it divides words into
matical or function words. This mapping is done          stressed/unstressed syllables and computes
following LFG principles (Bresnan, 1982) (Bres-          rhyming schemes at line and stanza level. To this
nan, 2001), where c-structure is mapped onto f-          end it uses grapheme to phoneme translations
structure thus obeying uniqueness, completeness          made available by different sources, amounting to
                                                         some 500K entries, and include CMU dictionary
4,  MRC Psycholinguistic Database 5 , Celex                  type of poetic and rhetoric devices, however it is
Database (H. et al., 1995), plus a proprietor                dependent on language: Italian line verse requires
database made of some 20,000 entries. Out of                 a certain number of beats and metric accents
vocabulary words are computed by means of a                  which are different from the ones contained in an
prosodic parser implemented in a previous project            English iambic pentameter. Rules implemented
(Bacalu and Delmonte, 1999) containing a big                 can demote or promote word-stress on a certain
pronunciation dictionary which covers 170,000                syllable depending on selected language, line-
entries approximately. Besides the need to cover             level syllable length and contextual information.
the majority of grapheme to phoneme conversions              This includes knowledge about a word being part
by the use of appropriate dictionaries, remaining            of a dependency structure either as dependent or
problems to be solved are related to ambiguous               as head.
homographs like “import” (verb) and “import”
(noun) and are treated on the basis of their lexical         4   The Experiment for the Automatic
category derived from previous tagging. Eventu-                  Annotation of the Sonnets using
ally there is always a certain number of Out Of                  SPARSAR
Vocabulary (OOV) words. The simplest case is
constituted by differences in spelling determined            The experiment we devised was organized as fol-
by British vs. American pronunciation. This                  lows: we downloaded SPARSAR from its dedi-
is taken care of by a dictionary of graphemic                cated website https://sparsar.wordpress.com/. At
correspondances. However, whenever the word is               first, following (Tsur, 1992), pag.15 and (Fonagy,
not found the system proceeds by morphological               1971), and on the basis of the complete Phono-
decomposition, splitting at first the word from              logical description of each word in the poem (see
its prefix and if that still does not work, its              (Delmonte, 2016)), the system creates a relation
derivational suffix. As a last resource, an ortho-           between sound and mood or attitude by means of
graphically based version of the same dictionary             the module for sentiment analysis. In particular, it
is used to try and match the longest possible                collapses together unvoiced, obstruent consonants
string in coincidence with current OOVW. Then                with high and back vowels to represent hatred
the remaining portion of word is dealt with by               and struggle, mystic obscurity, sad and aggressive
guessing its morphological nature, and if that fails         mood; the opposite is represented by voiced, sono-
a grapheme-to-phoneme parser is used. Some                   rants and continuants consonants associated to low
words thus reconstructed are wayfarer, gangrened,            and front vowels. These oppositions are then ap-
krog, copperplate, splendor, filmy, seraphic,                plied to the one created by polarity values, nega-
unstarred.                                                   tive vs. positive. We use these quantities to check
Other words we had to reconstruct are: shrive,               an existing correlation, by using ratios. Basic re-
slipstream, fossicking, unplotted, corpuscle,                lations are reported already in (Delmonte, 2016),
thither, wraiths, etc. In some cases, the problem            where however mood of each sonnet was man-
that made the system fail was the presence of a              ually computed. We report here relations inter-
syllable which was not available in VESD, our                vening between the output of the system, compar-
database of syllable durations. This problem has             ing ratios derived from sound relations with those
been coped with partly by manually inserting the             from polarity. As said above, polarity values are
missing syllable and by computing its duration               computed according to a lexicalized approach to
from the component phonemes; but also from the               sentiment analysis which takes into account also
closest similar syllable available in the database.          negation at propositional level (see (Taboada et al.,
We only had to add 12 new syllables for a set                2011) A ratio lower than 1 indicates a majority of
of approximately 1000 poems that the system                  Negative items, higher than 1 a majority of Posi-
computed. The system has no limitation on                    tive items. The same would apply to the remaining
                                                             ratios. We compute the mean value for the three
   4
     It         is        available        online       at   indices – Contrasting Vowels, Contrasting Conso-
              nants, Contrasting Voicing to indicate a generic
   5
     Previously, data for POS were merged in from            sound related mood, Positive when the mean is
a different dictionary (MRC Psycholinguistic Database,
, which   higher than 1 and negative when it is lower. We
uses British English pronunciation)                          then compare Results for polarity from sentiment
analysis with those obtained from sound evalua-           this purpose, we proceeded by extracting manu-
tions. We mark sonnets with a clash between the           ally those failed - we list them in the Appendix -
two parameters with 1 and with 0 whenever they            that the system found without (sufficient) contrast,
converge to the same value. From a perusal of             contrary to the decision of the critics. 6
the results, a total of 79 sonnets over 98 have a            After a careful perusal of the couplet of each
clash, amounting to a remarkably high percentage          such sonnet we came up with a double list. The
of 80%. However when we check the system out-             result is that for 26 sonnets the couplet is a clear
put with the critics’ choice we come up with a dif-       indicator of the subversion of mood, which may
ferent picture: only 77 of all sonnets match with         go from negative to positive, if the rest of the
critics opinion, i.e. exactly 50%. This is the list       sonnet was mostly negative; or from positive to
of those 77 sonnets that have been found to match         negative in the opposite case. As said above, the
between the critics’ list and the list of the sonnets     trigger for the reverted mood was to be found
recognized by the system as having some kind of           in the presence of a discourse marker at the
contrast:                                                 beginning of the first (sometimes the second) line
   1 2 4 5 6 10 12 14 17 18 19 20 21 27 30 32 33          of the couplet. Appropriate discourse markers
34 35 37 41 42 47 48 50 56 57 61 65 67 68 69 71           for mood reversal are adversatives, like "but",
72 74 75 77 78 79 81 82 84 87 92 95 97 98 101             but also concessives, like "yet" and resultatives
102 104 106 108 109 111 113 114 115 116 123               like "so, then". This only applies to 13 of the
125 126 127 129 134 136 137 139 142 144 145               sonnets, the remaining couplets are characterized
146 149 151 152 153 154                                   by presence of negation and negative items (while
                                                          the rest of the poem has a majority of positive
4.1   Extracting Couplets from Logical                    items). This rule was added to the system which
      Structure                                           raised accuracy on all sonnets to 66.88%. Here
                                                          below the list of 26 reclassified sonnets:
Considering the low accuracy reached with the
purely quantitative approach, we decided to look            3, 7, 8, 9, 13, 22, 40, 43, 49, 53, 58, 59, 60, 70,
into the semantic output of the system. We                73, 80, 120, 130, 131, 132, 133, 138, 140, 141,
deemed that one of the possible reasons for the           148, 150
relatively low accuracy of the system could be the
use of quantities to generate abstract evaluations:          The remaining sonnets require the system
in other words, it is not always the case that a con-     to look at the previous and last stanza where
trast is to be found by counting number of nega-          again an appropriate discourse marker - or a
tive vs. positive items present in the sonnet. As         negation plus negative items - must be present to
to semantic representation created by SPARSAR,            introduce the reversal of mood. However, this
we are here referring to the logical structure of         additional modification of the system was not
the Elizabethan sonnet where the argumentation is         fully successful and was abandoned. The list of
developed into three sections and the conclusion          these 19 sonnets is this:
usually comes in the final couplet. This conclu-
sion may revert the contents of the logical order           15, 16, 25, 26, 29, 31, 36, 55, 62, 85, 86, 88, 89,
as defined by the premises. The poet may defer            91, 93, 94, 121, 124, 143
the conclusion in the couplet to complete the logi-
cal argumentation by adding some further motiva-          5     Conclusion
tion. But in some cases the couplet is used to pro-
                                                          In this paper we have presented work carried out to
voke surprise in the reader/hearer, accompanied by
                                                          annotate and experiment with the theme of irony in
laughter or by indignation whenever sarcasm is in-
                                                          Shakespeare’s sonnets. The gold standard for the
tended. So eventually the opposition may only be
                                                          experiment has been created by collecting com-
present in the final two lines, and be hinted at by
                                                          ments produced by literary critics on the presence
presence of discourse markers like “Yet”, “But”.
                                                          of some kind of thematic, semantic and syntactic
In that case, it will not be sufficient for the sys-
                                                              6
tem to ascertain the required quantity for a con-               What we found is a list of 45 sonnets: 3, 7, 8, 9, 13, 15,
                                                          16, 22, 25, 26, 29, 31, 36, 40, 43, 49, 53, 55, 58, 59, 60, 62,
trast, unless some specific rule is inserted that trig-   70, 73, 80, 85, 86, 88, 89, 91, 93, 94, 120, 121, 124, 130, 131,
gers such unexpected, unpredictable ending. To            132, 133, 138, 140, 141, 143, 148, 150
opposition in the sonnets as to produce some sort         Ivan Fonagy. 1971. The functions of vocal style. In
or irony. We have used the system available on the           Seymour Chatman, editor, Literary Style: A Sympo-
                                                             sium, pages 159–174. Oxford UP, London.
web, SPARSAR, to produce an automatic evalu-
ation based on two parameters, phonetic features          Northrop Frye. 1957. Anatomy of Criticism: Four Es-
collapsed according to the theory that treats certain       says. Princeton University Press.
sounds to induce a negative rather than a positive        Baayen R. H., R. Piepenbrock, and L. Gulikers. 1995.
mood. The second parameter is polarity, derived             The CELEX Lexical Database (CD-ROM). Linguis-
from the output of the module for sentiment anal-           tic Data Consortium.
ysis available in the system. From a comparison           Justine Kao and Dan Jurafsky. 2012. A computational
between the critics’ choices and the system’s the            analysis of style, affect, and imagery in contempo-
result was at first rather disappointing, it stopped         rary poetry. In Proceedings of NAACL Workshop on
                                                             Computational Linguistics for Literature, pages 8–
at 50% of all sonnets. We then produced a new and            17, Stroudsburg, PA, USA. ACL.
much richer experiment by considering the logi-
cal structure of the sonnet and the content of the        S.-M. Kim and E. Hovy. 2004. Determining the senti-
                                                             ment of opinions. In Proceedings of the 20th inter-
couplet by means of sentiment analysis, discourse            national conference on computational linguistics -
markers and discourse relations. This allowed us             COLING, pages 1367–1373, Stroudsburg, PA, USA.
to reach a final accuracy of 68.88%.                         ACL.
                                                          Maria Antonietta Marelli. 2015. William Shakespeare,
                                                           I Sonetti – con testo a fronte. Garzanti, Milano.
References
                                                          Giorgio Melchiori. 1971.      Shakespeare’s Sonnets.
Salvatore Attardo. 1994. Linguistic Theories of Hu-         Adriatica Editrice, Bari.
  mor. Mouton de Gruyter, Berlin New York.
                                                          Antonio Reyes and Paolo Rosso. 2013. On the dif-
Salvatore Attardo. 2000. Irony as relevant inappropri-      ficulty of automatically detecting irony: beyond a
  ateness. Journal of Pragmatics, 84(32).                   simple case of negation. Knowledge and Informa-
                                                            tion Systems, 40:595–614.
Ciprian Bacalu and Rodolfo Delmonte.            1999.
  Prosodic modeling for syllable structures from the      Antonio Reyes, Paolo Rosso, and Davide Buscaldi.
  vesd - venice english syllable database. In Aspetti       2012. From humor recognition to irony detection:
  Computazionale in Fonetica, Linguistica e Didattica       The figurative language of social media. Data &
  delle Lingue: Modelli e Algoritmi - Atti 9 Convegno       Knowledge Engineering.
  GFS-AIA, pages 147–160, Venezia. GFS-AIA.
                                                          Michael Schoenfeldt. 2010. Cambridge introduc-
Joan Bresnan, editor. 1982. The Mental Represen-            tion to Shakespeare’s poetry. Cambridge University
  tation of Grammatical Relations. The MIT Press,           Press, Cambridge.
  Cambridge MA.
                                                          Alessandro Serpieri. 2002. Polifonia Shakespeariana.
Joan Bresnan. 2001. Lexical-Functional Syntax.              Bulzoni, Roma.
  Blackwell Publishing, Oxford.
                                                          M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and
Dario Calimani. 2009. William Shakespeare, I sonetti        M. Stede. 2011. Lexicon-based methods for
  della menzogna. Carrocci, Roma.                           sentiment analysis.  Computational Linguistics,
                                                            37(2):267–307.
P. Carvalho, L. Sarmento, M. Silva, and E. de Oliveira.
   2009. Clues for detecting irony in user-generated      Reuven Tsur. 1992. What Makes Sound Patterns Ex-
   contents: oh...!! it’s so easy;-). In Proceeding         pressive: The Poetic Mode of Speech-Perception.
   of the 1st international CIKM workshop on Topic-         Duke UP, Durham N.C.
   sentiment analysis for mass opinion, pages 53–56.
                                                          Cynthia Van Hee, Els Lefever, and Véronique Hoste.
   ACM.
                                                            2018. Semeval-2018 task 3: Irony detection in en-
Rodolfo Delmonte, Sara Tonelli, Marco Aldo Piccol-          glish tweets.). In Proceedings of The 12th Interna-
  ino Boniforti, Antonella Bristot, and Emanuele Pi-        tional Workshop on Semantic Evaluation, pages 39–
  anta. 2005. Venses – a linguistically-based system        50. ACL.
  for semantic evaluation. In Machine Learning Chal-      David K. Weiser.     1983.    Shakespearean irony:
  lenges, pages 344–371, Berlin. Springer.                  The ’sonnets’.     Neuphilologische Mitteilungen,
Rodolfo Delmonte. 2016. Exploring shakespeare’s             84(4):456–469.
  sonnets with sparsar. volume 4, pages 61–95.            David K. Weiser. 1987. Mind in Character – Shake-
                                                            speare’s Speaker in the Sonnets. The University of
R.L. Eagle. 1916. New light on the enigmas of Shake-        Missouri Press.
  speare’s Sonnets. John Long Limited, London.
   APPENDIX                                              Sonnet 130 And yet, by heaven, I think my love
List of couplets and quatrains from sonnets which     as rare As any she belied with false compare.
contain a discourse marker for reverted logical          Sonnet 131 In nothing art thou black save in
structure                                             thy deeds, And thence this slander, as I think, pro-
                                                      ceeds.
A   Section 1: Couplets Reverting the                    Sonnet 132 Then will I swear beauty herself is
    Logical Sequence                                  black, And all they foul that thy complexion lack.
                                                         Sonnet 133 And yet thou wilt, for I being pent
Sonnet 3 But if thou live remembered not to be,       in thee, Perforce am thine, and all that is in me.
Die single and thine image dies with thee.               Sonnet 138 Therefore I lie with her, and she
   Sonnet 7 So thou, thyself out-going in thy noon,   with me, And in our faults by lies we flattered be.
Unlooked on diest unless thou get a son.                 Sonnet 140 That I may not be so, nor thou be-
   Sonnet 8 Whose speechless song, being many,        lied, Bear thine eyes straight, though thy proud
seeming one, Sings this to thee: “Thou single wilt    heart go wide.
prove none.”                                             Sonnet 141 Only my plague thus far I count my
   Sonnet 9 No love toward others in that bosom       gain, That she that makes me sin awards me pain.
sits That on himself such murd’rous shame com-           Sonnet 148 O cunning love! With tears thou
mits.                                                 keep’st me blind, Lest eyes well seeing thy foul
   Sonnet 22 Presume not on thy heart when mine       faults should find.
is slain; Thou gav’st me thine not to give back          Sonnet 150 If thy unworthiness raised love in
again.                                                me, More worthy I to be beloved of thee.
   Sonnet 40 Lascivious grace, in whom all ill
well shows, Kill me with spites; yet we must not      B   Section 2: Couplet + (Part of) Previous
be foes.                                                  Stanza
   Sonnet 43 All days are nights to see till I see
                                                      Sonnet 15 Then the conceit of this inconstant
thee, And nights bright days when dreams do show
                                                      stay Sets you, most rich in youth, before my
thee me.
                                                      sight, Where wasteful time debateth with decay,
   Sonnet 49 To leave poor me, thou hast the
                                                      To change your day of youth to sullied night; And
strength of laws, Since why to love I can allege
                                                      all in war with time for love of you, As he takes
no cause.
                                                      from you, I engraft you new.
   Sonnet 53 In all external grace you have some
                                                         Sonnet 16 So should the lines of life that life re-
part, But you like none, none you, for constant
                                                      pair Which this time’s pencil or my pupil pen Nei-
heart.
                                                      ther in inward worth nor outward fair Can make
   Sonnet 58 I am to wait, though waiting so be       you live yourself in eyes of men. To give away
hell, Not blame your pleasure, be it ill or well.     yourself keeps yourself still, And you must live,
   Sonnet 59 O sure I am the wits of former days      drawn by your own sweet skill.
To subjects worse have giv’n admiring praise.            Sonnet 25 The painful warrior famousèd for
   Sonnet 60 And yet to times in hope my verse        worth, After a thousand victories once foiled, Is
shall stand, Praising thy worth, despite his cruel    from the book of honor razèd quite, And all the
hand.                                                 rest forgot for which he toiled. Then happy I that
   Sonnet 70 If some suspéct of ill masked not        love and am belovèd Where I may not remove nor
thy show, Then thou alone kingdoms of hearts          be removèd.
shouldst owe.                                            Sonnet 26 But that I hope some good conceit
   Sonnet 73 This thou perceiv’st, which makes        of thine In thy soul’s thought, all naked, will be-
thy love more strong, To love that well which thou    stow it. Till whatsoever star that guides my mov-
must leave ere long.                                  ing Points on me graciously with fair aspéct And
   Sonnet 80 Then, if he thrive and I be cast away,   puts apparel on my tattered loving, To show me
The worst was this: my love was my decay.             worthy of thy sweet respect. Then may I dare to
   Sonnet 120 But that your trespass now becomes      boast how I do love thee; Till then, not show my
a fee; Mine ransoms yours, and yours must ransom      head where thou mayst prove me.
me.                                                      Sonnet 29 Yet in these thoughts myself almost
despising, Haply I think on thee, and then my          of all men’s pride I boast; Wretched in this alone,
state, Like to the lark at break of day arising From   that thou mayst take All this away, and me most
sullen earth, sings hymns at heaven’s gate. For thy    wretched make.
sweet love remembered such wealth brings That             Sonnet 93 But heav’n in thy creation did de-
then I scorn to change my state with kings.            cree That in thy face sweet love should ever dwell;
   Sonnet 31 But things removed that hidden in         Whate’er thy thoughts or thy heart’s workings be,
thee lie. Thou art the grave where buried love doth    Thy looks should nothing thence but sweetness
live, Hung with the trophies of my lovers gone,        tell. How like Eve’s apple doth thy beauty grow, If
Who all their parts of me to thee did give; That       thy sweet virtue answer not thy show.
due of many now is thine alone. Their images I            Sonnet 94 But if that flow’r with base infection
loved I view in thee, And thou, all they, hast all     meet, The basest weed outbraves his dignity. For
the all of me.                                         sweetest things turn sourest by their deeds; Lilies
   Sonnet 36 I may not evermore acknowledge            that fester smell far worse than weeds.
thee, Lest my bewailèd guilt should do thee               Sonnet 121 Which in their wills count bad what
shame; Nor thou with public kindness honor me,         I think good? No, I am that I am, and they that
Unless thou take that honor from thy name. But         level At my abuses reckon up their own; I may
do not so; I love thee in such sort, As, thou being    be straight, though they themselves be bevel. By
mine, mine is thy good report.                         their rank thoughts my deeds must not be shown,
   Sonnet 55 Even in the eyes of all posterity That    Unless this general evil they maintain: All men are
wear this world out to the ending doom. So till the    bad, and in their badness reign.
judgment that yourself arise, You live in this, and       Sonnet 124 That it nor grows with heat nor
dwell in lovers’ eyes.                                 drowns with showers. To this I witness call the
   Sonnet 62 But when my glass shows me myself         fools of time, Which die for goodness, who have
indeed, Beated and chopped with tanned antiquity,      lived for crime.
Mine own self-love quite contrary I read; Self so         Sonnet 143 So run’st thou after that which flies
self-loving were iniquity. ’Tis thee, myself, that     from thee, Whilst I, thy babe, chase thee afar be-
for myself I praise, Painting my age with beauty       hind. But if thou catch thy hope, turn back to me,
of thy days.                                           And play the mother’s part, kiss me, be kind. So
                                                       will I pray that thou mayst have thy Will, If thou
   Sonnet 85 But that is in my thought, whose love
                                                       turn back and my loud crying still.
to you, Though words come hindmost, holds his
rank before. Then others for the breath of words
respect, Me for my dumb thoughts, speaking in
effect.
   Sonnet 86 As victors of my silence cannot
boast. I was not sick of any fear from thence;
But when your countenance filled up his line, Then
lacked I matter, that enfeebled mine.
   Sonnet 88 The injuries that to myself I do, Do-
ing thee vantage, double vantage me. Such is my
love, to thee I so belong, That for thy right myself
will bear all wrong.
   Sonnet 89 Thy sweet belovèd name no more
shall dwell, Lest I, too much profane, should do
it wrong And haply of our old acquaintance tell.
For thee against myself I’ll vow debate, For I must
ne’er love him whom thou dost hate.
   Sonnet 91 But these particulars are not my mea-
sure; All these I better in one general best. Thy
love is better than high birth to me, Richer than
wealth, prouder than garments’ cost, Of more de-
light than hawks or horses be; And having thee,