Annotation and Detection of Emotion Polarity in I Promessi Sposi: Dataset and Experiments

Abstract

Emotions play a crucial role in literature and are studied by various disciplines, e.g. literary criticism, psychology, anthropology and, more recently, also with computational methods in NLP. However, studies in the Italian context are still limited. This work therefore aims to advance the state of the art in the field of emotion analysis applied to historical texts by proposing a new dataset and describing the results of a set of emotion polarity detection experiments. The text analyzed is “I Promessi Sposi” in its final edition (published in 1840), one of the most important novels in the Italian literary and linguistic canon.

Keywords

• eol>emotion analysis
• annotation
• fine-tuning
• Italian
• literary texts

1. Introduction

author’s religious spirit and social and political polemic, and because it quickly became a model of the Italian Emotions play a key role in literature, representing a language, stably included in school curricula as mandabridge between the author’s purposes, the text, and the tory study material. This has led to a certain degree of reader’s personal background: literature collects experi- reluctance and lack of enthusiasm among the readers. ences and contains the emotions that accompany them, As a consequence, a study of emotions in “I Promessi in turn generating new experiences and new emotions. Sposi” can be beneficial from both an academic and eduTherefore, studying emotions in literary texts implies cational standpoint. Academically, it can provide new inthe possibility of providing valuable insights into the sights into a classic text, encouraging new interpretations deeper meanings and intentions behind a work, the form and scholarly discussions. For didactic purposes, analyzit may take, and the readers’ engagement with it. This ing the emotions in “I Promessi Sposi” can make the novel ifeld of study has recently experienced a flourishing na- more relatable and appealing for students, revealing the tional and international development involving diferent depth and complexity of the characters’ experiences in disciplines, from literary criticism to philosophy, from the context in which they live, and encouraging a closer anthropology to psychology. For example, in the Italian connection with them and with Manzoni’s social issues. context, Ginzburg et al. [ 1 ] analyzed how Matte Blanco’s Given this context, computational methods, already psychoanalytic theories on emotions are applied to lit- widely applied especially on user-generated contents erary criticism, taking into account authors like Tozzi, (such as reviews and social media posts), can be profPirandello, and Svevo, while Guaragnella [ 2 ] explored itably tested on the fictional text after developing specific the complex interaction between humor and sadness in datasets for training and evaluating new models. The 20th-century Italian literature from a both philosophical present work takes as a basis a preliminary annotation and literary point of view. of the Manzoni’s novel, expanding the number of manu

However, some literary works remained under- ally labeled sentences and proposing the development of explored. One such work is Alessandro Manzoni’s “I some models of varying complexity. Promessi Sposi”. Despite its emotional richness, the novel More specifically, two are the main contributions of has often been regarded as monolithic and static, both be- our work: i) we release1 a new dataset made of more cause of the narrated events, strongly influenced by the than 3.000 sentences taken from “I Promessi Sposi” manCLiC-it 2024: Tenth Italian Conference on Computational Linguistics, ually annotated with four emotion polarity classes (i.e. Dec 04 — 06, 2024, Pisa, Italy POSITIVE, NEGATIVE, NEUTRAL, MIXED); ii) we test var* Corresponding author. ious approaches for emotion polarity detection using the † This paper is the result of the collaboration between the two au- new dataset as-is but also augmenting it with other anthors. For the specific concerns of the Italian academic attribution notated Italian resources. system: Rachele Sprugnoli is responsible for Sections 2, 3.2, 3.3, 4; Arianna Redaelli is responsible for Sections 1, 3, 3.1, 5. $ [e-mail] (R. Sprugnoli); [e-mail] (A. Redaelli)

0000-0001-6861-5595 (R. Sprugnoli); 0000-0001-6374-9033 (A. Redaelli) 1https://github.com/RacheleSprugnoli/Emotion_Analysis_ ©At2tr0i2b4utCioonpy4r.0igIhnttefornratthioisnpaalp(CerCbByYit4s.0a)u.thors. Use permitted under Creative Commons License Manzoni

2. Related Work

of the novel) chosen to cover various phases of the plot, diferent characters and types of content. Specifically, we Emotion analysis, that is the automatic recognition of used Chapter III, in which Renzo (one of the protagonists) emotions conveyed in a text, is a Natural Language Pro- goes to the lawyer Azzeccagarbugli in an attempt to recessing (NLP) task applied to various types of texts. In solve the legal obstacle preventing him from marrying his fact, although most datasets and systems are developed beloved Lucia. However, this results in a misunderstandto process social media posts and reviews, there are also ing and the ultimate failure of his endeavor. Chapters applications on news [ 3 ], songs [ 4 ] and personal narra- IV and V describe the conversion of Fra Cristoforo, a tives [ 5 ]. After the so-called afective-turn in literary religious figure and friend of the betrothed couple, and studies [ 6 ], the attention towards this task has signifi- his heated discussion with Don Rodrigo, the lord who is cantly increased also in the humanities with studies on preventing Renzo and Lucia’s marriage, which also ends both historical and ancient languages and on various in failure. Chapters IX and X introduce the ambivalent textual genres.2 Among these we mention, as examples, story of the Nun of Monza, chosen as the protector of drama plays [ 9, 10 ], fairy tales [11], poems [12] and chil- Lucia who is fleeing from Don Rodrigo. Chapters XIV dren’s literature [13]. As for novels, Mohammad [14] and XV depict Renzo’s involvement in the bread riot in compared fairy tales and novels from the point of view of Milan, after which he gets drunk at the Full Moon Tavern, emotions identified with the NRC Emotion Lexicon [ 15]; is arrested, and eventually manages to escape. Chapters Zehe et al. [16] used emotion analysis for discriminating XX and XXI describe Lucia’s arrival at the house of the between German novels with and without happy end- Unnamed, the worst baron of that time, who, at Don ings; Stankovic et al. [17] presented various experiments Rodrigo’s request, kidnaps her – only to later repent on Serbian novels; Kim [18] tested the dictionary-based in a tormenting process of conversion to the Christian tool Syuzhet3 on a set of 19th-century British novels. faith. Chapter XXVIII contain an historical digression As regards the literary domain, however, the works on on Milan, devastated by famine, the invasion of the LanItalian are few: for example, Rebora [19] analyzed the squenets, and the threat of the plague. Chapter XXXIII annotation of a short story by Pirandello as performed by portrays Don Rodrigo on his deathbed, sufering from a group of students; Pavan [20] applied a lexicon-based the plague, and a flashback to Renzo, who, having resoftware to 16 novels and poems written in the twenti- covered from the disease, sets out to find Lucia. Finally, eth century; and Zhang et al. [21] released a dataset of the last chapter, Chapter XXXVIII, depicts the concluopera verses with which they performed various emotion sion of the story, with the serene reunion of the couple, recognition experiments. now ready to embark on their married life. As can be

The present work wants to advance the state of the seen, our choice provided very lively parts, others more art in the field of emotion analysis applied to historical introspective, and others that contain descriptions and novels; specifically, a previous preliminary annotation of historical digressions. “I Promessi Sposi” is taken up [22], expanding the number The annotation was carried out by the two authors of manually labelled sentences (from 338 to 3,095) and of this paper independently, following the guidelines reproposing new experiments for the automatic identifica- ported below and using a spreadsheet having a sentence tion of emotion polarity. Although the novel in question per row. While annotating, the annotators did not have is considered one of the most important in the history of access to each other annotator’s score. Each chapter conItalian literature and language, as far as we know, this sists of between approximately 170 and 330 sentences study is the first to address the topic of emotions in Man- and the average annotation time per chapter was about zoni’s work through computational methods, developing 1.5 hours (18 hours in total). Subsequently, the results of specific resources and models. the independently conducted annotations were placed in parallel columns to allow each annotator to revise any 3. Dataset Creation obvious errors or oversights. This preliminary phase was followed by a direct discussion between the two annotaThe dataset is composed of 3,095 manually split4 sen- tors to address the most problematic cases and achieve tences from 12 chapters (about 30% of the total chapters the gold annotation (see Sections 3.2 and 3.3). 2For a complete overview of sentiment and emotion analysis in the 3.1. Guidelines ifeld of literary studies please refer to the survey papers by Kim and Klinger [ 7 ] and Rebora [ 8 ]. The annotation was carried out at sentence level and was 3https://github.com/mjockers/syuzhet based on both the lexicon used and the images evoked by 4Sentence splitting was done manually because automatic segmen- the author, for example through the use of rhetorical figtaavtaiiolnabplerefosernIttaeldiansi,glnarificgaenlyt dcuhealtloenthgeesnofoverl’tshientmriocdateelspucunrcrteunattiloyn. ures. The annotation followed the flow of the text, so the Details are described in [23]. annotator can take into account the previous sentences but not the following ones. The polarity to be annotated marked: 0.63 for NEUTRAL, 0.65 for POSITIVE and 0.70 was the one expressed by the author, either through the for NEGATIVE. narrator or through the characters who take part in the From the analysis of the disagreements, it emerged events told, and not the one felt by the annotator while that some uncertainties were related to the presence of reading the sentence. The polarity could also concern irony. It was therefore decided to annotate these cases emotions related to a diferent time from that of the main as MIXED, since in such sentences two polarities coexist, story. To assign the correct label, the annotator had to i.e. the one expressed by the literal meaning and the one answer the question how are the emotions evoked by the due to the presence of irony. It is important to note that, author in the sentence being analyzed? with one of the in our annotation, irony was considered as a sentiment following options: shifter that changes the polarity of the literal meaning of a sentence. This interpretation of irony is much more • predominantly or solely positive (label narrowed compared to that of Manzoni’s literary critiPOSITIVE), such as caring, joy, relief, amuse- cism. In fact, in “I Promessi Sposi” irony is a complex ment; rhetorical device that can subtly influence the reader’s • predominantly or solely negative (label perception and understanding on multiple levels of the NEGATIVE), such as confusion, nervousness, novel [26]. Consequently, the term irony refers not only annoyance, resignation, disapproval, fear, to irony in its strict sense but also to humor, sarcasm, disappointment, embarrassment, sadness, pain, innuendo, and other related concepts, which the author anger and remorse; uses to suggest more in-depth information into charac• of the opposite type, thus it is not possible to find ters, situations, linguistic uses, and social problems [27].

a clearly prevalent emotion (label MIXED); However, for our purposes, it was not practical to apply • absent (label NEUTRAL). this broader concept of irony because it often requires a deep understanding of the author’s intentions that goes This distinction was inspired by previous annotation ef- far beyond the sequential interpretation of individual senforts, such as the one underlying the SENTIPOLC shared tences. Another aspect revised and better detailed in the task in which the four labels were applied to tweets guidelines was the annotation of approval expressions [24, 25]. The guidelines have been revised and enriched (such as “Sì, signore.”, EN: Yes, sir), that it was decided after the analysis of the disagreements, as will be de- to annotate as NEUTRAL and not as POSITIVE, unless scribed in the next subsection. they were accompanied by other elements expressing positive emotions. Descriptive sentences also had to be 3.2. Agreement annotated as NEUTRAL if they did not contain words that evoked specific emotions. For example, “Era un guazzThe Cohen’s kappa calculated for each chapter recorded abuglio di steli, che facevano a soverchiarsi l’uno con a minimum value of 0.51 (on Chapter III) and a maxi- l’altro nell’aria” (EN: It was a jumble of stems, which tried mum value of 0.71 (on Chapter XXIII). On average, there- to overwhelm each other in the air.) should have been fore, a moderate agreement of 0.62 was obtained. Specifi- annotated as NEGATIVE for the presence of words that cally, the most dificult class to annotate was MIXED (k = evoke confusion and oppression; on the contrary, “Per un 0.50), while for the other labels the diferences were less buon pezzo, la costa sale con un pendìo lento e continuo” (EN: For a good while, the coast rises with a slow and continuous slope) should have been annotated as NEUTRAL.

Lastly, courtesy titles (such as “reverendissimo”, EN: most reverend) also had to be assigned the NEUTRAL label because they represent a formal requirement and not a true positive emotional involvement. Annotating dialogue turns proved to be particularly dificult, especially when dealing with very short sentences, composed of 1 to 3 words. In these cases, the preceding context but also the presence of punctuation and interjections were essential for assigning the polarity label.

3.3. Final Dataset

The dataset resulting from the consolidation of dis

agreements is made up of 1,413 sentences annotated as NEGATIVE (corresponding to 46% of the total sentences), 692 NEUTRAL sentences (22%), 598 POSITIVE sentences (19%) and 392 MIXED ones (13%). The distribution of the four classes in the various chapters is shown by the bar graphs in Figure 1. The fact that most of the sentences The experiments were performed using the dataset conhave a negative polarity is in line with the topics covered sisting only of the novel’s chapters (divided into training, in the novel: kidnappings, misunderstandings, plague. development and test sets according to the proportions The only chapter in which the POSITIVE label prevails 80/10/10) but also adding data from other Italian linguisis the last one (XXXVIII) which tells the happy ending tic resources annotated with emotions in order to have of the novel, that is, the marriage and the new happy more training examples. In particular, the resources used life of the two protagonists. It is interesting to note that to augment the original dataset are the following: compared to the first tests of annotating emotion polarity [22], the NEUTRAL class is no longer the most frequent in the data. Since then, the guidelines had been enriched with details regarding the specific emotions to be considered as positive and negative: this allowed the annotators to be more precise in identifying the prevalent type of emotion even in the case of minimal nuances. the AdamW optimizer (learning rate: 2e-5, epsilon: 1e-8) and 2 epochs6; • a fine-tuned model of multilingual XLM-RoBERTa [28] using an Hugging Face PyTorch implementation7 and the following hyperparameters: 32 for batch size, 2e-5 for learning rate, 6 epochs,

AdamW optimizer; • a lexicon-based script employing both a polarity lexicon created for contemporary Italian (i.e., W-MAL, Weighted-Morphologically-inflected Affective Lexicon) [29] and one derived from 19thcentury Italian narrative texts8. A score is computed for each sentence by summing the polarity values of the tokens. If the score is greater than 0, the label is POSITIVE; if it is less than 0, the label is NEGATIVE; if it is equal to 0 because all tokens have this value or are not present in the lexicon, the label is NEUTRAL; if it is equal to 0 because the sum of tokens with positive and negative polarities is balanced, the label is MIXED. • MultiEmotions-it: a multi-labelled emotion dataset made of comments posted on Facebook and YouTube annotated following Plutchik’s basic emotions (anger, disgust, fear, joy, sadness, surprise, trust, anticipation) and dyads (such as love and disappointment) [ 30 ]; • FEEL-IT: a benchmark corpus of tweets annotated with four emotions, that is fear, joy, sadness, anger [ 31 ]; • EMit: a dataset of multi-labelled tweets annotated with Plutchik’s basic emotions plus love and neutral [ 32 ]; • XED: a multilingual emotion dataset in which the annotation performed on Finnish and English sentences are projected on the corresponding items in 30 languages, including Italian, using parallel corpora [ 33 ]. The eight Plutchik’s basic emotions are adopted for the annotation;

4. Experiments

The annotated dataset described in the previous Section was used to train and evaluate various approaches of diferent complexity, namely: • a Linear Support Vector classifier (SVC) developed using the scikit-learn library with default parameters and to be considered as a baseline; • a ifne-tuned model of bert-base-italian-xxl-cased5 using better than the others, so we will focus on it in the remainder of 5Provided by the MDZ Digital Library team of the Bavarian State the paper.

Library through the Hugging Face framework: https://huggingface. 6We adapted the notebook https://www.kaggle.com/code/ co/dbmdz/bert-base-italian-xxl-cased. We tested all the BERT neerajmohan/fine-tuning-bert-for-text-classification. models available in the MDZ Digital Library repository, that is 7We adapted the following implementation: https://gist.github.com/ bert-base-italian-cased, bert-base-italian-uncased, sayakmisra/b0cd67f406b4e4d5972f339eb20e64a5. bert-base-italian-xxl-cased and 8https://github.com/RacheleSprugnoli/Emotion_Analysis_ bert-base-italian-xxl-cased; the latter performed Manzoni

The lexicon-based approach outperforms the baseline

(i.e., the Support Vector Classifier); the latter does not F1 Lexicon-Based Approach benefit from increasing the size of the training set and

W-MAL XIX cent. performs very poorly in recognizing sentences annotated POSITIVE 0.45 0.44 as MIXED (F1 < 0.1). Using an in-domain lexicon specially NEGATIVE 0.35 0.31 created starting from nineteenth-century texts yields betNEUTRAL 0.15 0.48 ter results with respect to using the W-MAL lexicon. This MIXED 0.00 0.19 improvement is noted both in terms of macro average F1 Macro Avg. 0.24 0.35 (+ 0.11) and in the recognition of NEUTRAL and MIXED instances, +0.33 and +0.19 respectively. The fine-tuned

XLM-RoBERTa model achieves the best F1 both overall • TwIT: a corpus of tweets annotated with six difer- (0.53) and for all classes even if using diferent training ent emotions (i.e., happiness, trust, sadness, sets. Interestingly, in the case of fine-tuned models (both anger, fear and disgust) [ 34 ]; using BERT and XLM-RoBERTa) the All training set, • AriEmozione 2: a dataset of verses of opera although significantly larger than the others, does not arias written in 18th-century Italian annotated provide the greatest benefits. Indeed, the most beneficial with one out of six emotions (i.e., love, joy, training set is Manz-Multi-EMit which combines the admiration, anger, sadness, fear) [21]. most similar datasets from the annotation point of view, as both MultiEmotions-it and EMit contain NEUTRAL and MIXED sentences.

Figure 2 shows the confusion matrix for the best model.

We can notice an over-prediction of the NEGATIVE label even if this is not the most frequent class of the dataset, covering 35.8% of the total (while the POSITIVE class represents 38.1% of the total). Examples of sentences incorrectly classified as NEGATIVE are:

The original emotion labels of the aforementioned re

sources were mapped onto our four classes on the basis of their polarity. Data labelled with ambiguous emotions (such as surprise and anticipation) were left out. Please note that only MultiEmotions-it and EMit contain the class NEUTRAL and that their multi-label structure allowed us to convert the original annotation to the MIXED class when the emotions assigned to the same sentence were of opposite polarity.

Based on the characteristics of the aforementioned datasets, three training sets were prepared: one with only sentences taken from “I Promessi Sposi” (Manzoni, 2,771 instances), one adding MultiEmotions-it and EMit to the sentences taken from Manzoni’s novel (Manz-Multi-EMit, 10,755 instances), and one joining all the available datasets (All, 21,923 instances).

4.1. Results

• "Per i nostri fu una nuova cuccagna." EN: For our people it was a new bonanza. Gold label = POSITIVE • “Già principiava a farsi buio.” EN: It was already starting to get dark. Gold label = NEUTRAL • “Io ho perdonato tutto: non ne parliam più: ma me n’avete fatti dei tiri.” EN: I’ve forgiven everything: we don’t talk about it anymore: but you played tricks on me. Gold label = MIXED

5. Conclusions

This paper presents a new manually annotated dataset and a set of experiments for the automatic detection of emotion polarity. More specifically, the dataset contains

3,095 sentences taken from “I Promessi Sposi” and the experiments cover diferent approaches, namely lexiconbased, SVC and the fine-tuning of an Italian BERT model and of the multilingual XLM-RoBERTa model. The impact of the training set size is also evaluated by increasing the in-domain dataset by combining other annotated Italian resources.

We are aware that for the emotion analysis task, as for all NLP tasks, Large Language Models are now widely used [ 35 ] but these require computational powers currently not available to the authors of the paper. In the future, our work will focus on this aspect in order to be in line with the current state of the art. Another future work will concern the annotation of emotions with more granular labels, extending an activity already started on Chapter VIII only, on which the label scheme proposed for the GoEmotions dataset [ 36 ] was applied [22]. Additionally, we plan to pay greater attention to the annotation of irony, a crucial aspect of the novel. This could be incorporated into the dataset using a binary 0/1 value to indicate its presence or absence, as we have already begun to implement 9. Finally, we would like to explore the applications of our work in the school context. Concerning the study of emotions in Manzoni’s novel, computational methods and tools could provide inputs and data useful for didactic practical activities, such as visual representations of afective scenes, roleplaying exercises, or even crowd-sourced annotation that allows students to express their personal interpretations of the characters’ emotions in diferent chapters and situations. Activities like these can make the whole learning experience more dynamic and captivating, promoting a deeper connection between the students and the novel and, meanwhile, improving their critical thinking and empathy.

9https://github.com/RacheleSprugnoli/Emotion_Analysis_

Manzoni

Acknowledgments

The authors thank Giovanni Moretti for the help given

with the fine-tuning scripts.

Questa pubblicazione è stata realizzata da ricercatrice con contratto di ricerca cofinanziato dall’Unione europea - PON Ricerca e Innovazione 2014-2020 ai sensi dell’art. 24, comma 3, lett. a, della Legge 30 dicembre 2010, n. 240 e s.m.i. e del D.M. 10 agosto 2021 n. 1062. [10] F. Debaene, K. van der Haven, V. Hoste, Early Mod- and Litta, E.(eds), Atti del IX Convegno Annuale ern Dutch comedies and farces in the spotlight: In- AIUCD. La svolta inevitabile: sfide e prospettive’per troducing EmDComF and its emotion framework, l’Informatica Umanistica. Università Cattolica del in: R. Sprugnoli, M. Passarotti (Eds.), Proceedings Sacro Cuore, Milano (2020) (2020) 216–221. of the Third Workshop on Language Technologies [20] L. Pavan, A survey of some Italian literature works for Historical and Ancient Languages (LT4HALA) using sentiment analysis, International Journal of @ LREC-COLING-2024, ELRA and ICCL, Torino, Linguistics, Literature and Translation (2022) 117– Italia, 2024, pp. 144–155. URL: https://aclanthology. 121.

org/2024.lt4hala-1.17. [21] S. Zhang, F. Fernicola, F. Garcea, P. Bonora, [11] E. P. Volkova, B. Mohler, D. Meurers, D. Gerdemann, A. Barrón-Cedeño, AriEmozione 2.0: Identifying H. H. Bülthof, Emotional perception of fairy tales: Emotions in Opera Verses and Arias, IJCoL. Italian achieving agreement in emotion annotation of text, Journal of Computational Linguistics 8 (2022) 7–26. in: Proceedings of the NAACL HLT 2010 Workshop [22] R. Sprugnoli, A. Redaelli, How to Annotate Emoon Computational Approaches to Analysis and Gen- tions in Historical Italian Novels: A Case Study on I eration of Emotion in Text, 2010, pp. 98–106. Promessi Sposi, in: Proceedings of the Third Work[12] R. Sprugnoli, F. Mambrini, M. Passarotti, G. Moretti, shop on Language Technologies for Historical and The Sentiment of Latin Poetry. Annotation and Au- Ancient Languages (LT4HALA)@ LREC-COLINGtomatic Analysis of the Odes of Horace, IJCoL. Ital- 2024, 2024, pp. 105–115. ian Journal of Computational Linguistics 9 (2023) [23] A. Redaelli, R. Sprugnoli, Is Sentence Splitting a 53–71. Solved Task? Experiments to the Intersection Be[13] S. Rebora, M. Lehmann, A. Heumann, W. Ding, tween NLP and Italian Linguistics, in: Proceedings G. Lauer, Comparing ChatGPT to Human Raters of the 10th Italian Conference on Computational and Sentiment Analysis Tools for German Chil- Linguistics (CLiC-it 2024), Pisa, Italy, 2024. dren’s Literature, in: CHR 2023: Computational Hu- [24] V. Basile, A. Bolioli, V. Patti, P. Rosso, M. Nissim, manities Research Conference, Paris, France, 2023, Overview of the evalita 2014 sentiment polarity pp. 333–343. classification task, in: Proceedings of the First Ital[14] S. Mohammad, From once upon a time to happily ian Conference on Computational Linguistics CLiCever after: Tracking emotions in novels and fairy it 2014 & and of the Fourth International Workshop tales, in: K. Zervanou, P. Lendvai (Eds.), Proceed- EVALITA 2014: 9-11 December 2014, Pisa, Pisa Uniings of the 5th ACL-HLT Workshop on Language versity Press, 2014, pp. 50–57.

Technology for Cultural Heritage, Social Sciences, [25] F. Barbieri, V. Basile, D. Croce, M. Nissim, and Humanities, Association for Computational N. Novielli, V. Patti, et al., Overview of the evalita Linguistics, Portland, OR, USA, 2011, pp. 105–114. 2016 sentiment polarity classification task, in: URL: https://aclanthology.org/W11-1514. EVALITA Evaluation of NLP and Speech Tools for [15] S. M. Mohammad, P. D. Turney, Crowdsourcing a Italian. Proceedings of the Final Workshop, AILC, word-emotion association lexicon, Computational 2016, pp. 146–155.

Intelligence 29 (2013) 436–465. [26] E. Raimondi, L’ironia polifonica in manzoni, in: La [16] A. Zehe, M. Becker, L. Hettinger, A. Hotho, I. Reger, dissimulazione romanzesca: antropologia manzoniF. Jannidis, Prediction of happy endings in German ana, Il mulino, Bologna, 1990, pp. 45–80. novels based on sentiment information, in: 3rd [27] L. Manfreda, Figure dell’ironia nei Promessi sposi : Workshop on Interactions between Data Mining il ruolo doppio a rovescio dei personaggi, Metauro, and Natural Language Processing, Riva del Garda, Pesaro, 2006.

Italy, volume 5, 2016. [28] A. Conneau, K. Khandelwal, N. Goyal, V. Chaud[17] R. Stankovic, M. Kosprdic, M. Ikonic-Nesic, hary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, T. Radovic, Sentiment Analysis of Sentences from L. Zettlemoyer, V. Stoyanov, Unsupervised crossSerbian ELTeC corpus, in: Proceedings of the lingual representation learning at scale, in: ProSALLD-2 Workshop at Language Resources and ceedings of the 58th Annual Meeting of the AssociaEvaluation Conference (LREC), Marseille, France, tion for Computational Linguistics, Association for 2022, pp. 31–38. Computational Linguistics, online, 2020, pp. 8440– [18] H. Kim, Sentiment analysis: Limits and progress of 8451. URL: https://aclanthology.org/2020.acl-main. the syuzhet package and its lexicons., DHQ: Digital 747. doi:10.18653/v1/2020.acl-main.747.

Humanities Quarterly 16 (2022). [29] M. Vassallo, G. Gabrieli, V. Basile, C. Bosco, et al., [19] S. Rebora, et al., Shared Emotions in Reading Polarity imbalance in lexicon-based sentiment analPirandello. An Experiment with Sentiment Anal- ysis, in: Proceedings of the Seventh Italian Conysis, Marras, C., Passarotti, M., Franzini, G., ference on Computational Linguistics, CEUR, 2020,

References

[1] A. Ginzburg , R. Luperini , V. Baldi (Eds.), Emozioni e letteratura. La teoria di Matte Blanco e la critica letteraria contemporanea , Fabrizio Serra, Firenze , 2009 .

[2] P. Guaragnella , I volti delle emozioni. Riso, sorriso e malinconia nel Novecento letterario italiano , Società Editrice Fiorentina, Firenze, Italy, 2015 .

[3] A. M. Patronella , Covering Climate Change: A Sentiment Analysis of Major Newspaper Articles from 2010-2020 , Inquiries Journal 13 ( 2021 ).

[4] D. Edmonds , J. Sedoc, Multi-emotion classification for song lyrics , in: Proceedings of the Eleventh Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis , 2021 , pp. 221 - 235 .

[5] A. Tammewar , A. Cervone , E.-M. Messner , G. Riccardi, Annotation of emotion carriers in personal narratives , in: Proceedings of the Twelfth Language Resources and Evaluation Conference , 2020 , pp. 1517 - 1525 .

[6] S. Keen , Introduction: Narrative and the Emotions, Poetics Today 32 ( 2011 ) 1 - 53 . URL: https: //doi.org/10.1215/ 03335372 - 1188176 . doi: 10 .1215/ 03335372 - 1188176 .

[7] E. Kim , R. Klinger , A survey on sentiment and emotion analysis for computational literary studies , Wolfenbüttel , 2019 .

[8] S. Rebora , Sentiment analysis in literary studies. a critical survey , DHQ: Digital Humanities Quarterly 17 ( 2023 ).

[9] T. Schmidt , K. Dennerlein , C. Wolf , Emotion classification in German plays with transformer-based language models pretrained on historical and contemporary language , in: S. Degaetano-Ortlieb , A. Kazantseva , N. Reiter , S. Szpakowicz (Eds.), Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage , Social Sciences, Humanities and Literature , Association for Computational Linguistics, Punta Cana, Dominican Republic (online) , 2021 , pp. 67 - 79 . URL: https://aclanthology.org/ 2021 .latechclfl- 1 .8. doi: 10 . 18653/v1/ 2021 .latechclfl- 1 .8. pp. 1 - 7 .

[30] R. Sprugnoli , MultiEmotions-it: A new dataset for opinion polarity and emotion analysis for Italian , in: Proceedings of the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020 ), Accademia University Press, online, 2020 , pp. 402 - 408 .

[31] F. Bianchi , D. Nozza , D. Hovy , FEEL-IT: emotion and sentiment classification for the Italian language, in: The 16th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics , online, 2021 , pp. 76 - 83 .

[32] O. Araque , S. Frenda , R. Sprugnoli , D. Nozza , V. Patti , et al., EMit at EVALITA 2023 : overview of the categorical emotion detection in Italian social media task, in: EVALITA Proceedings of the Eighth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop , AILC - Associazione Italiana di Linguistica Computazionale , 2023 , pp. 37 - 44 .

[33] E. Öhman , M. Pàmies , K. Kajava , J. Tiedemann, XED: A multilingual dataset for sentiment analysis and emotion detection , in: D. Scott , N. Bel , C. Zong (Eds.), Proceedings of the 28th International Conference on Computational Linguistics , International Committee on Computational Linguistics , Barcelona, Spain (Online) , 2020 , pp. 6542 - 6552 . URL: https: //aclanthology.org/ 2020 .coling-main. 575 . doi: 10 . 18653/v1/ 2020 .coling-main. 575 .

[34] A. Chiorrini , C. Diamantini , A. Mircoli , D. Potena , E. Storti, EmotionAlBERTo: Emotion Recognition of Italian Social Media Texts Through BERT , in: 2022 26th International Conference on Pattern Recognition (ICPR) , IEEE, 2022 , pp. 1706 - 1711 .

[35] C. Diamantini , A. Mircoli , D. Potena , S. Vagnoni , et al., An experimental comparison of large language models for emotion recognition in italian tweets ., in: ITADATA 2023. Italian Conference on Big Data and Data Science 2023 , CEUR Workshop Proceedings, Napoli, Italy, 2023 .

[36] D. Demszky , D. Movshovitz-Attias , J. Ko , A. Cowen , G. Nemade, S. Ravi, GoEmotions: A dataset of ifne-grained emotions, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics , Online, 2020 , pp. 4040 - 4054 . URL: https://aclanthology.org/ 2020 .acl-main. 372 . doi: 10 . 18653/v1/ 2020 .acl-main. 372 .