Emotions and language are strongly associated. In recent years, many resources have been created to investigate this association and automatically detect emotions from texts. Presenting ELIta (Emotion Lexicon for Italian), this study provides a new language resource for the analysis and detection of emotions in Italian texts. It describes the process of lexicon creation, including lexicon selection and annotation methodologies, and compares the collected data with existing resources. By ofering a non-aggregated lexicon, ELIta fills a crucial gap and is applicable to various research and practical applications. Furthermore, the work utilises the lexicon by analysing the relationships between emotions and gender.
CEUR ceur-ws.org
Emotions and language are deeply interrelated humgoarnies that have evolved to serve various adaptive
funccharacteristics. Language serves as a tool to commtuinoin-s through specific neural signatures, facial
exprescate our feelings, while afective studies have shown thsaiotns, cognitive evaluations, and behavioral action
tendencies [24, 25]. These basic emotions typically include
[14, 15] to crowdsourcing16[, 17].
as morphology 3[, 4, 5], phonology 6[, 7], and semantics joy, sadness, disgust, anger, fear, surprise, whereas
[8, 9]. This intricate relationship has recently attraPclutetdchikalso considertsrust andanticipation.
Designificant attention in fields such as computational lsipni-te objections to the basic emotions mo2d7e]l, i[t has
guistics, natural language processing (NLP), and afectiinvespired the creation of resources such as the NRC
Lexicomputing. Research focusing on the identification coofn (EmoLex) [17] (translated into over 100 languages,
emotions from texts has produced various languageitr’es-the most widely used lexicon in emotion detection),
sources, particularly emotion lexicons developed usainndg the datasets Feel2I8t][and Multiemotion I2t9[].
diverse annotation methodologies, ranging from manuaMlore recently, the field of computational linguistics
[
tions 1[8, 19]. According to this perspective, the PAfDor lexicons created from texts in the target language (Pleasure, Arousal, Dominance2)0][ or VAD (Valence, and manually annotate3d0, [31, 32]. This approach has
For the Italian language several language resources iological activation, ranging from sleep to excitemweintht)emotional annotations have been produced over the anddominance (the degree of control a person feels ovyeerars. The initial ItEM lexico3n3][began by collecting a situation) explain the majority of the emotional mseeaend-words through an association task linking words to ing of words. This approach has been highly productilvaebelsP(lutchi’ks basic emotions), then employed cosine for research on emotional language and the creatisoinmiolfarity to expand the lexicon, assuming that neighlanguage resources, exemplified by the ANEW (Afective boring words in semantic space share similar emotional
0000-0003-2154-2696 (E. D. Palma) © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
connotations. The results, validated through crowdsourcing, showed low reliability for the emottirounsst, anticipation (translated as ‘attese’) asunrdprise. The more recent Depeche Mood ++34[] was automatically created from judgements given by readers of articles on the ‘Corriere della Sera’ newspaper website and uses a unique scale of emotions not directly comparable to(eo.tgh.-‘beh’, ‘boh’) have been included. ers, such as ANNOYED, AFRAID, SAD, AMUSED, and Consistent with the researchMoofntefinese et al.and
create resources by adapting the English model withmaant-ical meanings. notations in L1 languages other than English has resulted in Afective Norms for several languages, including SpaAnn-notation Schema In order to collect a versatile ish [35] and Dutch 3[6]. For Italian, there has been adataset adaptable to diferent research approaches, the specific adaptation of the ANEW collected3b7y].[ data collection involved an annotation process that in
table gap persists. There is a lack of manually [a2n6]- and the evaluation of the items according to VAD notated Italian language resources that combinedbiomtehnsions [20]. In the case of basic emotions, it was discrete emotion annotations and dimensional evadleucaid-ed to use the translation ‘aspettatiavna’tifcoirtions. Furthermore, no available resource providpeastaion instead of ‘attese’ as in ItEM in order to avoid non-aggregated version of the data. misunderstandings and associations of the “attese-treno”
This paper presentEsLIta (EmotionLexicon forItal- type. Furthermore, to provide additional context for the ian), an innovative resource designed for the analaynsaislysis, participants were asked to share their demoof emotions in the Italian language and emotion degtreacp-hic information. tion from textE.LIta aims to bridge this gap by providing a lexicon annotated using both categorical anDdadtai-collection Data were collected from two primary mensional approaches, and by ofering a non-aggregatesdources from April 2023 to May 2024. An online quesversion of the data. This aligns with the perspectitviiosntnaire in the form of a webs1ictreeated from scratch viewpoint, which values disagreement as valuablewina-s used to rate the words. The website was shared formation3[8, 39, 40, 41]. The development process offor annotation via mailing lists (such as LinguistList and ELIta, including lexicon selection, annotation methAoILdC-) and social networks. The participation was on a ologies, and a comparative analysis with existing Itavloialnuntary basis and without payment. In this system, sentiment lexicons, is thoroughly described. Finally, atnhael-words to be rated in each questionnaire were ranyses of the relationships between emotions and betwedeonmly chosen. That is, each time the questionnaire was dimensions and gender are presented. accessed, the system randomly chose the words from the entire list o6f,905 words. Thus, each participant rated a
Lexicon Selection The lexicon for this study is cont-o an informed consent. Then, they were given the guidestructed from existing resources in the literaturel.inTehse for both categorical and dimensional annotation. major pool from which it draws is De Mauro’s ‘NuovOon the third screen, they had to provide the demographic Vocabolario di Base’ (NVdB4)2[]. This selection is made information concerning age and gender, and select the by reason of its representativeness of contemporarytIitmael-slot to spend on the annotation process (from 3 to 10 ian language usage in diferent types of text. In line wwitohrds, with the possibility of extending the annotation EmoItaly 4[3], 186 emoji have been added to the lexiconprocess at the annotator’s discretion). so that it can also be used for texts from Social NetworPkasr. ticipants were asked to rate the extent to which
Furthermore, as a foundational layer, the seed-wordeascohf word is associated to a list of emotions, using a scale ItEM [33] were incorporated. To ensure broad coverafgreo,m “non associated0”),( “weakly associated”0.2(5), high-frequency words (recurring more than 200 tim“ems)oderately associated0”.7(5) to “strongly associated” from the Depeche Mood ++3[4] lexicon byAraque et al. (1) [17]. Next, participants were shown the dimensions were included. using the Self-Assessment-Manikin from the ANE2W1] [
The selection process favoured content words (ve(rsebes,Fig. 1) to assess the extent to which each word connouns, adjectives and adverbs) over function wordsv(edye-valence, arousal, anddominance using a 1 to 9 terminers, conjunctions). scale. The guidelines in the latter case were mutuated
The final lexicon comprises6905 items, including bothfromMontefinese et al(.2014). words and emojis. The data set contains 21 % adjectivesA,dditionally, the Prolific platform was used to recruit 50 % of nouns, 21 % verbs and 8 % of words that can nbaetive Italian speakers as participants from Marc2h. 2024 considered both as adjective and noun. In addition, a smaller number of adverbs, expressions (e.g. ‘restar1ehtatps://emotionlexicon.com/ bocca apertab’e looking open-mouthed) and interjections2In this case, the annotators were paid according to the rules established by the platform.
RAW Version including all annotations and demographic information with an inter-annotator agreement (calculated with Krippendor45f[]) of0.67, which can be explained by the subjective nature of the task (associating words with emotions in isolation). Various factors such as gender, age and socio-cultural background can influence the IAA in such subjective tasks. Description of ELIta The collected data underwenINtTENSITY One of the aggregate versions created from the golden version retains the intensity values of a rigorous filtering process to ensure quality and accuracy. Participants with exceptionally fast completthieonoriginal annotations, with the single value calculated times were excluded. Additionally, despite the subajesct-he average of the six annotations (five original + one tive nature of the task, annotations with clear anomgoalldieesn). The decision to use the golden version is to balance the few annotations with one representative of the were removed, such as associations deemed illogical (e.g., ‘worsening’peggioramento strongly associated wjiothy). majority. In this case, the labelsloovfe automatically
The total number of annotations gather3e5,d41is2. calculated from the valuejsooyfandtrust and ‘neutral’ For each of th6e905 words/emojis in the lexicon, fromwere also added. a minimum of 5 annotations to a maximum 1o0fannotations were collected (on ave5r.a1g3eannotations per BINARY The second aggregated version, converts the word). aggregated float values to integers, providing a binary
From the demographic metadata, it can be observreedpresentation of the basic emotions: 0 for values below that the majority of annotations come from women 0a.n50dand 1 for values above 0.50. the most frequent age group is 25-34 years old (see table 1).
developedELIta lexicon, it was conducted a comparative analyses with other language resources for Italian: EmoLex (NRC-AIL) [46], ItEM [33], and ANEW [37]. The
and VAD dimension against the italian translation of
cosine values of ItEM33[], and the dimensions of the
ELIta vs. EmoLex Comparing the2, 388 shared items, In contrast, terms like ‘unpleasaspniatc’evole, ‘discourthe results showed a moderate correlati0o.5n1.ofjoy aged’scoraggiato, ‘boredom’noia, ‘cold’freddo, and ‘rain’ exhibited the highest correlatio=n0(.65 ), whilean- pioggia are associated with less activation in the ELIta ticipation ( = 0.38 ) andsurprise ( = 0.35 ) showed lexicon. the lowest. The results show even more the need to useFordominance, there is an increased sense of domlexicons specifically created for the target languagei.nance associated with terms such as ‘hatordeiod,’‘optimism’ ottimismo, ‘triumph’trionfo, ‘triumphantt’rionELIta vs. ItEM With3, 299 shared items, Pearson cor-fante, ‘to sleep’dormire and ‘to travevli’aggiare. Conrelations were calculated between the degree of vaesrssoe-ly, the sense of submission is associated with terms ciation of ELIta for each basic emotion and the colsiiknee‘eartht’erra, ‘naturen’atura, ‘circus’circo, and states similarities between the words and emotion-labels ofotfhiellness such as ‘feveferb’bre. basic emotions. Correlations were generally low, witThhese diferences may reflect the diferent sensibilities the highest foarnger ( = 0.29 ). The lower correlationsof the annotators. The afective norms3o7f] [were pubare in line with previous observations on the dificulltiyshed in 2014, while the majority of the annotators of of annotating emotions suchtrausst ( = 0.18 ), antic- the proposed ELIta lexicon belong to the age range of ipation ( = 0.14 ) andsurprise ( = 0.13 ). 25-34 years. ELIta can thus be seen as a limited update to the norms proposed bMyontefinese et al.(2014).
ELIta vs. ANEW The two resources shar7e62 items. Although generational characteristics may influence The analysis revealed a strong correla t=ion0.8(8 ) the results, it is important to consider that the compariforvalence, whilearousal ( = 0.48 ) anddominance son was based on the means of responses from approxi( = 0.61 ) showed lower correlations. The observed omuta-tely 20 persons for the Norms and 5 persons per word comes are consistent with research showairnogusal and for ELIta. The lower number of annotators for ELIta dominance as the dimensions most variabl3e5,[37]. could imply that the individuality and socio-cultural background of each participant have a greater impact on the
groups provided significantly diferent ratings, a linearersults. Therefore, further analyses should be conducted. regression was used. This statistical model allows to estimate the extent to which ELIta ratings can be predic3t.2ed. Correlations and Gender Variation by Afective Norms ratings and to identify the words for which this relationship is weak4e.r. Once the data as a whole had been analysed in compari
The results show a more negative connotation of wosrodnswith other lexicons, the annotations were analysed linked to the religious sphere, for example, ‘chuchricehsa’ to examine the relationship between the diferent emoand ‘god’dio have shifted from positive to negative. Simi-tions and dimensions, and whether there were diferences larly, ‘furp’elliccia ‘circus’circo and ‘justiceg’iustizia have between genders in the association between words and also transitioned from positive to negative. Converesmeolty,ions. the terms ‘lesbianle’sbica and ‘mad’folle have shifted from negative to positive. Correlations Firstly, Pearson correlations between cat
Examining the associations of these words with baesgicories and dimensions were calculated. (see2F).ig. emotions, it can be noted, for example, that the predomTi-he results show a moderate correlation between nant emotion associated with the word ‘churacnhg’iesr arousal and negative emotions, particulaferalry ( = with a mean intensity 0o.f54, followed bysadness and 0.45) andanger ( = 0.40 ). Consequently, the correlation disgust. Analogously, ‘fur’ is associated most stronbgeltyweenarousal andvalence turns out to be weakly with0.75 tosadness and with0.70 toanger, and ‘circus’ negative (= −0, 17 ). is more associated witdhisgust andsadness. The word Furthermore, it can be noticed that negative emotions ‘god’ presents an interesting contrast. Although ittehnadsto co-occur, suggesting that words associated with a negativevalence ( = 4.6 ) compared to the ANEW sadness may also be linked taonger, disgust, orfear result (= 8.3 ), the primary emotions associated with [it47]. Converselyj,oy shows a moderate to strong coraretrust (0.67) andanticipation(0.46). The word ‘les- relation witthrust ( = 0.66 ), anticipation ( = 0.62 ) bian’ does not appear to be associated with any emotiaonnd, surprise ( = 0.49 ). except very weakly witjhoy ( = 0.20 ), while ‘mad’ re- Interestingly, there is a moderate correlation between sults associated more wijtohy andsurprise ( = 0.42 ). dominance andjoy ( = 0.53 ), indicating that words with
Regardingarousal, terms such as ‘optimism’ot- positivevalence are also associated with a greater sense timismo, ‘erotic’erotico, ‘success’successo, ‘food’cibo, of control=( 0.7 ), while negative ones are associated to and ‘in lovei’nnamorato have shown increased activatioan.sense of submission (= −0.40 to = −0.53 ) [48]. An exception is given by words such as ‘nature’ which, as 4Plots are available in the appendix, see.6F.ig. we have seen, has a low rating (M = 3.5) fdoorminance 6 Dominance but is strongly associated wjoityh(M = 1). lsa 1−4
Surprise shows positive correlations both modestly rou5 4−6 withjoy andanticipation, and weakly withfear, and A4 6−9 trust, although it is a more neutral emotion than the oth3ers, it is generally more prone to have a posvitailveence ( = 0.32 ). 2
viously identified byFerré et al(2.016) for Spanish and 1 2 3 4 5 6 7 8 9
The correlations and regression analyses revealedFpigautr-e 3: Scatterplots of the distributions of ELIta according terns consistent with the other resources: a U-shatopeddominance and arousal dimensions, and valence and relationship betweevnalence andarousal, dominance arousal dimensions. The lines represent the linear regression andarousal (see Fig. 4, and a linear relationship beac-cording to the values before the valence median (in purple) tweendominance andvalence (see Fig. 3). These re- or the dominance median (in red), and after the valence sults suggest that highly negative or positive itemms,eadsian (in green) or the dominance median (in teal). well as words associated with low or high control, tend to elicit greater emotional and physiological activation.
Meanwhile, greater positivity corresponds to a gretaatteiron, a subgroup of words/emojis annotated by both sense of control. men and women (n=6, 219) was considered. For each
These analyses have positioneEdLIta as a valuable word, the mean emotional ratings provided by the difresource for emotional language research. Despite vfearrein-t gender groups were calculated. Subsequently, the ations in sample size, the data mirror the trends caonrdrelation between the mean ratings was assessed, and distributions observed in existing emotion analysisstlaitt-istical tests were conducted to identify any significant erature3[5, 21, 48, 22, 49]. ConsequentlyE,LIta can be diferences between the groups. considered a psychologically valid resource for emotioTnhe most significant diferences were found in annoresearch. tations oafrousal, with a correlation0o.2f0 and a statistically significant diference calculated using a t-test Gender variation Gender is a significant factor influ-with a − < 0.005 (M = 5.39 for women and M = encing the annotation of subjective constructs su5c.h13afsor men). As also reported in the literature, women emotions. Previous research has shown that men antednd to annotate words not only as more arousing, but women often respond diferently to the same stimuli also with more extreme values on the valence scale, i.e. [37, 21, 22]. rating unpleasant words as more negative and pleasant To investigate the impact of gender on emotion anwnoor-ds as more positive. 1 2 3 4 5 6 7 8 9
Valence valence also showed a significant diference (− = 0.017 ), with women assigning highearrousal and lowervalence ratings compared to men (M5=.08 for women and M =5.15 for men), although it showed a 0.4 stronger correlatio=n0(.64 ) than the other dimensions. à
b
significant diferences in dominance ( − > 0.05 , 0.2 = 0.30 ).
Regarding basic emotions, women reported significantly higher levelsfoefar ( < 0.001 ) and lower levels oftrust andsurprise (both < 0.01 ) compared to men, according to the t-test. For example, female participan0.0ts expressed significantly lower levelstorfust towards 2.5 Ra5ti.n0gs 7.5 relationship-related words than male participants, with mean scores for ‘partnepra’rtner, ‘spouse’ sposo, and Figure 5: Dimensions distribution in the annotations of men ‘wedding’nozze ranging from 0.5 to 0.87 compared t o(bottom) and women (top) mean male rating of 1.
These findings indicate that gender significantly influences emotion annotation, particularlayrofoursal and the data, reflecting a ‘perspectivist’ approach that values valence (see Fig. 5). The outcomes again corroboratdeisagreement as valuable information, such as women trends observed in the literature for other lan4g9u]a,gessh[owing a greater tendency towards negavatliveence underlining the importance of ofering non-aggregataendd higherarousal ratings than man. Analyses using resources to better represent the diferences betweceonrrelations between basic emotions and dimensions, speakers. along with comparisons to existing resources such as
understanding of the interplay between emotions and 4. Conclusions language. WhilEeLIta represents a significant step forward in capturing the complexity of emotion-language This research introduces a new lexicon for Italianitnhtaetractions in Italian, continued development will be collects word-emotion associations. Notably, it is the first
essential to addressing its current limitations and maxilexicon, to the authors’ knowledge, to be annotated umsiiznigng its utility as a comprehensive tool for emotional both categorical and dimensional approaches. Furtahnearl-ysis. more, it ofers an innovative non-aggregated version of guistics, Los Angeles, CA, 2010, pp. 26–34. URL: coverage emotion annotation, Lang. https://aclanthology.org/W10-0.204 Resour. Eval. 56 (2022) 857–879. URL: [18] J. A. Russell, A circumplex model of afect., Jour- https://doi.org/10.1007/s10579-021-09565-.1 nal of Personality and Social Psychology 39 (1980) doi:10.1007/s10579-021-09565-1.
1161–1178. [31] K. Kajava, E. Öhman, H. Piao, J. Tiedemann, [19] J. A. Russell, A. Mehrabian, Evidence for a Emotion preservation in translation: Evaluating three-factor theory of emotions, Journal of datasets for annotation projection, in: Proceedings
URL: https://www.sciencedirect.com/science/ Nordic Countries, 2020, pp. 38–50. article/pii/009265667790037.X doi:https: [32] B. Maia, D. Santos, Language, emotion, and the //doi.org/10.1016/0092-6566(77)90037-X. emotions: The multidisciplinary and linguistic [20] A. Mehrabian, Pleasure-arousal-dominance: A gen- background, Language and Linguistics Compass eral framework for describing and measuring indi- 12 (2018) e12280. doi:https://doi.org/10.1111/ vidual diferences in temperament, Current Psy- lnc3.12280, e12280 LNCO-0696.R3.
chology 14 (1996) 261–292. [33] L. Passaro, L. Pollacci, A. Lenci, Item: A vector [21] M. M. Bradley, P. J. Lang, M. M. Bradley, P. J. Lang, space model to bootstrap an italian emotive lexi
struction manual and afective ratings, Technical tional Linguistics CLiC-it 2015, Academia UniverReport C-1, The Center for Research in Psychophys- sity Press, 2015, pp. 215–220.
iology, University of Florida, 1999. [34] O. Araque, L. Gatti, J. Staiano, M. Guerini, De[22] A. Warriner, V. Kuperman, M. Brysbaert, Norms pechemood++: A bilingual emotion lexicon built of valence, arousal, and dominance for 13,915 en- through simple yet powerful techniques, IEEE glish lemmas, Behavior research methods 45 (2013). Transactions on Afective Computing 13 (2022) doi:10.3758/s13428-012-0314-x. 496–507. doi:10.1109/TAFFC.2019.2934444. [23] S. M. Mohammad, Obtaining reliable human ra[t3-5] J. Redondo, I. Fraga, I. Padrón, M. Comesaña, The ings of valence, arousal, and dominance for 20,000 spanish adaptation of ANEW (afective norms for english words, in: Proceedings of The Annual Con- english words), Behav. Res. Methods 39 (2007) ference of the Association for Computational Lin- 600–605.
guistics (ACL), Melbourne, Australia, 2018. [36] L. J. Speed, M. Brysbaert, Ratings of valence, [24] P. Ekman, An argument for basic emotions, Cogni- arousal, happiness, anger, fear, sadness, disgust, tion & emotion 6 (1992) 169–200. and surprise for 24, 000 dutch words, Be[25] P. Ekman, Emotion in the human face, 2 ed., Cam- havior Research Methods (2023). URhLt:tp://dx.
bridge University Press, New York, 1982. doi.org/10.3758/s13428-023-02239-6.doi:10.3758/ [26] R. Plutchik, A general psychoevolutionary theory s13428-023-02239-6.
of emotion, in: Theories of emotion, Elsevier, 198[03,7] M. Montefinese, E. Ambrosini, B. Fairfield, N. Mampp. 3–33. marella, The adaptation of the afective norms for [27] L. F. Barrett, K. A. Lindquist, M. Gendron, Language english words (anew) for italian, Behavior Research as context for the perception of emotion, Trends in Methods 46 (2014) 887–903.
Cognitive Sciences 11 (2007) 327–332. URLh:ttp:// [38] V. Basile, It’s the end of the gold standard as dx.doi.org/10.1016/j.tics.2007.06.0 0.3doi:10.1016/ we know it. on the impact of pre-aggregation j.tics.2007.06.003. on the evaluation of highly subjective tasks, in: [28] F. Bianchi, D. Nozza, D. Hovy, FEEL-IT: Emotion DP@AI*IA, 2020. URL: https://api.semanticscholar. and sentiment classification for the Italian language, org/CorpusID:229344921. in: Proceedings of the Eleventh Workshop on Com[3-9] V. Basile, M. Fell, T. Fornaciari, D. Hovy, S. Paun, putational Approaches to Subjectivity, Sentiment B. Plank, M. Poesio, A. Uma, We need to consider and Social Media Analysis, Association for Compu- disagreement in evaluation, Proceedings of the tational Linguistics, Online, 2021, pp. 76–83. URL: 1st Workshop on Benchmarking: Past, Present and https://aclanthology.org/2021.wassa. -1.8 Future (2021). URLh:ttps://api.semanticscholar.org/ [29] R. Sprugnoli, Multiemotions-it: A new dataset for CorpusID:236486317.
opinion polarity and emotion analysis for ital[i4a0n], J. Baan, R. Fernández, B. Plank, W. Aziz, Interpretin: Proceedings of the Seventh Italian Conference ing predictive probabilities: Model confidence or on Computational Linguistics (CLiC-it 2020), Ac- human label variation?, 2024. URhLt:tps://arxiv. cademia University Press, Torino, 2020, pp. 402–408. org/abs/2402.1610 2. arXiv:2402.16102.
[30] D. Santos, A. Simões, C. Mota, Broad with disagreements: Looking beyond the majority
Montefinese et al.. For each dimension, it is possible to see the regression line and the words that are furthest from the line, i.e. the words that were rated diferently by the annotators between the two lexicons.
fucile cimitero folle vampiro
perso rivolta insulto
penetigre lesbicaassurdo m ufrteadzdioone
vespa anasffiloizsione protetto
utile facile curioso lafuvnagreo
gioioso madre scusa pelliccia
lago gicuisrctiozia impiego chiesa ristorante dio 1 2 3 4 6 7 8
9 5
ANEW aria scrittore
luce leggenda mondo
erotico viaggiare innamuosraisctouaccesso
ottimismo leacucreczaeitorovntearliloeonfo cloibraegragzi oiosnoe strillo guerra statua vcaofislolore
orgasmo scoraggiato dicsipciaatcricueto tagliereocesagnraodevole motrourveido pioggia caviglia freddsorpresa sorfraiscocia noia 2 4 3 2 1 odio leone aria
libreria sporcizia cimitero tossico vittoroiattimismo
trionfante cfaofacralnilbgetaegsriotiàaso trvioanftoaggio
tvailaegngtoiare dormire
circo terra danneggiare schipaeffroicolo sdifmeilipdbreaersaisnziozsaincteuoro impotente 8