Exploring YouTube Comments Reacting to Femicide News in Italian

Abstract

In recent years, the Gender Based Violence (GBV) has become an important issue in modern society and a central topic in diferent research areas due to its alarming spread. Several Natural Language Processing (NLP) studies, concerning Hate Speech directed against women, have focused on misogynistic behaviours, slurs or incel communities. The main contribution of our work is the creation of the first dataset on social media comments to GBV, in particular to a femicide event. Our dataset, named GBV-Maltesi, contains 2,934 YouTube comments annotated following a new schema that we developed in order to study GBV and misogyny with an intersectional approach. During the experimental phase, we trained models on diferent corpora for binary misogyny detection and found that datasets that mostly include explicit expressions of misogyny are an easier challenge, compared to more implicit forms of misogyny contained in GBV-Maltesi. Warning: This paper contains examples of ofensive content.

Keywords

• eol>Hate Speech
• Misoginy Detection
• Femicide
• Social media
• News
• Responsibility framing

1. Introduction

statistics become even more alarming when we consider studies that show the correlation between misogynistic Nowadays, the term Gender Based Violence (GBV) is online posts and GBV [ 4 ]. used to identify all forms of abuse based on gender hatred Like other countries, Italy is afected by GBV, with the and sexist discrimination [ 1 ]. Scholars in social science national observatory managed by the “Non Una di Meno” have defined as “rape culture” the society that normalizes association reporting 117 femicides in 2022, 120 in 2023 sexist behaviours: from more common occurrences like and more than 40 until June 20243. victim blaming, slut shaming and gender pay gap to the Several studies about Hate Speech (HS) directed toapex of violence with femicide [ 2 ]. While general vio- wards women often focus on developing taxonomies [ 5 ] lent crimes decreased over time, GBV did not, alarming rather than investigating low resource subjects in comvarious bodies in modern society1. A report from the EU putational linguistics like GBV. These works often gather commission2 states that 31%, 5% and 43% of European corpora by keyword search of gender slurs [6], retrieving women sufered respectively from physical, sexual and comments left on misogynistic spaces like incel blogs psychological violence. Regarding the Internet sphere, a [ 5, 7 ] or considering messages directed towards popular survey found that 73% of women journalists experienced women figures highly debated on social media [8]. online violence (threats, belittling, shaming,...) [ 3 ]. These As GBV is a broad topic, we want to clarify that we foCLiC-it 2024: Tenth Italian Conference on Computational Linguistics, cus on GBV in Western societies, particularly in Italy. The Dec 04 — 06, 2024, Pisa, Italy main goal of this project is to show what is the current * Corresponding authors. perception of femicides expressed through comments on † These authors contributed equally. social media, focusing on the specific case of Carol Mal$ [e-mail] (C. Ferrando); tesi. We chose this femicide because the victim was a [e-mail] (M. Madeddu); sex worker, meaning that she presented an intersectional sbveeavtraiscielv.aina.tpoalas@[e-mail](Ba.viAa.nitto(Sla.)S;. Pasini); trait, and it was a popular case in the media, enabling [e-mail] (G. Telari); [e-mail] us to select enough material for the study. Further, we (M. Lai); [e-mail] (V. Patti) want to highlight how the socio-demographic character© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License istics of the victims determine the way they are described Attribution 4.0 International (CC BY 4.0). 1https://www.interno.gov.it/it/stampa-e-comunicazione/ and how this influences the perception of the news. For 2dhatttpi-se:/-/sctoatmismticishseio/onm.eiucridoip-av.oeluo/nsttararit-eeg-yv-iaonledn-zpao-lgiceyn/ere instance, victim’s features such as age, job, origin, skin policies/justice-and-fundamental-rights/gender-equality/ gender-based-violence/what-gender-based-violence_en 3https://osservatorionazionale.nonunadimeno.net/anno/ color, nationality, religion have diferent weight and de- Misogyny has become a pervasive phenomenon, termine the lesser or greater spread of the news [ 9 ]. To widespread in very diferent spheres and expressed in overcome the cited issues in current literature, in this both explicit and implicit forms [ 5, 18 ]. For this reason, research we considered the phenomenon by focusing on even in online conversation about a dramatic act such users’ reactions in social media to news about femicides. as femicide, it is possible to find examples of veiled or We collected YouTube comments in response to videos explicit hostility towards the victims. The femicide phetalking about a specific case. In order to overcome the nomenon has been studied from diferent points of view. constraints of traditional sentiment analysis schemas, we Several studies focused on GBV representation in Italian annotated the data following a new semantic grid that media [ 19, 20 ]. In 2020, Mandolini focused on the journalcan be used as a standard for comments regarding GBV. istic narratives of femicide in newspapers by means of a

In the experimental phase of this work, we created qualitative discourse analysis on two specific case studies models based on diferent Italian misogyny datasets (in- [21]. The researcher attempted to describe changes in cluding ours). The goal of such experiments is to analyze attitudes in the portrayal of femicide, focusing on disthe diferent features of these corpora and what forms cursive strategies that (directly or indirectly) blame the of misogyny are harder to detect. We performed both a victim and implicitly excuse the perpetrator, referring to quantitative and qualitative analysis of the results. gender stereotypes and romantic love rhetoric.

In the next sections, we describe: related work on hate Other studies focused on the responsibility framing speech and misogyny detection(Section 2), the annota- in femicides news, by conducting an experiment where tion scheme and both a quantitative and qualitative anal- annotators rated excerpts from local newspapers on how ysis of the dataset (Section 3), and the results obtained much responsibility was given to the perpetrator [22]. in our experiments (Section 4). Lastly, we present some As far as we know, there is only one line of work in NLP conclusions and delineate possible future developments on GBV [23, 24, 25], which focuses on reader’s percep(Section 5). tion of femicide news headlines and analyses the perception of responsibility attributed to victim and perpetrator; whereas, to our knowledge, there is no other study 2. Related Work analysing social media reactions to GBV cases. In recent times, the creation and dissemination of hate speech are increasingly pervasive on online platforms, 3. Dataset making social media a fertile ground for hateful discussions [ 10 ]. The escalation of ofensive and abusive lan- 3.1. Corpus Background guage, understood as content that discriminates a person or group on the basis of specific characteristics such In a preliminary phase of our work, we conducted a as ethnicity, gender, sexual orientation, and more has research on the femicide case of Sara Di Pietrantonio4, 22 aroused considerable interest in various fields. In fact, years old, a white Italian student, from a wealthy family, over the last decade, a large number of computational murdered by her ex boyfriend on May 2016 [21]. In this methods involving NLP and Machine Learning have been preliminary research we set out to develop a corpus by proposed for automatic online hate speech detection collecting Twitter users’ comments to femicide news on [11, 12]. Most of prior works have mainly considered newspapers published online 5. We created an annotation hate speech as a classification task, by distinguishing scheme for the data corpus consisting of two layers: the between hate and non-hate speech. Hate speech takes first focused on the dimensions of sentiment analysis on diferent nuances depending on the target groups at and composed of three subtasks (subjectivity, polarity which it is directed, i.e. depending on the specific features and irony), relevant for the detection of sentiment in that the target group have in common. Moreover, in some social media [26]; the second focused on hate speech cases, these traits may intersect with each other, leading detection, including labels for misogyny, aggressiveness to diferent degrees of discrimination. This concept takes and its target. For more details on the annotation scheme the name of intersectionality [13]. and corpus description, please read below Appendix A.

Among abusive languages, misogyny, considered as a Observing the results of the preliminary study, we specific ofensive language against women, has become discovered how the victim’s characteristics influence a contemporary research topic [ 14 ]. In automatic hate the way newspapers present her femicide and users talk speech detection field, the Automatic Misogyny Identi- about it on social media. In fact, analyzing Di Pietrantoifcation (AMI) [ 15] series of shared tasks launched in nio’s case, as she was a young, white, wealthy and Italian EVALITA [6] and the SemEval-2019 HatEval challenge [16] have produced evaluation frameworks to identify misogynous tweets in English, Italian and Spanish [17].

4https://www.agi.it/cronaca/news/2019-09-11/sara_di_

pietrantonio_processo_tappe-6170806/ 5the dataset is available at https://github.com/madeddumarco/ GBV-Maltesi student, we found very few examples of misogyny and, voluntarily participated to the project. The annotation in most cases, the aggressiveness was directed against guidelines were decided with the annotators after a pithe perpetrator. Furthermore, the scheme was not con- lot study and a subsequent group discussion where the sidered suficiently suitable for bringing out important raters pointed out the main faults of the schema. Each elements of femicide cases. In fact, the annotators ex- annotator analyzed all the comments according to the pressed their dificulties caused by the scheme developed following guidelines: as it was deficient and too simplistic to recognise complex features of femicide events. In order to solve these issues, we decided to direct our eforts on another case study in which the victim exhibits intersectionality traits, which we assume may lead to more misogynistic content.

In addition, we developed new schema and guidelines to have more accurate annotations specifically related to the femicide domain.

3.2. Data Collection

In this section we provide a description of the new dataset

built and the methodology used.

As mentioned above, we focused our research on the femicide of Carol Maltesi6, a 26 years old, white Italian woman, mother and online sex worker, who was brutally murdered in January 2022 by her ex partner, Davide Fontana, a 44 years old white Italian bank employee.

With the aim of collecting users’ responses to femicide, we chose to collect comments using YouTube Data API, as it is freely available and allows us to easily access comments focused on specific news. The process of obtaining data followed several steps: first, we selected the 31 most popular YouTube videos based on number of views and comments. We chose videos about Maltesi femicide from diferent types of sources: national (mainly the Italian broadcaster RAI) and local news. The selection of videos is diachronic spanning from March 2022 to June 2023; this was done because the various media channels covered the story as it evolved starting from the discovery of the nameless body and ending with the sentence given to the perpetrator. Afterwards, we collected comments from all the videos selected. Due to the API policy, we were restricted to collect only first-level comments and at most 5 oldest responses to them. In total, we retrieved 3,821 comments.

3.3. Annotation Scheme

From the previous experience of the Di Pietrantonio corpus, we decided that a generic sentiment analysis schema proved to be too rigid to understand such a complex phenomenon. We created an annotation scheme and a new online platform to facilitate the raters work. We involved 5 annotators, 4 of them self-identified as women and 1 as a man, all interested in the topic and mostly coming from humanistic background. They were all students and 6https://www.agi.it/cronaca/news/2024-02-21/ omicidio-maltesi-condannato-ergastolo-ex-davide-fontana-25397937/7https://github.com/madeddumarco/GBV-Maltesi • Non classifiable: if the comment cannot be analysed because it is not written in Italian, because it consists only of emojis, because it is not comprehensible or not relevant to the topic (any comment that was marked as NC from at least 1 annotator was removed from the corpus); • Empathy: whether, in the comment, there are expressions of empathy in support of the victim, her family or the event in general (i.e., condolences); • Misogyny: whether, in the comment, there is a presence of discriminatory expression against women, including blaming, objectifying, discriminatory and sexist practices used towards them and their life choices. If misogyny is present, we asked annotators to indicate its target (group or individual) based on [16]. Moreover, we asked to specify if the expressed misogyny contained intersectionality traits and to select from a list what other dimensions were involved: age, religion, job, nationality, skin color, class, sexual orientation, gender, physical condition, educational background, language and culture; • Aggressiveness: whether there is aggressiveness in the comment and to whom it is directed (allowing multiple choices): victim, perpetrator, social network (family, friends, colleagues), media, rape culture; • Responsibility: if there is explicit attribution of responsibility for the murder in the text, state who is blamed (allowing multiple choices): victim, perpetrator, social network (family, friends, colleagues), media, rape culture; • Humor: specify whether the text conveys humorous content through irony, sarcasm, word games or hyperbole; • Macabre: specify whether there are macabre as

pects detailing how the victim was killed; • Context: indicate whether the context was helpful to better understand the meaning of the comments; • Notes: free space for suggestions, observations or

doubts.

3.4. Dataset Analysis

The dataset, GBV-Maltesi 7, is composed of 2,934 comments annotated on all categories by all annotators. We (a) Distribution of the misogyny label and its subcategories (b) Distribution of the aggressiveness label (c) Distribution of the responsibility label aggregated dimensions through majority voting. As our as they lacked ambiguity. On the other hand, we can see schema is composed by many diferent labels, we will that aggressiveness towards the victim is much lower focus only on the dimensions that we consider the most (0.28). In our discussions with the raters, it emerged how relevant, but all statistics can be found in Appendix C. attacks towards the victim were harder to identify as

Starting from misogyny, in Appendix C and in Figure they were more subtle leading to disagreement among 1a, we can see that 9.03% of cases are positive. This un- annotators. balance is typical of hate speech datasets [27] and we consider it surprisingly high if we take into account the tragic theme of GBV. It is very interesting that intersec- 4. Experiments tionality represents over 50% of misogynous examples indicating how the personal traits of the victim afect We conducted experiments to validate our resource and the perception of the users commenting. Unsurprisingly, to gain more insight into the dificulty of the misogyny as the victim was a sex worker, ‘work’ is almost always detection task. The goal of this analysis is to understand the category chosen by the annotators. The target of how the presence of diferent forms of misogyny (implicit misogyny was mostly individual, confirming the findings and explicit) afect the evaluation of modern classificaof SemEval-2019 Task 5 [16]. The annotators explained tion models. We consider as explicit misogyny discourses to us how the misogyny target was a dificult category that intentionally spread hate towards women mostly to annotate as often comments used the victim as an through slurs and other aggressive behaviors. Meanexample to ofend the broader group of women and sex while, we intend implicit misogyny as more subtle and workers. less conscious practices like victim blaming, slut sham

Aggressiveness is more present than misogyny in our ing, de-responsibilization of the perpetrator and more. In dataset, with 24% positive examples mostly directed to- addition to our corpus, we used 3 other datasets regardwards the perpetrator. Responsibility follows a similar ing the topic in Italian: AMI [6], PejorativITy [29] and trend with 32.89% positive examples most directed to- Inters8 [8]. The former two have been mainly gathered wards the perpetrator. Unlike aggressiveness, we can by keyword search of sexist terms8, meanwhile, Inters8 see a significant amount of comments holding the victim and our corpus are focused on more implicit forms of responsible (6.55%). sexist hate directed towards a specific woman (i.e., Silvia

In Appendix B, we reported the inter-annotator agree- Romano and Carol Maltesi). Details about all the datasets ment (IAA) scores for all dimensions. As our dataset is can be found in Appendix D. fully annotated by multiple people, the metric we chose To explore the potential bias of models towards explicit is Fleiss’ Kappa [28]. The metric has a possible range of forms of misogyny, we created 4 diferent models for [ -1,1 ], with 1 indicating perfect agreement, and any value binary misogyny detection: BERT-Maltesi, BERT-AMI, onfotat≤ ors0tihnadniceaxtpeescmteodrbeydicshaagnrceee.mWeentcabnetsweeeetnhatthmeoanst- tBivEeRlTy-tPraeijnoeradtoivnI TthyeaGnBdV,B-MERalTte-Isni,teArMs8I,tPheajtowraetriveIrTeyspaencddimensions have a in the [0.2, 0.7] range, indicating variable levels of agreement depending on the label. The 8AMI is created following an hybrid approach selecting also comdimensions with the highest agreement at 0.69 are em- ments from known misogynistic accounts and responses directed pathy towards the event and aggressiveness towards the wtoefefomuinndistthpautbltihcefigmuriesso.gWyneycocnodnutacitneeddaiqsuaallmitoatsitvealawnaaylysseisxpanlidcit perpetrator. In fact, annotators explained to us that these and depending on slurs. This lead us to place it in the keyword two categories were the easiest phenomena to annotate category. Inters8 datasets. The models were just trained on the com- most challenging datasets. This is especially true when ments and were not given any other extra-information observing the average f1-score on the positive label with such as video transcriptions. The only label we analyzed the score being in the [0.2, 0.3] range, compared to much was misogyny and all datasets were divided in training, higher scores for PejorativITy and especially AMI. These validation and, test sets following a 60%, 20% and, 20% trends indicate how misogyny detection is a much harder split. We used the existing splits when provided in the task when considering datasets that contain less explicit papers9, else, we randomly created them. All models are forms of hate (e.g., not gathered by keyword search of binary classifiers created by fine-tuning BERT [ 30], in sexist slurs). particular we used the Italian version AlBERTo [31]. Due In addition, we conducted a qualitative analysis on the to the imbalanced nature of most corpora, the models errors of the various classifiers. We found that for each were trained with a focal loss [32] setting the hyperpa- test set most classifiers misclassified the same type of rameter = 2. Models were trained for 5 epochs but, examples. Models almost never recognized texts which to avoid overfitting, we implemented an early stopping contained victim blaming and slut shaming in the GBVfunction which ends training after 2 epochs that report Maltesi Dataset. The errors made on Inters8 mostly coinan increase in validation loss. We tested all models on cide with examples that are also racist and Islamophobic. their own test set and the other 3 corpora. The cases which proved to be more dificult in Pejora

We want to underline that our goal is not to compare tivITy and AMI contain less explicit animal epithets like performance of the diferent models between each other “cavalla” and nouns that refers to sex worker in a less as they have diferent number of training sets and positive explicit way like “cortigiana”. examples. Rather, we intend to focus on how diferent test sets are more dificult compared to others which helps us understand what the current challenges in misogyny 5. Conclusion and Future Works detection are.

In Table 1, we reported the positive label and the macro In this paper, we presented GBV-Maltesi which is the average f1-scores of all experiments. In addition, we also ifrst dataset regarding social reactions to GBV, in particcalculated the average scores for each test set. The best ular to a femicide case. The topic was chosen to shed scores achieved on a certain test set are in bold, mean- light on the importance of having misogyny corpora while, we underlined the best scores for cross-dataset that include forms of sexism that are more implicit and testing. As expected, we can observe that all models had complicated to detect compared to the existing ones the highest score for their own set. Meanwhile, for cross- that focus on slurs and ofensive terms. We also fodataset testing, we can see that the models that tend to cused on the intersectionality aspects to better explore perform the best are BERT-PejorativITy and BERT-AMI. online hate. GBV-Maltesi is composed of 2,934 comWe suspect that this is caused by the dataset composi- ments all annotated by 5 annotators and it is available at tion as their training sets present more positive examples https://github.com/madeddumarco/GBV-Maltesi. In orcompared to the others. der to overcome limitations of generic semantic schema,

Interestingly, we can observe that certain models the corpus has been annotated following a new schema recorded higher scores on other test sets that were not specifically created for cases of GBV. In the experimental their own. This mostly happens when focusing on BERT- phase of our work, we created diferent misogyny binary Maltesi and BERT-Inters8, which record higher scores on classifiers and tested them in a cross-dataset way. We AMI and PejorativITy. Even PejorativITy increseases its found that datasets gathered on keyword collection are scores when tested on AMI. Observing the average scores easier benchmarks as the model showed bias towards for each test, we can see that Maltesi and Inters8 are the slurs and not identifying more implicit cases of misogyny. This research on online discourse about GBV is not meant to be exhaustive, as several questions are still open.

9PejorativITy provides a training and test split, but analyzing the

code we found that the test set was used as a validation set so we decided to create a new one.

As future works, we intend to focus on how diferent framing of news can cause diferent online reactions, analyzing the diferences between video transcripts of femicide news and the comments collected, in terms of words used, implicit references, attributions of guilt and descriptions of the people involved in the story. We also intend to gather more annotated corpora regarding femicides to explore how other characteristics of the victim (e.g., origin or skin color) and time of the murder diferently influence the online reactions. In this regard, we intend to explore the question by investigating whether and how the discourse on misogyny changes depending on whether it is addressed to living or dead women (i.e., Giulia Cecchettin femicide and abusive discourse against her sister, Elena Cecchettin). Lastly, we would like to extend our research by following an intersectional approach, considering all the dimensions and characteristics that make up the identity of both victim and perpetrator. To conclude, we strongly advocate the importance of write the news correctly, as this has deep consequences on the readers’ perception and the way they talk about it. Subjectivity Misogyny Polarity-Negative Polarity-Positive Aggressiveness Irony Context

Yes % Misoginy Target Intersectionality Aggressiveness Agg. Perpetrator Agg. Victim Agg. Social Network Agg. Media Agg. Rape Culture Responsibility Resp. Perpatrator Resp. Victim Resp. Social Network Resp. Media Resp. Rape Culture Empathy towards the event Humor Macabre Context Misoginy Intersectionality Aggressiveness Agg. Perpetrator Agg. Victim Agg. Social Network Agg. Media Agg. Rape Culture Responsibility Resp. Perpetrator Resp. Victim Resp. Social Network Resp. Media Resp. Rape Culture Empathy towards the event Humor Macabre Context

A. Details about the Di

Pietrantonio Dataset

The dataset GBV-DiPietrantonio is composed of 691 tweets fully annotated by 3 annotators, 2 of which selfidentified as women and 1 as a man. The tweets were collected by gathering responses to news which covered the news of Di Pietrantonio femicide. The annotation scheme is composed of the slightly modiefid SENTIPOLC scheme[33] which consists of Subjectivity, Polarity (Positive, Negative) and Irony. In addition the semantic grid contained Misogyny, Aggressiveness and Target of Aggressiveness (towards Perpetrator, Victim, Other), Context, and Notes.

The statistics of the gold standard for the Di Pietranto- Table 4 nio dataset are in Table 2. Distribution of the binary dimensions of the Maltesi Dataset

B. Agreement of the Maltesi

Dataset

C. Distributions of the Maltesi

Dataset

Table 4 contains the distribution of the binary labels in the Maltesi dataset. Table 5 contains the type of intersectionality and table 6 contains the type of misogyny target. Work Age Work and Education Work and Gender 96.32% 0.73% 0.73% 2.20%

D. Distributions of the Misogyny

Dataset

Table 7 contains the details of the other existing misogyny datasets used in the experimental phase.

Dataset Inters8 AMI Pejorativity

Topic Intersectional Hate focusing on Islamophobia in the case of hate towards Silvia Romano Misogynistic slurs, attacks towards important figures who expressed support for women rights and posts from misogynistic account Words that can be used as misogynistic pejoratives in online discussion (e.g.

Cavalla, cagna,...) 1,500 5,000 1,200 288 2,340 397

References

[1] M. L. Bonura , Che genere di violenza: conoscere e afrontare la violenza contro le donne , Edizioni Centro Studi Erickson , 2018 .

[2] C. Vagnoli , Maledetta sfortuna, Rizzoli, 2021 .

[3] J. Posetti , K. Bontcheva , D. Maynard , N. Aboulez , A. Lu , B. Gardiner , S. Torsner , J. Harrison , G. Daniels , F. Chawana , O. Douglas , A. Willis , F. Martin , L. Barcia , A. Jehangir , J. Price , G. Gober, J. Adams , N. Shabbir , The Chilling: A global study of online violence against women journalists , 2022 .

[4] K. R. Blake , S. M. O'Dean , J. Lian , T. F. Denson , Misogynistic tweets correlate with violence against women , Psychological science 32 ( 2021 ) 315 - 325 .

[5] E. Guest , B. Vidgen , A. Mittos , N. Sastry , G. Tyson, H. Margetts , An expert annotated dataset for the detection of online misogyny , in: P. Merlo, J. Tiedemann , R. Tsarfaty (Eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Association for Computational Linguistics , Online, 2021 , pp. 1336 - 1350 . URL: https://aclanthology.

Ethics Statement org/ 2021 .eacl-main. 114 . doi: 10 .18653/v1/ 2021 .

eacl-main.114.

The dataset was created in accordance with YouTube's [6 ] E. Fersini , D. Nozza , P. Rosso , Overview of the Terms of Service. Considering the large number of users evalita 2018 task on automatic misogyny identificawriting comments collected in the dataset, it was not tion (ami) , in: EVALITA@CLiC-it, 2018 . URL: https: possible to explicitly ask for their consent . No sensitive //api.semanticscholar.org/CorpusID:56483156.

data are provided in the dataset and users' mentions have [7] S. Gemelli , G. Minnema, Manosphrames: explorbeen anonymized to protect their privacy. ing an Italian incel community through the lens All the annotators involved in this research were free of NLP and frame semantics , in: P. Sommerauer, to participate without pressure or obligation. From the T . Caselli, M. Nissim , L. Remijnse , P. Vossen (Eds.), initial stages, they were aware of being free to leave at Proceedings of the First Workshop on Reference, any time without negative consequences. During the Framing, and Perspective @ LREC-COLING 2024, annotation phase, we met several times to make sure that ELRA and ICCL, Torino , Italia, 2024 , pp. 28 - 39 . URL: the topic did not disturb them psychologically or emo- https://aclanthology.org/ 2024 .rfp- 1 .4.

tionally. We informed them to take their time, doing the [8 ] I. Spada , M. Lai , V. Patti , Inters8: A corpus to annotation only when they felt like it and to contact us study misogyny and intersectionality on twitter., for support. This approach continued for all the research in: CLiC-it , 2023 .

stages. [9] P. Lalli , L' amore non uccide. Femminicidio e discorso pubblico: cronaca, tribunali , politiche, Il Acknowledgements Mulino, 2020 .

[10] A. Tontodimamma , E. Nissi , A. Sarra , L. Fontanella , We would like to thank Chiara Zanchi for discussing with Thirty years of research into hate speech: topics us the direction of this work in its early stages. In addi- of interest and their evolution, Scientometrics 126 tion, we would like to thank Sara Gemelli and Andrea ( 2021 ) 157 - 179 .

Marra for their contribution to the creation of the anno- [11] A. Ollagnier , E. Cabrio , S. Villata , Unsupervised tation scheme and guidelines . Also, we reiterate our grat- ifne-grained hate speech target community detecitude to the annotators who professionally worked on a tion and characterisation on social media, Social dificult topic like GBV . This work was also partially sup- Network Analysis and Mining 13 ( 2023 ) 58 .

ported by “HARMONIA” project - M4-C2 , I1 .3 Partenar- [12] F. Poletto , V. Basile , M. Sanguinetti , C. Bosco, iati Estesi - Cascade Call - FAIR - CUP C63C22000770006 - V. Patti , Resources and benchmark corpora PE PE0000013 under the NextGenerationEU programme. for hate speech detection: a systematic review , Lang. Resour. Evaluation 55 ( 2021 ) 477 - 523 . URL: https://doi.org/10.1007/s10579-020-09502-8. doi:10. NLP in italian: Analyzing responsibility framing in 1007/S10579-020-09502-8 . femicide news reports, in: E. Fersini, M. Passarotti , [13] K. W. Crenshaw , Mapping the margins: Inter- V. Patti (Eds.), Proceedings of the Eighth Italian sectionality, identity politics, and violence against Conference on Computational Linguistics, CLiC-it women of color, in: The public nature of private 2021, Milan , Italy, January 26 - 28 , 2022 , volume 3033 violence, Routledge, 2013 , pp. 93 - 118 . of CEUR Workshop Proceedings, CEUR-WS.org, 2021 .

[14] K. Manne , Down Girl: The Logic of Misogyny , URL: https://ceur-ws. org/ Vol- 3033 /paper32.pdf .

Oxford University Press, 2018 . URL: https://books. [24] G. Minnema , S. Gemelli , C. Zanchi , T. Caselli, google.it/books?id= LqPoAQAACAAJ. M. Nissim , Dead or murdered? predicting respon[15] E. W. Pamungkas , A. T. Cignarella , V. Basile , V. Patti, sibility perception in femicide news reports , in: et al., Automatic identification of misogyny in en- Y. He , H. Ji , S. Li , Y. Liu , C.-H. Chang (Eds.), Proglish and italian tweets at evalita 2018 with a multi- ceedings of the 2nd Conference of the Asia-Pacific lingual hate lexicon , in: CEUR Workshop Proceed- Chapter of the Association for Computational Linings , volume 2263 , CEUR-WS , 2018 , pp. 1 - 6 . guistics and the 12th International Joint Confer[16] V. Basile , C. Bosco , E. Fersini , D. Nozza , V. Patti , F. M. ence on Natural Language Processing (Volume 1 : Rangel Pardo , P. Rosso , M. Sanguinetti, SemEval- Long Papers), Association for Computational Lin2019 task 5: Multilingual detection of hate speech guistics , Online only , 2022 , pp. 1078 - 1090 . URL: against immigrants and women in Twitter , in: https://aclanthology.org/ 2022 .aacl-main. 79 .

J. May , E. Shutova , A. Herbelot , X. Zhu , M. Apidi- [25] G. Minnema , H. Lai , B. Muscato , M. Nisanaki , S. M. Mohammad (Eds.), Proceedings of the sim, Responsibility perspective transfer for Ital13th International Workshop on Semantic Evalu- ian femicide news , in: A. Rogers , J. Boydation , Association for Computational Linguistics, Graber, N. Okazaki (Eds.), Findings of the AsMinneapolis, Minnesota , USA, 2019 , pp. 54 - 63 . URL: sociation for Computational Linguistics : ACL https://aclanthology.org/S19-2007. doi: 10 .18653/ 2023, Association for Computational Linguistics, v1/ S19 -2007. Toronto, Canada, 2023 , pp. 7907 - 7918 . URL: https: [17] E. W. Pamungkas , V. Basile , V. Patti , Misogyny de- //aclanthology.org/ 2023 .findings-acl. 501 . doi: 10.

tection in twitter: a multilingual and cross-domain 18653 /v1/ 2023 .findings-acl. 501 .

study , Inf. Process. Manag . 57 ( 2020 ) 102360 . URL: [26] V. Basile , N. Novielli , D. Croce , F. Barbieri , M. Nishttps://doi.org/10.1016/j.ipm. 2020 . 102360 . doi:10. sim , V. Patti, Sentiment polarity classification at 1016/J.IPM. 2020 . 102360 . evalita: Lessons learned and open challenges , IEEE [18] P. Zeinert , N. Inie , L. Derczynski, Annotating on- Transactions on Afective Computing 12 ( 2021 ) 466 - line misogyny, in: Proceedings of the 59th Annual 478 .

Meeting of the Association for Computational Lin- [27] B. Vidgen , L. Derczynski, Directions in abusive languistics and the 11th International Joint Conference guage training data, a systematic review: Garbage on Natural Language Processing (Volume 1: Long in, garbage out , Plos one 15 ( 2020 ) e0243300 .

Papers) , 2021 , pp. 3181 - 3197 . [28] J. Fleiss , Measuring nominal scale agreement [19] F. Formato , Gender, discourse and ideology in Ital- among many raters , Psychological Bulletin 76 ian , Springer, 2019 . ( 1971 ) 378 -. doi:10 .1037/h0031619.

[20] L. Busso , C. R. Combei , O. Tordini , Narrating gender [29] A. Muti , F. Ruggeri , C. Toraman , L. Musetti, violence a corpus-based study on the representa- S. Algherini , S. Ronchi , G. Saretto, C. Zapparoli, tion of gender-based violence in italian media , in: A. Barrón-Cedeño , Pejorativity: Disambiguating Language, Gender and Hate Speech: A Multidisci- pejorative epithets to improve misogyny detection plinary Approach , 2020 . in italian tweets, arXiv preprint arXiv:2404 .02681 [21] N. Mandolini , Femminicidio, prima e dopo. ( 2024 ).

un'analisi qualitativa della copertura giornalistica [30] J. Devlin , M. Chang , K. Lee , K. Toutanova, dei casi stefania noce (2011) e sara di pietrantonio BERT: pre-training of deep bidirectional trans( 2016 ), Problemi dell'informazione 45 ( 2020 ) 247 - formers for language understanding , CoRR 277 . abs/ 1810 .04805 ( 2018 ). URL: http://arxiv.org/abs/ [22] E. Pinelli , C. Zanchi , Gender-Based Violence in 1810 . 04805 . arXiv: 1810 .04805.

Italian Local Newspapers: How Argument Struc- [31] M. Polignano , P. Basile , M. de Gemmis, G. Semerture Constructions Can Diminish a Perpetrator's aro , V. Basile, AlBERTo: Italian BERT Language Responsibility , 2021 , pp. 117 - 143 . doi: 10 .1007/ Understanding Model for NLP Challenging Tasks 978-3-030-70091-1_6. Based on Tweets, in: Proceedings of the Sixth [23] G. Minnema , S. Gemelli , C. Zanchi , V. Patti , Italian Conference on Computational Linguistics T. Caselli, M. Nissim , Frame semantics for social (CLiC-it 2019 ), volume 2481 , CEUR , 2019 . URL: 19 .2% 46 .8% 33%