Nowadays, the term Gender Based Violence (GBV) is used to identify all forms of abuse based on gender hatred and sexist discrimination [1]. Scholars in social science have defined as "rape culture" the society that normalizes sexist behaviours: from more common occurrences like victim blaming, slut shaming and gender pay gap to the apex of violence with femicide [2]. While general violent crimes decreased over time, GBV did not, alarming various bodies in modern society 1 . A report from the EU commission 2 states that 31%, 5% and 43% of European women suffered respectively from physical, sexual and psychological violence. Regarding the Internet sphere, a survey found that 73% of women journalists experienced online violence (threats, belittling, shaming,...) [3]. These 1 https://www.interno.gov.it/it/stampa-e-comunicazione/ dati-e-statistiche/omicidi-volontari-e-violenza-genere 2 https://commission.europa.eu/strategy-and-policy/ policies/justice-and-fundamental-rights/gender-equality/ gender-based-violence/what-gender-based-violence_en statistics become even more alarming when we consider studies that show the correlation between misogynistic online posts and GBV [4]. Like other countries, Italy is affected by GBV, with the national observatory managed by the "Non Una di Meno" association reporting 117 femicides in 2022, 120 in 2023 and more than 40 until June 2024 3 .

Several studies about Hate Speech (HS) directed towards women often focus on developing taxonomies [5] rather than investigating low resource subjects in computational linguistics like GBV. These works often gather corpora by keyword search of gender slurs [6], retrieving comments left on misogynistic spaces like incel blogs [5,7] or considering messages directed towards popular women figures highly debated on social media [8].

As GBV is a broad topic, we want to clarify that we focus on GBV in Western societies, particularly in Italy. The main goal of this project is to show what is the current perception of femicides expressed through comments on social media, focusing on the specific case of Carol Maltesi. We chose this femicide because the victim was a sex worker, meaning that she presented an intersectional trait, and it was a popular case in the media, enabling us to select enough material for the study. Further, we want to highlight how the socio-demographic characteristics of the victims determine the way they are described and how this influences the perception of the news. For instance, victim's features such as age, job, origin, skin color, nationality, religion have different weight and determine the lesser or greater spread of the news [9]. To overcome the cited issues in current literature, in this research we considered the phenomenon by focusing on users' reactions in social media to news about femicides. We collected YouTube comments in response to videos talking about a specific case. In order to overcome the constraints of traditional sentiment analysis schemas, we annotated the data following a new semantic grid that can be used as a standard for comments regarding GBV.

In the experimental phase of this work, we created models based on different Italian misogyny datasets (including ours). The goal of such experiments is to analyze the different features of these corpora and what forms of misogyny are harder to detect. We performed both a quantitative and qualitative analysis of the results.

In the next sections, we describe: related work on hate speech and misogyny detection(Section 2), the annotation scheme and both a quantitative and qualitative analysis of the dataset (Section 3), and the results obtained in our experiments (Section 4). Lastly, we present some conclusions and delineate possible future developments (Section 5).

In recent times, the creation and dissemination of hate speech are increasingly pervasive on online platforms, making social media a fertile ground for hateful discussions [10]. The escalation of offensive and abusive language, understood as content that discriminates a person or group on the basis of specific characteristics such as ethnicity, gender, sexual orientation, and more has aroused considerable interest in various fields. In fact, over the last decade, a large number of computational methods involving NLP and Machine Learning have been proposed for automatic online hate speech detection [11,12]. Most of prior works have mainly considered hate speech as a classification task, by distinguishing between hate and non-hate speech. Hate speech takes on different nuances depending on the target groups at which it is directed, i.e. depending on the specific features that the target group have in common. Moreover, in some cases, these traits may intersect with each other, leading to different degrees of discrimination. This concept takes the name of intersectionality [13].

Among abusive languages, misogyny, considered as a specific offensive language against women, has become a contemporary research topic [14]. In automatic hate speech detection field, the Automatic Misogyny Identification (AMI) [15] series of shared tasks launched in EVALITA [6] and the SemEval-2019 HatEval challenge [16] have produced evaluation frameworks to identify misogynous tweets in English, Italian and Spanish [17].

Misogyny has become a pervasive phenomenon, widespread in very different spheres and expressed in both explicit and implicit forms [5,18]. For this reason, even in online conversation about a dramatic act such as femicide, it is possible to find examples of veiled or explicit hostility towards the victims. The femicide phenomenon has been studied from different points of view. Several studies focused on GBV representation in Italian media [19,20]. In 2020, Mandolini focused on the journalistic narratives of femicide in newspapers by means of a qualitative discourse analysis on two specific case studies [21]. The researcher attempted to describe changes in attitudes in the portrayal of femicide, focusing on discursive strategies that (directly or indirectly) blame the victim and implicitly excuse the perpetrator, referring to gender stereotypes and romantic love rhetoric.

Other studies focused on the responsibility framing in femicides news, by conducting an experiment where annotators rated excerpts from local newspapers on how much responsibility was given to the perpetrator [22]. As far as we know, there is only one line of work in NLP on GBV [23,24,25], which focuses on reader's perception of femicide news headlines and analyses the perception of responsibility attributed to victim and perpetrator; whereas, to our knowledge, there is no other study analysing social media reactions to GBV cases.

In a preliminary phase of our work, we conducted a research on the femicide case of Sara Di Pietrantonio4 , 22 years old, a white Italian student, from a wealthy family, murdered by her ex boyfriend on May 2016 [21]. In this preliminary research we set out to develop a corpus by collecting Twitter users' comments to femicide news on newspapers published online 5 . We created an annotation scheme for the data corpus consisting of two layers: the first focused on the dimensions of sentiment analysis and composed of three subtasks (subjectivity, polarity and irony), relevant for the detection of sentiment in social media [26]; the second focused on hate speech detection, including labels for misogyny, aggressiveness and its target. For more details on the annotation scheme and corpus description, please read below Appendix A.

Observing the results of the preliminary study, we discovered how the victim's characteristics influence the way newspapers present her femicide and users talk about it on social media. In fact, analyzing Di Pietrantonio's case, as she was a young, white, wealthy and Italian student, we found very few examples of misogyny and, in most cases, the aggressiveness was directed against the perpetrator. Furthermore, the scheme was not considered sufficiently suitable for bringing out important elements of femicide cases. In fact, the annotators expressed their difficulties caused by the scheme developed as it was deficient and too simplistic to recognise complex features of femicide events. In order to solve these issues, we decided to direct our efforts on another case study in which the victim exhibits intersectionality traits, which we assume may lead to more misogynistic content. In addition, we developed new schema and guidelines to have more accurate annotations specifically related to the femicide domain.

In this section we provide a description of the new dataset built and the methodology used.

As mentioned above, we focused our research on the femicide of Carol Maltesi6 , a 26 years old, white Italian woman, mother and online sex worker, who was brutally murdered in January 2022 by her ex partner, Davide Fontana, a 44 years old white Italian bank employee.

With the aim of collecting users' responses to femicide, we chose to collect comments using YouTube Data API, as it is freely available and allows us to easily access comments focused on specific news. The process of obtaining data followed several steps: first, we selected the 31 most popular YouTube videos based on number of views and comments. We chose videos about Maltesi femicide from different types of sources: national (mainly the Italian broadcaster RAI) and local news. The selection of videos is diachronic spanning from March 2022 to June 2023; this was done because the various media channels covered the story as it evolved starting from the discovery of the nameless body and ending with the sentence given to the perpetrator. Afterwards, we collected comments from all the videos selected. Due to the API policy, we were restricted to collect only first-level comments and at most 5 oldest responses to them. In total, we retrieved 3,821 comments.

From the previous experience of the Di Pietrantonio corpus, we decided that a generic sentiment analysis schema proved to be too rigid to understand such a complex phenomenon. We created an annotation scheme and a new online platform to facilitate the raters work. We involved 5 annotators, 4 of them self-identified as women and 1 as a man, all interested in the topic and mostly coming from humanistic background. They were all students and voluntarily participated to the project. The annotation guidelines were decided with the annotators after a pilot study and a subsequent group discussion where the raters pointed out the main faults of the schema. Each annotator analyzed all the comments according to the following guidelines:

• Non classifiable: if the comment cannot be analysed because it is not written in Italian, because it consists only of emojis, because it is not comprehensible or not relevant to the topic (any comment that was marked as NC from at least 1 annotator was removed from the corpus);

The dataset, GBV-Maltesi7 , is composed of 2,934 comments annotated on all categories by all annotators. We As our schema is composed by many different labels, we will focus only on the dimensions that we consider the most relevant, but all statistics can be found in Appendix C. Starting from misogyny, in Appendix C and in Figure 1a, we can see that 9.03% of cases are positive. This unbalance is typical of hate speech datasets [27] and we consider it surprisingly high if we take into account the tragic theme of GBV. It is very interesting that intersectionality represents over 50% of misogynous examples indicating how the personal traits of the victim affect the perception of the users commenting. Unsurprisingly, as the victim was a sex worker, 'work' is almost always the category chosen by the annotators. The target of misogyny was mostly individual, confirming the findings of SemEval-2019 Task 5 [16]. The annotators explained to us how the misogyny target was a difficult category to annotate as often comments used the victim as an example to offend the broader group of women and sex workers.

Aggressiveness is more present than misogyny in our dataset, with 24% positive examples mostly directed towards the perpetrator. Responsibility follows a similar trend with 32.89% positive examples most directed towards the perpetrator. Unlike aggressiveness, we can see a significant amount of comments holding the victim responsible (6.55%).

In Appendix B, we reported the inter-annotator agreement (IAA) scores for all dimensions. As our dataset is fully annotated by multiple people, the metric we chose is Fleiss' Kappa [28]. The metric has a possible range of [-1,1], with 1 indicating perfect agreement, and any value of 𝜅 ≤ 0 indicates more disagreement between the annotators than expected by chance. We can see that most dimensions have a 𝜅 in the [0.2, 0.7] range, indicating variable levels of agreement depending on the label. The dimensions with the highest agreement at 0.69 are empathy towards the event and aggressiveness towards the perpetrator. In fact, annotators explained to us that these two categories were the easiest phenomena to annotate as they lacked ambiguity. On the other hand, we can see that aggressiveness towards the victim is much lower (0.28). In our discussions with the raters, it emerged how attacks towards the victim were harder to identify as they were more subtle leading to disagreement among annotators.

We conducted experiments to validate our resource and to gain more insight into the difficulty of the misogyny detection task. The goal of this analysis is to understand how the presence of different forms of misogyny (implicit and explicit) affect the evaluation of modern classification models. We consider as explicit misogyny discourses that intentionally spread hate towards women mostly through slurs and other aggressive behaviors. Meanwhile, we intend implicit misogyny as more subtle and less conscious practices like victim blaming, slut shaming, de-responsibilization of the perpetrator and more. In addition to our corpus, we used 3 other datasets regarding the topic in Italian: AMI [6], PejorativITy [29] and Inters8 [8]. The former two have been mainly gathered by keyword search of sexist terms 8 , meanwhile, Inters8 and our corpus are focused on more implicit forms of sexist hate directed towards a specific woman (i.e., Silvia Romano and Carol Maltesi). Details about all the datasets can be found in Appendix D.

To explore the potential bias of models towards explicit forms of misogyny, we created 4 different models for binary misogyny detection: BERT-Maltesi, BERT-AMI, BERT-PejorativITy and, BERT-Inters8 that were respectively trained on the GBV-Maltesi, AMI, PejorativITy and 8 AMI is created following an hybrid approach selecting also comments from known misogynistic accounts and responses directed to feminist public figures. We conducted a qualitative analysis and we found that the misogyny contained is almost always explicit and depending on slurs. This lead us to place it in the keyword category. split. We used the existing splits when provided in the papers 9 , else, we randomly created them. All models are binary classifiers created by fine-tuning BERT [30], in particular we used the Italian version AlBERTo [31]. Due to the imbalanced nature of most corpora, the models were trained with a focal loss [32] setting the hyperparameter 𝛾 = 2. Models were trained for 5 epochs but, to avoid overfitting, we implemented an early stopping function which ends training after 2 epochs that report an increase in validation loss. We tested all models on their own test set and the other 3 corpora.

We want to underline that our goal is not to compare performance of the different models between each other as they have different number of training sets and positive examples. Rather, we intend to focus on how different test sets are more difficult compared to others which helps us understand what the current challenges in misogyny detection are.

In Table 1, we reported the positive label and the macro average f1-scores of all experiments. In addition, we also calculated the average scores for each test set. The best scores achieved on a certain test set are in bold, meanwhile, we underlined the best scores for cross-dataset testing. As expected, we can observe that all models had the highest score for their own set. Meanwhile, for crossdataset testing, we can see that the models that tend to perform the best are BERT-PejorativITy and BERT-AMI. We suspect that this is caused by the dataset composition as their training sets present more positive examples compared to the others.

Interestingly, we can observe that certain models recorded higher scores on other test sets that were not their own. This mostly happens when focusing on BERT-Maltesi and BERT-Inters8, which record higher scores on AMI and PejorativITy. Even PejorativITy increseases its scores when tested on AMI. Observing the average scores for each test, we can see that Maltesi and Inters8 are the 9 PejorativITy provides a training and test split, but analyzing the code we found that the test set was used as a validation set so we decided to create a new one. most challenging datasets. This is especially true when observing the average f1-score on the positive label with the score being in the [0.2, 0.3] range, compared to much higher scores for PejorativITy and especially AMI. These trends indicate how misogyny detection is a much harder task when considering datasets that contain less explicit forms of hate (e.g., not gathered by keyword search of sexist slurs).

In addition, we conducted a qualitative analysis on the errors of the various classifiers. We found that for each test set most classifiers misclassified the same type of examples. Models almost never recognized texts which contained victim blaming and slut shaming in the GBV-Maltesi Dataset. The errors made on Inters8 mostly coincide with examples that are also racist and Islamophobic. The cases which proved to be more difficult in Pejora-tivITy and AMI contain less explicit animal epithets like "cavalla" and nouns that refers to sex worker in a less explicit way like "cortigiana".

In this paper, we presented GBV-Maltesi which is the first dataset regarding social reactions to GBV, in particular to a femicide case. The topic was chosen to shed light on the importance of having misogyny corpora that include forms of sexism that are more implicit and complicated to detect compared to the existing ones that focus on slurs and offensive terms. We also focused on the intersectionality aspects to better explore online hate. GBV-Maltesi is composed of 2,934 comments all annotated by 5 annotators and it is available at https://github.com/madeddumarco/GBV-Maltesi. In order to overcome limitations of generic semantic schema, the corpus has been annotated following a new schema specifically created for cases of GBV. In the experimental phase of our work, we created different misogyny binary classifiers and tested them in a cross-dataset way. We found that datasets gathered on keyword collection are easier benchmarks as the model showed bias towards slurs and not identifying more implicit cases of misogyny. This research on online discourse about GBV is not meant to be exhaustive, as several questions are still open.

As future works, we intend to focus on how different framing of news can cause different online reactions, analyzing the differences between video transcripts of femicide news and the comments collected, in terms of words used, implicit references, attributions of guilt and descriptions of the people involved in the story. We also intend to gather more annotated corpora regarding femicides to explore how other characteristics of the victim (e.g., origin or skin color) and time of the murder differently influence the online reactions. In this regard, we intend to explore the question by investigating whether and how the discourse on misogyny changes depending on whether it is addressed to living or dead women (i.e., Giulia Cecchettin femicide and abusive discourse against her sister, Elena Cecchettin). Lastly, we would like to extend our research by following an intersectional approach, considering all the dimensions and characteristics that make up the identity of both victim and perpetrator. To conclude, we strongly advocate the importance of write the news correctly, as this has deep consequences on the readers' perception and the way they talk about it.