This paper presents our research on the detection of online misogyny on social media and its intersection with other hate categories. Focusing on the phenomenon of misogyny, we carried out a corpus-based data analysis around victims of online hate campaigns. Targets were selected to study how misogyny and sexism intersect with other categories of social hatred and discrimination such as xenophobia, racism, and Islamophobia. This study includes an event-driven analysis of hate on Twitter concerning specific targets, the process of developing the Inters8 corpus, and its manual annotation according to a novel multi-level scheme designed to assess the presence of intersectional hatred.
perceptions that can greatly vary according to personal experiences and cultural backgrounds.
Several eforts in terms of automatic detection have been provided by scholars for countering hate speech [5, 6]. About misogyny, a first computational efort for the detection of misogyny in English tweets has been provided in [7], while [8] attempts to address the problem of measuring and mitigating unintended bias in machine learning models trained for misogyny detection. A first automatic Misogyny Identification (AMI) shared task has been organized within EVALITA and IberEval 2018 evaluation campaigns [8, 9] to detect misogyny in tweets in various languages (English, Spanish, and Italian). In particular, participants were specifically requested to identify if the message is misogynistic, and then to categorize the target (person or not) and the type of misogyny using the categories developed by [10]. Based on the availability of multilingual datasets targeting misogyny and other kind of abusive language, in [11] a multilingual and cross-domain study on misogyny identification in Twitter is proposed, where some insights on features of misogyny and on the interaction between misogyny and related phenomena are provided. In this work, we try to shed more light on misogyny and intersectionality with other co-existing forms of discrimination such as xenophobia, islamophobia, and stereotype proposing a new annotation scheme. In Section 4.1 we specifically analyze the contributions that inspired our work. ulation forced to stay at home was making substantial use of social media [16]. The analysis process starts with prior knowledge of the Italian context obtained from consulting services that convey information such as newspapers, news broadcasts, and TV shows. It was noticed that some streams of discourse on Twitter were eventdriven, these included: reporting news, inviting guests, and discussing known people on TV shows.
Focusing on misogyny (the explicit and implicit attitude of generic aversion to women), the additional social categories considered are inspired by those in Vox’s Intolerance Map n.73. The categories taken into account in this study are as follows: misogyny, xenophobia, antisemitism, Islamophobia, homolesbobitransphobia, political opinions and physical appearance.
Through scraping news and TV shows available on RaiPlay4 (the Italian television streaming service), we 3. Methodology viewed episodes of 23 TV shows, qualitatively analyzed and selected 17 well-known people in Italy5 who fall into In this section, we describe the methodological pipeline multiple dimensionalities considered in this case study. we designed in order to collect data to analyze intersec- The manual analysis of the language, expressed in TV protional hate and discrimination. The target- and event- grams and Twitter interactions concerning the selected oriented nature of hate speech in social media has been targets, allowed the extraction of information framing the the object of recent studies [12, 13, 11, 14, 15]. Hate subjects: dimensionalities, topics, events, debates, hashdiscourses may vary in relation to events and victims tags, and the most common and narrow keywords. The belonging to multiple dimensionalities. However, it is following were annotated for each victim: Twitter usernecessary to take into account that recognition of the name (if they joined the social network), characteristics discriminatory phenomenon may be partly subjective potentially exposed to hate and discrimination, hashtags, and influenced by the social and cultural context. time period analyzed, TV shows, and links to episodes
In order to contextualize and explore the phenomenon where they were invited or talked about. of intersectionality among multiple social categories sub- In addition, an table6 was compiled to make the jected to hate and discrimination, we conducted an anal- target comparison visually easier, where refers to the ysis of discourses concerning public people, known to Italian society, selected specifically for this task. 3VOX, University of Milan, Sapienza University of Rome, Our pipeline consists of a sequence of steps (Figure 1). Aldo Moro University of Bari and ItsTime (2022). The map of intolerance n.7. Available at: http://www.voxdiritti.it/ 3.1. Discourse analysis regarding targets la-nu4ohvttap-sm:/a/wppwaw-d.realliipnltaoyl.liet/ranza-7/ and events 5Giovanna Botteri, Carola Rachete, Cathy La Torre, Cécile Kyenge, Chiara Appendino, Diletta Leotta, Emma Bonino, Greta The period chosen for analysis is the first half of 2020, the Thunberg, Ilaria Cucchi, Laura Boldrini, Liliana Segre, Michela Muryear when the COVID-192 pandemic began and the pop- gia, Rula Jebreal, Silvia Romano, Teresa Bellanova, Virginia Raggi and Vladimir Luxuria.
2https://www.who.int/health-topics/coronavirus 6https://github.com/ivsnp/inters8/tree/main/targets people and to the hate categories. Since misogyny was the focus of this study, all selected victims had the misogyny column marked. Other categories were marked when present.
Output: selection of related targets, events, and hashtags useful for the next phase.
Data Collection Given the targets and events obtained from the previous phase, Italian tweets regarding targets in the temporal surroundings of the detected events were extracted from TWITA [2]. These tweets contained at least one hashtag among those found during the first phase or both the first and last names of the corresponding target or aliases.
We collected the following metadata for each Twitter interaction: tweetId, date, text, type (tweet, retweet, quote, or reply).
Output: collection of tweets related to targets and events useful for the next phase.
regarding the selected target and event.
Sampling of Data to Annotate The process of creating the Inters8 subset followed these steps: (1) retweets removal, (2) similar tweets removal using cosine similarity by setting the threshold to 0.7, (3) the collection was filtered to include tweets with and without the Italian flag emoji with 50% proportion keeping the same distribution for days and hours. The latter decision was made so that the annotation could be compared according to the presence of the Italian flag emoji, days, and hours. Indeed, the presence of the Italian flag seemed to convey hateful content in the pilot study. The collection related to Silvia Romano and the selected event reached 3006 contents, 1500 were randomly extracted for creating a sample to be annotated Inters8_SRomano.
The subset was cleaned of user mentions and URLs. The metadata used to describe the tweets were as follows: id, parentTweetText (if exists), and tweetText.
Output: Inters8_SRomano annotated according to the proposed annotation schema
in output from the previous phase, we set out to create a 3.3. Annotation process corpus containing tweets related to events concerning targets that were potentially victims of intersectional hatred. The manual annotation process was divided into two This decision aimed to provide a set of Twitter interac- phases: (1) pilot - a sampling of 50 tweets was selected tions (tweets, replies, quotes, and retweets) to perform an to evaluate the annotation scheme and (2) operational analysis of the intersection of various dimensionalities the annotation of the Inters8_SRomano sample dataset. in a target-event context. Our annotation scheme is described in Section 4.
We proceeded with target-event filtering in order to Twelve annotators, balanced by gender, were employed obtain a case study on which to start analyzing the phe- in order to ensure a diversified and representative group nomenon. The Inters8 corpus was populated with the covering a wide range of perspectives and experiences. Twitter interactions collected by selecting the deliver- Guidelines provided to annotators were refined after a ance from captivity and homecoming of the target Silvia discussion within the pilot phase. Romano7 to Italy on May 9-10, 2020. This choice of target- The subset was annotated as follows: we collected 3 event pair was made because there were more Twitter independent annotations for 1373 tweets. The remaining interactions compared with others [17]8. Thus, it was 127 were annotated only by two independent annotators; intended to create a pilot on a specific case study. Given a third annotation was collected in order to solve the this target-event choice, the goal was to explore the in- disagreement. tersectionality between the following dimensionalities: The quantitative analysis conducted by manual annomisogyny, xenophobia, and Islamophobia. tation enabled the assessment of the actual coexistence
Inters8 contains 248240 interactions concerning the of multiple discriminatory and hate dimensions and the chosen target-event pair distributed during May 9-24, construction of a gold standard. 2020. It consists of contents distributed per interaction Annotators features table, annotation scheme, guidetype as follows: retweet 75%, tweets 18%, reply 4%, lines in Italian and English, and annotation results oband quote 2%. The metadata is as follows: tweetId and tained through majority vote are available here. type (tweet, retweet, quote, or reply). Output: subset of Inters8 annotated according to the proposed annotation scheme.
8According to the Italian observatory “Map of Intolerance http: //www.voxdiritti.it/la-nuova-mappa-dellintolleranza-5/ the peak of social attacks against Muslims occurred right around the time of Silvia Romano’s liberation, a media shitstorm that prompted the Special Operations Group (Ros) to open up an investigation ad hoc into the matter.
ones designed in [18, 19]. The scheme used in the AMI@EVALITA18 shared task challenged the participants not only to determine whether misogynous content was expressed in the tweets, but also to classify the misogynistic behavior, by proposing the categories: Stereotype & Objectification , Dominance, Derailing, Sexual Harassment & Threats of Violence, and Discredit. A deeper analysis presented in [20], suggested some insights and motivations to simplify the fine-grained misogynistic behavior to be annotated in our scheme.
The distinction between specific individuals and generic groups of people, also mentioned in [19], was not introduced in our scheme, since the debates around the selected victims turn out to be particularly specific.
A further contribution, annotating an Italian immigration corpus, measured the intensity of hate speech on a scale of 0 to 4 [21]. The idea of measuring hate inspired the comparison of intensities and prevalence among coexisting dimensionalities.
Stance analysis in [22] is performed to check the behavior of tweets in response to others. They used the following labels: agree-accept (support), reject (deny), info-request (question), and opinion (comment). In the case under analysis, it was suficient to consider: support, against, and neutral.
Moreover, as highlighted also in [19, 21], it is important to diferentiate aggressive language from hate speech: in fact, aggressive content is not necessarily expressed through hateful vocabulary and vice versa. Since hatred and discrimination can appear implicitly, it is not always easy and immediate to recognize them in negative and aggressive content on social media. Moreover, not all expressions of disapproval and disagreement with groups imply discrimination. Such findings were useful in order to highlight the importance of annotating both implicit and explicit forms of hate speech.
The multi-level scheme9 was meant to bring out the dimensionalities, and the cohesiveness and prevalence among them. The coarse-grained level is intended to annotate the presence of misogyny, xenophobia, and Islamophobia. The fine-grained analysis, first, aims to recognize which dimensionality prevails over the others. Secondly, a sub-classification of misogyny, if there is any, is proposed to classify it into sexual harassment / derailing and discrediting / dominance. Finally, an annotation
github.com/ivsnp/inters8/blob/main/annotation of stereotyping, victim defense, and stance toward any parent tweet is proposed. A detailed description of the labels involved, supported by examples, follows.
Misogynistic behavior [/]: explicit and implicit forms, including aversion, repulsion, target silencing and instrumentalization of pregnancy, following the definition “misogynistic behavior is about hostility towards women who violate patriarchal norms and expectations, who aren’t serving male interests in the ways they’re expected to. So there’s this sense that women are doing something wrong: that they’re morally objectionable or have a bad attitude or they’re abrasive or shrill or too pushy” [23].
Annotate the two following sub-labels only if = : • Sexual harassment and/or derailing [/]: the first includes avance, requests for sexual favors, and any form of harassment involving sex or speech in which abuse of women is justified by belittling or evading male responsibility. The latter refers to the intention to divert support toward the victim by directing the discourse to a more comfortable alternative issue while ignoring the discriminatory problem.
ITA: La z****la, appassionata ai c**zi talebani, ha orchestrato una messinscena con il tipo che se la s**pa e si è sistemata a vita con il riscatto.
ENG: That s**t, fond of Taliban c***s, orchestrated a setup with the guy who fu**ed her and set herself up for life with the ransom money • Discredit and/or dominance [/]: discrediting occurs when an individual S, through a communicative act, damages the image of another individual T in front of a third party (individual or group A) by referring to actions or characteristics of T that are considered negative by A.
Dominance is typically expressed as an assertion of superiority by highlighting gender inequality.
ITA: Conte dacci le prove del riscatto pagato dagli italiani per questa odiosa nullità e vergogna nazionale! È una bambina indottrinata, senza cervello e stupida. È andata in terre cesso per seguire le sue idiozie apparentemente umanitarie.
ENG: Conte, give us evidence of the ransom paid by the Italians for this odious nothingness and national disgrace! She is an indoctrinated brat, braindead and stupid. She went to toilet-lands to follow her supposedly humanitarian nonsense Xenophobia and/or racism [/]: explicit and implicit forms, i.e., expressions of racism based on the arbitrary assumption of the existence of biologically and historically "superior" human races, aversion to foreigners, and what is foreign. The latter manifests itself in attitudes and actions of intolerance and hostility toward the culture and inhabitants of other countries. In order to analyze ingroup and outgroup dynamics [24, 25, 26], we also consider texts where the target subject and other Italians are insulted or rejected as members of the ingroup, or stigmatized as anti-Italian, because of their proximity to (or support for) foreign populations or immigrants, to be expressions of xenophobia.
ITA: Una donna bianca convertita all’Islam, esce indenne dai ne***ni, belve inferocite, andate tutti a fare in c*lo.
ENG: A white woman converted to Islam, comes out untouched by the nig***s, raging beasts, f**k you all.
Note: This tweet is an example of the intersection of multiple dimensionalities of hatred toward vulnerable groups. The term “italiota” means “Italian idiot” and “ingrassata” refers to the target’s pregnancy.
Stereotypes [/]: negative sexist, xenophobic, racist and Islamophobic stereotypes concerning vulnerable groups targeted by discrimination and hate speech considered in this study on intersectional hatred. Stereotyping is a generalization conducted about a group of people, in which characteristics are attributed to all members of the group [27]. Stereotyping is based on a set of beliefs, not based on experience, that people enact to interpret their surroundings and move through them.
ITA: È venuta qui per fare attentati, è una terrorista, è anche incinta di un musulmano. Se stava bene in Islam rimpatriatela #convertita.
Islamophobia [/]: explicit and implicit forms of Target defense [/]: it indicates whether the user strong aversion, dictated by prejudicial reasons, toward who posted the tweet defends the hate target, contributIslamic culture and religion. The main manifestations in- ing to creating a counter-narrative efect. It includes clude criminalizing targets by describing them as threat- both support without discrimination and support that ening and violent. It is often joined by xenophobia and redirects hatred toward other people (without actually may appear in the form of dehumanization of targets. counteracting hate speech).
ITA: È venuta qui per fare attentati, è una terrorista, è anche incinta di un musulmano. Se stava bene in Islam rimpatriatela #convertita.
ENG: She came here to do bomb attacks, she is a terrorist, she is also pregnant by a Muslim. If she was fine in Islam then send her back #converted Prevalence [/ℎ / /ℎ/]: in case of coexistence of at least two of the main categories to be analyzed, indicate which one prevails over the others within the tweet.
ITA: Il governo ruba 4 milioni di euro agli italioti per pagare uno specie di riscatto al marito islamico che la mette incinta e la converte. Arriva in Italia contenta, ingrassata e viene accolta come una santa. Popolo idiota! ENG: The government steals 4 million euros from the Italians to pay some kind of ransom to her Islamic husband who impregnates her and converts her. She arrives in Italy happy, fat and is welcomed as a Saint.
Idiotic people! ITA: Il privato di Silvia Romano non dovrebbe essere nel dibattito pubblico. È stata liberata da una prigione fisica ma intrappolata in una di violenze psicologiche e pregiudizi inutili e ingiusti.
ENG: Silvia Romano’s private life should not be in the public debate. She was released from a physical prison but trapped in one of psychological violence and unnecessary and unjust prejudice.
Stance [///]: it identiifes the stance of the user who posted a tweet reacting to another one. The label takes the value support if it agrees with the parent tweet, against if it disputes it, and neutral if no stance can be inferred from the text. If there is no parent tweet the default value is absent.
ITA Parent tweet: La liberazione di Silvia Romano è una bella notizia. L’aspettiamo in Italia, ringraziamo i nostri servizi di Intelligence e coloro che hanno contribuito a questo importante obiettivo.
Child, or reply, tweet (contestazione): Assolutamente no! Vanno a fare le splendide in Africa ma quando si accorgono che ci sono i ne***ni cattivi chiedono aiuto a mamma Italia.
ENG: Parent tweet: The liberation of Silvia Romano is good news. We are waiting for her in Italy, we thank our intelligence services and those who contributed to this important achievement.
Child, or reply, tweet (against): Absolutely not! They go show of in Africa, but when they realise there are bad nig***s they ask Mamma Italia for help.
and the harmonization stage by majority vote yielded the results shown below.
Islamophobia is the most annotated dimensionality, followed by misogynistic behavior and xenophobia/racism.
Label annotation detected the following amounts of tweets in the subset (see Figure 2): misogynistic behavior 288 (19.2%), sexual harassment and/or derailing 36 (12.5% of misogynistic behavior), discredit and/or dominance 247 (85.8% of misogynistic behavior), xenophobia and/or racism 153 (10.2%), Islamophobia 317 (21.1%), stereotype 394 (26.3%), target defence 501 (33.4%), and stance [against=119, support=108, absent=42, neutral=24]([40.6%, 36.9%, 14.4%, 8.2%] out of 293 reply tweets).
Stance, on the other hand, appeared dificult to annotate because the stances often went of-topic.
Among tweets labeled with at least one of the three main dimensions of hate included in the proposed annotation scheme, the Italian flag (in name, screen-name, bio or tweet) appears as follows: 78.5% of misogynistic behavior, 73.2% of Xenophobia and/or racism and 74.4% of Islamophobia.
The subset of tweets annotated with an intersection between the three main dimensionalities (at least 2) has cardinality 222. The most present and prevalent dimension was Islamophobia.
The following label distributions were obtained from the annotation of intersectional tweets (see Figures 3 and 4): misogynistic behavior 184 (82.9%), xenophobia and/or racism 117 (52.7%), Islamophobia 199 (89.6%), and prevalence [Islamophobia=91, misogynistic behavior=59, absent=42, xenophobia and/or racism=30]([41%, 26.6%, 18.9%, 13.5%] out of 293 reply tweets).
harassment / derailing 0.29, discredit / dominance 0.44, Recurrent topics Among the tweets annotated, some xenophobia / racism 0.32, Islamophobia 0.53, prevalence recurring topics emerged: (1) dissent on the economic 0.33, stereotyping 0.33, target defense 0.59 and stance plan arguing that payment of the alleged ransom for 0.81. The calculation of Fleiss’ Kappa for tweets with release was not necessary, (2) aesthetic appearance by three independent annotations showed how complicated seeing physical appearance and clothing as objects of it was to classify the domain across the dimensionalities dialogue, (3) being ungrateful, selfish and a traitor to her of the established multi-level scheme. (Silvia Romano) country for converting to the Islamic religion, (4) victimization, (5) pregnancy, and (6) politics. 5.2. Considerations In the former case, it was not always possible to recognize the discriminatory nature since a variable related Misogyny has been labeled more by female annotators, to discontent about the Italian economic situation was only 1/3 of male annotators come close to the former. also present. The second focused on arrival at the airport This is an expected result as the former tend to be more wearing the hijab and a watch pointed to as lavish despite sensitive to the issue confirming that, for creating an the fact that it was not possible to distinguish it from the unbiased annotated dataset, is important to employ an- content disseminated by the media. The third appeared notators belonging to heterogeneous social categories. describing her as a member of the outgroup. The remain
Inter-Annotator Agreement Following an overview ing ones also appeared frequently in the narrative of the of the annotated subset and comparison with the annota- annotated subset. tors involved in this experiment, it appeared that many Counter-speech It has been observed that counterdisagreements occurred on the label stereotypes, some speech, carried out by users who take the defense of vicannotators recognizing many more than others indepen- tims, sometimes proposes an alternative narrative to hate dently from their self-identified gender and age. The speech. Other times they follow defensive strategies that annotations appeared quite subjective and often repeti- are themselves ofensive generating further hate speech. tive because the presence of other discriminatory dimensionality often involves stereotypes. Concerning stance, the presence of irony and rhetorical questions inside the 6. Conclusions and future work dataset complicates the valuation of the attribute.
The complexity of some annotation scheme labels em- Developing the Inters8 corpus built considering an interphasizes the dificulty of annotating tweets about this sectional target-event pair allowed us to explore a case domain and brings out the presence of bias. study and analyze Twitter interactions related to Silvia
In addition, the phenomenon of premediation [29] has Romano on social media. The manual annotation apbeen observed in this case study. Indeed, Twitter’s users plied highlights how multiple dimensionalities coexist expressed their own opinions favoring immediacy and and intertwine in cases of intersectional hate. emotionality in communication as a preliminary reaction Despite the evidence of the phenomenon and its dyto the first information about the news. Tweets and in- namics, results presented here are related to the specific teractions began immediately, despite the fact that the case study taken into account, and to the target and the full picture of the afair was not clear at that moment, event selected. In fact, at the current stage of developbringing the event to the platform’s trending topics. ment, the Inters8 corpus includes content related to the
Target Comparing the Twitter interactions regard- specific intersectional Silvia Romano’s liberation targeting Silvia Romano with those around Luca Tacchetto10, event pair. We plan to expand the corpus with additional Alessandro Sandrini11, and Sergio Zanotti12, the following targets, events, and social categories. It would then be emerged. The three Italian people listed were victims of interesting to compare multiple targets in the same inkidnapping like Silvia Romano. Among them, Tacchetto tersection and study how local culture might influence and Sandrini were converted to Islam. Considering the the phenomenon over several countries. ifrst week after the target subjects’ homecoming, the fol- As the corpus is built around the Italian context, the lowing amounts of interactions were detected: Romano data are exclusively in Italian. The integration of multiple 237031, Tacchetto 1668, Sandrini 546, and Zanotti 2206. languages would allow for greater generalization and the Interestingly, the volume of reactions related to the Silvia study of the geographic distribution of the phenomenon Romano’s liberation is much higher (see [30] for a deeper around known people and events. analysis about this topic), suggesting that the intersec- Finally, there are many interactions on social networks tionality with misogyny matters. and the experimental study for automatic detection of intersectional hate may be a challenge of particular interest.
10https://www.nytimes.com/2020/03/14/world/africa/ mali-hostages-released.html 11https://apnews.com/article/---0acfda0fe2974efab72081965cb7d3c6 12https://apnews.com/article/01e8c9c94f8e4441b7ad68de3d58e667 discriminazione, hate speech e crimini d’odio 1093/oso/9780190604981.001.0001. doi:10.1093/ contro le donne musulmane in Italia, Technical oso/9780190604981.001.0001. Report, Deliverable 2.1, project “TRUST: Tack- [24] G. Comandini, V. Patti, An impossible dialogue! ling Under-Reporting and Under-Recording of nominal utterances and populist rhetoric in an ItalHate Speech and Hate Crimes Against Muslim ian Twitter corpus of hate speech against immiWomen”, co-funded by European Union, Grant grants, in: Proceedings of the Third Workshop on Agreement no. 101049611, 2022. URL: https: Abusive Language Online, Association for Com//www.trust-project-eu.info/trust/wp-content/ putational Linguistics, Florence, Italy, 2019, pp. uploads/2023/04/TRUST-D2.1-ITA-1.pdf. 163–171. URL: https://aclanthology.org/W19-3518. [18] E. Fersini, D. Nozza, P. Rosso, Overview of the doi:10.18653/v1/W19-3518. evalita 2018 task on automatic misogyny identifi- [25] B. Sauer, A. Krasteva, A. Saarinen, Post-democracy, cation (AMI), in: T. Caselli, N. Novielli, V. Patti, party politics and right-wing populist communicaP. Rosso (Eds.), Proceedings of the Sixth Evalua- tion, Routledge, 2018, pp. 14–35. tion Campaign of Natural Language Processing and [26] G. Mazzoleni, R. Bracciale, Socially mediSpeech Tools for Italian. Final Workshop (EVALITA ated populism: the communicative strate2018) co-located with the Fifth Italian Conference gies of political leaders on facebook, Palon Computational Linguistics (CLiC-it 2018), Turin, grave Communications 4 (2018) 1–10. URL: Italy, December 12-13, 2018, volume 2263 of CEUR https://EconPapers.repec.org/RePEc:pal:palcom:v: Workshop Proceedings, CEUR-WS.org, 2018, pp. 1–9. 4:y:2018:i:1:d:10.1057_s41599-018-0104-x.
URL: https://ceur-ws.org/Vol-2263/paper009.pdf. [27] E. Aronson, T. Wilson, R. Akert, Social Psychology, [19] V. Basile, C. Bosco, E. Fersini, D. Nozza, V. Patti, F. M. Always Learning series, Pearson, 2013. URL: https: Rangel Pardo, P. Rosso, M. Sanguinetti, SemEval- //books.google.it/books?id=wr9uvgAACAAJ. 2019 task 5: Multilingual detection of hate speech [28] J. Cohen, A coeficient of agreement for nominal against immigrants and women in Twitter, in: Pro- scales, Educational and Psychological Measurement ceedings of the 13th International Workshop on Se- 20 (1960) 37 – 46. mantic Evaluation, Association for Computational [29] R. A. Grusin, Premediation, Criticism 46 (2004) 17 Linguistics, Minneapolis, Minnesota, USA, 2019, – 39. pp. 54–63. URL: https://aclanthology.org/S19-2007. [30] I. Spada, V. Patti, M. Lai, IntersHate: un corpus doi:10.18653/v1/S19-2007. italiano per lo studio di misoginia e intersezion[20] S. Lazzardi, V. Patti, P. Rosso, Categorizing misog- alità in Twitter, Technical Report, Department of ynistic behaviours in italian, english and spanish Computer Science, University of Turin, Italy, 2020. tweets, Proces. del Leng. Natural 66 (2021) 65–76. URL: https://github.com/ivsnp/inters8/blob/main/ URL: http://journal.sepln.org/sepln/ojs/ojs/index. spada_thesis_IntersHate.pdf.
php/pln/article/view/6323. [21] M. Sanguinetti, F. Poletto, C. Bosco, V. Patti, M. A.
Stranisci, An italian twitter corpus of hate speech A. Online Resources against immigrants, in: Proceedings of the Eleventh International Conference on Language Resources The annotated corpus and the guidelines are available on and Evaluation (LREC 2018), European Language GitHub at the following link: https://github.com/ivsnp/ Resources Association (ELRA), Miyazaki, Japan, inters8. 2018, pp. 2798–2805. URL: https://aclanthology.org/
L18-1443". [22] E. W. Pamungkas, V. Basile, V. Patti, Stance classification for rumour analysis in twitter: Exploiting afective information and conversation structure, in: A. Cuzzocrea, F. Bonchi, D. Gunopulos (Eds.), Proceedings of the CIKM 2018 Workshops co-located with 27th ACM International Conference on Information and Knowledge Management (CIKM 2018), Torino, Italy, October 22, 2018, volume 2482 of CEUR Workshop Proceedings, CEUR-WS.org, 2018, pp. 1–7. URL: https://ceur-ws.org/Vol-2482/ paper37.pdf. [23] K. Manne, Down Girl: The Logic of Misogyny, Oxford University Press, 2017. URL: https://doi.org/10.