The phenomenon of online dangerous speech is a growing challenge and various organisations try to prevent its spread answering promptly to hateful messages online. In this context, we propose a new dataset of activists' and users' comments on Facebook reacting to specific news headlines: AmnestyCounterHS. Taking into account the literature on counterspeech, we defined a new schema of annotation and applied it to our dataset, in order to examine the most used counter-narrative strategies in Italy. This research aims to support the future development of automatic counterspeech generation. This paper presents also a comparative analysis of our dataset with other two datasets in Italian (Counter-TWIT and multilingual CONAN) containing dangerous speech and counter narratives. Through this analysis, we will understand how the environment (artificial vs. ecological) and the topics of discussions online influence the nature of counter narratives. Our findings highlight the predominance of negative sentiment and emotions, the varying presence of stereotypes, and the strategic diferences in counter narratives across datasets.
a very important role. On DS detection, the literature
is vast [4, 5] and covers various nuances of DS [
CLiC-it 2024 – Tenth Italian Conference on Computational Linguistics, Among them, currently, only two datasets contain Italian 4 – 6 December 2024, Pisa, Italy texts: CONAN [13] and Counter-TWIT [14]. * Corresponding author. The creation environment of CONAN is artificial (i.e., $ vittoria.tonini@edu.unito.it (V. Tonini); activists have been asked to write CN to specific hatesmi marocnoa.s.tfrraennidsac@i@aaeeqquuaa--tetecchh.c.coomm(S(M.F.rAe.nSdtar)a;nisci); ful comments) and the one of Counter-TWIT is entirely viviana.patti@unito.it (V. Patti) ecological (i.e., collection of tweets written by users). In 0000-0002-6215-3374 (S. Frenda); 0000-0001-9337-7250 this scenario, in our work we propose a new dataset, (M. A. Stranisci); 0000-0001-5991-370X (V. Patti) AmnestyCounterHS, that diferently from the existing ©At2tr0i2b4utCioonpy4r.0igIhnttefornratthioisnpaalp(CerCbByYit4s.0a)u.thors. Use permitted under Creative Commons License ones, reflects the real action of activists online. Indeed, 1phattgpe:/v/wiswitewd.voonxjduirlyitt2i0.i2t/4la)-nuova-mappa-dellintolleranza-7/ (web- our dataset, compiled from Facebook, includes interac2https://www.amnesty.it/entra-in-azione/task-force-attivismo/ tions guided by the Amnesty Task Force on Hate Speech (webpage visited on july 2024) (HS), representing an ecological and spontaneous con3As reported in Bonaldi et al. [2], the terms ‘counterspeech’ and text. Here, the intervention of counterspeech is guided ‘counter narratives’ are used interchangeably in Natural Language by Amnesty International activists who decided to intertPirvoecaecsstiionngsfiealdim(NedLPa)t, raenfudtbinogthhcaatne sbpeeceocnhstihderoreudghasth“cooumghmtufunlicaan-d vene under certain posts potentially dangerous spread cogent reasons, and true and fact-bound arguments” [3]. by online newspapers or users (e.g., verbal attacks to women, immigrants, and so on). Counter-TWIT7 dataset is made up of 624 pairs of
Moreover, inspired by existing strategy taxonomies tweets and their replies. Data were collected in an eco[15, 13, 14], we mapped a more complete taxonomy in- logical environment using keywords to take texts from clusive of both existing and new strategies found in our profiles of activists, organisations, or pages especially dedataset. This new resource allows us to analyze the used voted to calling out common instances of discrimination. strategies of CN in the Italian language across diferent In this data we encounter both DS(16) and CN(81), but types of messages and contexts ( CONAN, Counter-TWIT, they are not DS/CN pairs such as in CONAN, but rather AmnestyCounterHS). By comparing these datasets, we consist of tweets and their replies. propose to examine: 1) which strategy of CN is the most 2) Tweet: "In Italia spesso funziona cosi: La vittima diventa used in the diferent contexts and discussions online; 2) automaticamente il colpevole."8 which the diferences are in terms of sentiments, emo- Reply: "Nelle violenze in particolare"9 tions, and the presence of stereotypes, between poten- AmnestyCounterHS is a collection of posts and relative tially dangerous messages posted online and the coun- comments gathered from Facebook. The data collection terspeech produced by activists/users in all the datasets. strategy was driven by the work of the Amnesty Task
The importance of understanding how these strate- Force on HS, a group of activists that produce CN against gies of CN are used relies on the need to raise social discriminatory contents spread by online newspapers and awareness about real events, the necessity to be correctly users. During the task force, the activists identified some informed about facts (avoiding fake news), as well as to posts containing news headlines that probably convey be conscious of the consequences of dangerous speech or incite hate speech and assigned them a topic based on in the target groups [16]. the specific target of the news headline. Among the various topics covered in the dataset are: women, migrants, 2. Datasets LGBTQIA+, solidarity, and environmental issues. During their activities they built a database of hateful contents In this section, we describe existing dataset of CN in against which they got activate between 2020 and 2023. Italian (CONAN and Counter-TWIT), and the creation of Starting from this database, we collected all the news AmnestyCounterHS. headlines detected by activists in the March 2020, 2021, CONAN4 is a multilingual and expert-based dataset of 2022, and 2023. Then we gathered and anonymized all DS/CN pairs in English, French and Italian, focused on the comments in reply to them, for a total of 39,582 users’ Islamophobia. The original dataset consists of 4078 pairs comments and 2,010 activists’ comments. For our work, over the 3 languages. The dataset has been augmented we used only 10,670 users’ comments selected from users through translation (from Italian/French to English) and who replied at least 5 times. This approach allowed us paraphrasing, which brought the total number of pairs to to focus on users with more interactions. Table 1 reports 14.988. The dataset was created by Chung et al. [13] in an the information of all corpora. This enabled us to obtain artificial environment and consists of expert-based data. three collections of text: i. a set of news headlines that The DS/CN pairs were collected through niche sourcing incite the use of dangerous speech; ii. a set of comments from three diferent NGOs in the United Kingdom, France, written by activists replying to users or written directly and Italy. Consequently, both the responses and the dan- under post; iii. a set of comments written by users replygerous speech content are expert-based, composed by ing to activists or other users, or written directly under operators specifically trained to counteract online dan- posts. Table 2 shows the number of comments written gerous speech. For this paper we considered only the by users and activists per type of interaction. Italian pairs, which are 3,213 in total. Here is an example 3) Headline: "Migranti, riprendono gli sbarchi. E il coronavirus of a pair from the CONAN dataset: ora avanza in Africa"10 1) DS: "Noi li ospitiamo nel nostro paese, forniamo un aiuto Comment: "salve, legga l’articolo per favore, non sono ripresi economico e loro ci uccidono: sono da considerarsi più gli sbarchi, in realtà stanno diminuendo costantesimili agli animali che alle persone."5 mente, non si preoccupi....è "il Giornale" che fa gli CN: "I criminali sono in tutti i popoli e di tutte le religioni, per scherzoni"11 fortuna una minoranza, non si deve mai generalizzare.
Lei è italiano quindi mafioso?" 6 4https://github.com/marcoguerini/CONAN 5"We host them in out country, provide them an economic aid, and they kill us: they should be considered more like animals than humans." 6"Criminals exist among all people and religions, fortunately as a minority, one should never generalise. You are italian, so are you a mafioso?"
Readme.md 8"In Italy it often works like this: the victim becomes guilty." 9"Particularly in cases of violence" 10"Migrants, the landings resume. Coronavirus is now spreading in
Africa" 11"Hi, please read the article, landings did not resumed, in fact they are decreasing, don’t worry ... "il Giornale" is playing tricks" Dataset CONAN [13] Counter-TWIT [14] AmnestyCounterHS
informal style[
Looking at the comments, we noticed that some of them are ofensive and impolite but not dangerous towards certain categories. They reflect the intensity of discussions on specific topics, displaying hostility towards the interlocutor rather than targeting specific categories. For instance:
mative that is a comment with a statement that seeks to debunk or fact-check the claims made by the attacker, Alternative when alternatives to the statement made by the attacker are proposed, Suggestion, Explicitation in the case of a comment that explicitly clarifies something that was implicit in the DS comment, Question made to cause reflections in the writer of the DS comment, Denouncing and explaining when the writer explains why things said by the perpetrator are not acceptable, Positive in the case of a polite comment, Hostile when the writer uses aggressive tone and words, 12The guidelines and the dataset have been released in https://github.
com/aequa-tech/external-resources. 13You can see some examples of the various annotation layers in Table 7 in Appendix A. 4) Comment: "come scusa, forse non è consapevole di essere lei stessa non saper utilizzare la punteggiatura, continui pure fare figure di merda, i commenti sono pubblici"16 14"Read the article, please" 15"Go consult the historical documents about them and verify what they have done" 16"Excuse me, perhaps you are not aware that you yourself do not know how to use punctuation, keep making an ass of yourself, comments are public"
menti è un insulto verso noi stessi"17
Another interesting observation regards the presence of negative stereotypes that in various cases have been identified as implicit dangerous speech:
che faccia la ballerina"18 7) Comment: "Non chiede di sbarcare...ordina di sbarcare il che è diverso. Loro decidono dove sbarcare e quando sbarcare altrimenti speronano"19
biases are embedded in the discourse, often contributing to the perpetuation of harmful stereotypes. This is one of the reasons why we decided to do an analysis of stereotypes in our comparative analysis.
Finally, we noticed that various comments are featured
with irony. Irony is frequently used to convey dangerous
or ofensive sentiments in a less direct manner [
disgraziato"21 Stereotype: to understand if not only user comments Annotation and inter-annotator agreement The anno- contained stereotypes but also if activists or non-activists tation has been carried out for 307 comments by two an- who wrote CN somehow contributed to spreading them. notators with linguistics background using the LabelStu- Strategies: to identify the most used strategies in CN dio platform (Figure 2 in Appendix C). The Cohen’s kappa depending on the context and topic of discussion online. was computed to examine the inter-annotator agreement for all labels obtaining the results shown in Table 4. The 3.1. Afective Analysis highest results were obtained for the counter-narrative The afective analysis (Figure 1) has been performed au(0.66) and dangerous speech (0.62) labels. For counter- tomatically, detecting sentiment (positive, negative and narrative strategies, the easiest to identify was Ques- neutral) and emotions (joy, sadness, fear, and anger) tion, followed by Positive, and Informative. There inferring labels from the following fine-tuned models were some dificulties related to the Support label. For available on the HuggingFace hub: lxyuan/distilbertinstance, the sentence: "nessun problema, si boicotta la base-multilingual-cased-sentiments-student for sentiDisney."22 was annotated as dangerous speech support by ment, and Taraassss/sentiment_analysis_IT for emotion. one annotator, while the other one did not consider it as In order to compare sentiment and emotions identified in such. It would be helpful to provide further information potential dangerous speech and CN, we selected: 3,213 about this label in the annotation scheme. DS and 3,213 CN from CONAN; 543 tweets and 81 replies annotated as CN from Counter-TWIT; 10,670 users’ com3. Comparative Analysis ments and 2,010 activists’ comments from AmnestyCounterHS23.
In order to investigate the diferences in terms of sen- As can be clearly seen from the sentiment analysis timents, emotions, and the presence of stereotypes, be- graphs, both in the message datasets and in the counter tween potentially dangerous messages posted online and narrative datasets, there is a predominance of negative the counterspeech produced by activists/users in all the polarity. Regarding emotions, anger is the most prevadatasets, we performed three diferent types of analysis. lent emotion. Therefore, we observed this notable trend, despite the diferent origins of the datasets. However, it is important to point out that anger is not always a purely negative sentiment. While it often reflects strong emotions associated with dissatisfaction or conflict, it 17"Nowadays, I just limit myself to laughing, answering certain
comments is an insult to ourselves" 18"a son who is a motorcycle champion or a daughter who is a dancer" 19"They don’t ask to land...They order to land, which is diferent.
They decide where and when to land, otherwise they ram" 20"Maghrebian assaults and robs 63-year-old outside the supermar- 23The assumption that these texts from activists are counterket" narratives is based on the way the data was collected (activist21"Now let’s also give this miserable a medal" comment): the data collection strategy was influenced by the 22"No problem, we’ll boycott Disney." methodology established by the Amnesty Task Force on HS. (a) Sentiment distribution in messages (b) Sentiment distribution in CN (c) Emotion distribution in messages (d) Emotion distribution in CN 3.2. Analysis of Stereotype Like in previous analysis, the presence of stereotypes (see Table 5) has been performed automatically, inferring can also highlight important debates and drive positive labels from the fine-tuned model aequa-tech/stereotypechange, such as in the following example: "un po’ di ver- it available on the HuggingFace hub. The set of examined gogna per un commento fuori luogo come il suo davanti data is the same of afective analysis. a tanto dolore, no?"24. The comment, despite containing Dataset Type of text % stereotype a provocation, aims to be constructive because it tries to CONAN DS 85.6% spark a reaction in the user’s thinking. In many cases, CONAN CN 47.5% anger can be a powerful force for tackling issues and Counter-TWIT Tweet 12.2% making progress. So, the anger seen in these datasets AmCnoeustnytCero-TuWnteITrHS Users’RCeopmlyments 2197..66%% might not just show the seriousness of the issue but also AmnestyCounterHS Activists’ Comments 20.4% the possibility for meaningful discussion and action.
For AmnestyCounterHS, we also wanted to carry out Table 5 a sentiment analysis by dividing the comments based Percentage of presence of stereotypes. on the year of publication to see if the sentiment of the users who wrote various comments, and thus interacted more with the activists, changed over time. We expected their behaviour could become more positive after several interactions with activists. Unfortunately, we did not observe significant changes over the years, as can be seen in the figures provided in Appendix D).
In Table 5, we can see that in the CONAN dataset, dangerous speech messages may be more likely to contain stereotypes, while responses often serve oppositions to stereotypes present in the original messages. This pattern is not the same in Counter-TWIT and AmnestyCounterHS. Indeed, these two datasets, containing data extracted from ecological environments (respectively, Twitter and Facebook), reflect the spontaneous interaction between users and activists, where the activists themselves can explicitly mention stereotypes to oppose them or may be contributing to the creation or amplification of stereotypes. 24"a little shame for a comment out of place like yours in the face of so much pain, no?"
Dataset CONAN CounterTWIT Amnesty CounterHS
Denouncing Informative Alternative Suggestion Explicitation Question and ex
plaining 48.3% - - - 16.1% 22.7% - 6.3% - 8.4% - 34.8% 6.7% 4.3% 4.4% 11.2% 19.8%
Positive
Hostile
Humour 3.3. Analysis of CN Strategies
The third type of analysis focuses on the various types of In this paper, we examine the strategy of CN used in varicounter narrative strategies used across all three datasets. ous contexts, looking at their characteristics and typology Firstly, we had to map the strategy types to our guidelines, across diferent datasets in Italian: CONAN, Counteradapting the strategy labels from the diferent datasets TWIT, and AmnestyCounterHS. Thanks to this comparto match the labels in our dataset (see Table 3). Secondly, ative analysis, we noticed that diferent environments we examined the distribution of strategies across datasets and topics afect the type of strategy used by activists or considering the type of environment (ecological, artifi- users who want to counter DS [18]. cial) and the diferent topics. One of the main points that we want to underline is
In an artificial context such as that of the CONAN the importance of the conversational context [19, 20, 21, dataset, the most commonly used strategy is informa- 22]. In our dataset, AmnestyCounterHS, the annotators tive. This prevalence is expected because, in controlled showed dificulties to understand the position of the auenvironments, there is often a focus on providing factual thor of the message, without the entire conversational information and raising awareness to counteract misin- thread. For instance, let us consider this comment writformation efectively. This is also the most used strategy ten under some news about COVID-19: "Infatti. Ampiain our dataset, where CN were written by activists. In an mente dimostrato"26. Without the full conversation, it is ecological context like that of the Counter-TWIT dataset, challenging to determine whether this comment is supthe most frequently used category is hostile. This is un- porting or contradicting an argument about COVID-19. derstandable, as real-world interactions often involve Similarly, let us take a look at the comment: "Grande argomore emotional and aggressive responses, reflecting mentazione, scuola di Demostene? #posailfiasco" 27 written the more spontaneous and less regulated nature of on- under this newstitle: "Un milione di profughi sono ostagline discourse. The use of this CN strategy is interesting, gio di Erdogan"28. We can clearly see that the comment is because usually it is not suggested to use it. Despite this, ironic, but we cannot understand its stance on integrait can happen that ones get irritated when facing dan- tion. For this reason, future developments in automatic gerous speech. The hostile strategy can be considered counterspeech generation should focus on incorporating somewhat the opposite of positive, which instead repre- comprehensive conversational threads to enhance accusents a very polite attitude. Moreover, we wanted to see racy and relevance. This approach will be fundamental also which the most used strategies were according to to create efective AI-driven counter-narrative systems. the topic. Analysing our dataset we obtained that for the topics LGBTI, migrants and solidarity, the most frequent strategy was informative. For the topic "women", the 5. Ethical Statement and most used strategy was alternative, while for the topic Limitation "environment", the prevalent strategy was denouncing and explaining. The data in the corpus was collected from public pages
We also conducted a manual analysis of the corpus to and has been anonymised. IDs were created by us, and understand if there were any interactions between users the links from which the comments were taken have been and activists that proved more efective than others. In removed, therefore it is not possible to trace the origiparticular, we observed that an activist who employed the nal comments. Moreover, in the released version, the Polite strategy in some comments managed to engage identities of the annotators are not revealed. An ethical quite well with a user. An example of a comment written concern is related to the characteristics of the annotators by the activist is: "interessante. Mi permetta, senza polemica, di puntualizzare alcune inesattezze che ha riportato, forse nella velocità"25 a few inaccuracies you mentioned, perhaps due to haste." 26"Indeed. It’s been extensively demonstrated" 27"Great argument, is it from the school of Demosthenes? #giveitup" 25"Interesting. Allow me, without being argumentative, to point out 28"One million of refuges are hostage to Erdogan"
Amnesty Task Force on HS for supporting us in the collection and creation of the AmnestyCounterHS dataset. participating in data annotation. Data were annotated by two young Italian females with a background in linguistics. The limited diversity among annotators may narrow the variety of perspectives included, and their personal biases could influence the data annotation process. Layers Formal style Informal style Dangerous speech support Counterspeech support Explicit dangerous speech Implicit dangerous speech Informative strategy of CN
Suggestion strategy of CN
Examples Comment: "salve, comprendo la sua polemica, ma non sono arrivati qui per "essere un peso", sono migranti, chi arriva dalla Libia, chi dalla Nigeria, [...]"29 Comment: "stai tergiversando, situazioni diverse, qui si parla di omosessuali, completamente diverso dai giochi con talco e tutto il resto che hai citato. Ognuno però può fare quello che vuole non sono problemi miei. Ciao buona giornata"30 Comment: "avrà tanti morti sulla coscienza, oltre ai nostri anche i migranti, dovete chiudere i porti"31 News title: "Disney, la carica dei 101 generi: "Entro il 2022 la metà dei personaggi sarà Lgbt"32 Comment: "idealmente potrebbe essere vero che per una piena inclusione non ci dovrebbe essere bisogno di dare etichette, ma ognuno dovrebbe essere libero di essere chi è e amare chi vuole liberamente. Ma conviene con me che nelle società di [...]"33 News title: "Il Giornale Pescara, magrebino aggredisce e deruba 63enne fuori dal supermercato"34 Comment: "Adesso vediamo di dargli anche la medaglia sto disgraziato"35 Comment: "Il suo desiderio da padre era quello di avere un figlio che giocasse rugby, come tanti che sperano di aver un figlio che sia campione di moto o una figlia che faccia la ballerina."36 Comment: "guardi che gli unici due sbarchi di Marzo sono stati subito controllati e messi in quarantena preventiva, non ci sono stati altri sbarchi tutto il mese, c’è eccome lo spazio per gestire questi pochi arrivati. Prima di accusare il prossimo[...]"37 Comment: "Mi perdoni, ma anziché ironizzare sugli altri o sentirsi addirittura più accorti degli altri, perché non cercare di argomentare il proprio pensiero? [...]"38 a more detailed description of facts. Example: "Non gigante buono, ma femminicida"40
attacker to encourage them to rethink their views. Example: "Le consiglio di leggere degli articoli sull’argomento"41 4. Explicitation: the writer explicitates/reveals what was implicit in the statement made by the attacker. Example: “Stanno equiparando la pedofilia all’omosessualità”42 5. Question: questions that would challenge the speaker’s chain of reasoning and compel them to either answer convincingly or recant their original remark. Example: “Si potrebbe almeno riportare qualche fatto prima di trarre queste conclusioni?”43 Indirect questions should be annotated too. Example: “mi dia qualche link che riporti esempi concreti di quanto aferma” 44 6. Denouncing and explaining: when you convey the impression that the opinions put forth by the hate speaker are not acceptable and you try to explain to the user why. Example: “C’è un grosso errore di fondo in quanto scritto nell’introduzione 40"Not a good giant, but a femicide" 41"I suggest you to read some papers on the topic" 42"They are equating pedophilia with homosexuality" 43"Could you at least present some facts before drawing these conclusions?" 44"Please provide some links that present concrete examples of what you’re claiming" di questo articolo. Rendere l’interruzione di gravidanza un diritto garantito dall’assistenza sanitaria pubblica non significa che lo Stato imponga alcunché.”45
Example: “Insegnare ai bambini che ci sono tanti modi diferenti per essere felici e che i loro sentimenti valgono è una cosa su cui concordo totalmente.”46 8. Hostile: the user expresses hostility, aggressiveness towards the initial content, using insults or aggressive words. Example: “Bisogna davvero essere degli stupidi idioti retrogradi a credere alla negatività sull’Islam.”47
humoristic, ironic, sarcastic intent whether positive or negative. Example: "E meno male che era buono. Se era cattivo che faceva, se la magnava?"48 It is possible to identify more than a single counterspeech strategy in a single comment. 45"There’s a big mistake in what’s written in the introduction of this article. Making abortion a right guaranteed by public healthcare does not mean that the state is imposing anything." 46"Teaching children that there are many diferent ways to be happy and that their feelings matter is something I completely agree with." 47"One must truly be a stupid, backward idiot to believe the negativity about Islam." 48"Good thing he was nice. If he had been bad, what would he have done, eat her?"
(a) Sentiment distribution of users replying to activists. (b) Emotion distribution of users replying to activists. (c) Sentiment distribution of users replying to users. (d) Emotion distribution of users replying to users. (e) Sentiment distribution of users replying to posts. (f) Emotion distribution of users replying to posts. (a) Sentiment distribution of activists replying to users. (b) Emotion distribution of activists replying to users. (c) Sentiment distribution of activists replying to posts.