English. In this study, we aim to enhance hate speech detection in Italian incel posts. We pre-train monolingual (Italian) and multilingual Transformer models on corpora built from two incel forums, one in Italian and one in English, using masked language modeling. Then, we fine-tune the models on combinations of English and Italian corpora, annotated for hate speech. Experiments on a hate speech corpus derived from the Italian incel forum show that the best results are achieved by training multilingual models on bilingual data, rather than training monolingual models on Italian-only data. This emphasizes the importance of using training and testing data from a similar linguistic domain, even when the languages difer. Italiano. In questo studio, ci proponiamo di migliorare il rilevamento dei discorsi d'odio in post tratti da un forum italiano di incel. Addestriamo modelli Transformer mono (italiano) e multilingue su corpora ottenuti da due forum di incel, uno in italiano e uno in inglese, con il masked language modeling. Facciamo quindi il fine-tuning dei modelli su corpora in italiano e inglese con annotazioni indicanti se un post esprime odio. Sperimentando su un corpus annotato per i discorsi di odio ottenuto da un forum italiano di incel mostriamo che i risultati migliori si ottengono addestrando modelli multilingue su combinazioni bilingue di corpora e non con modelli italiani e dati monolingue. Ciò sottolinea l'importanza di utilizzare dati di addestramento appartenenti a un contesto linguistico simile a quello dei dati di valutazione, anche con lingue diferenti.
training on general target-language labelled corpora, de- and ofensive Reddit messages, obtaining a model called spite part of the training data not being in the language of HateBERT, capable of outperforming BERT on hate the downstream task. In addition, the results show that speech identification on various benchmark datasets. this strategy can be used to improve model performance In multilingual settings, Pelicon et al. [21] use a multiwhen in-domain target-language data is scarce, by using lingual combination of corpora annotated for hate speech in-domain data from other languages. to improve the performance of classifiers in zero-shot,
The rest of the paper is organized as follows: Section 2 few-shot and well-resourced settings. Gokhale et al. [22] presents related work on hate speech detection in Italian use MLM training to improve the hate speech detection and English, as well as multilingual approaches to the performance of BERT in Hindi and Marathi, separately. problem. Section 3 describes the corpora used in this We follow such approaches in improving the performance study. Section 4 presents the employed models. Section 5 of our models, with a specific focus on monolingual vs. describes the experiments conducted and discusses the re- bilingual pre-training, compared to Gajo et al. [23]. sults. Section 6 closes the contribution with conclusions and future work.
2. Related Work We leverage three labelled Italian-language corpora from
past EVALITA campaigns, along with two labelled
corPrior work on Italian hate speech detection has been con- pora compiled from two incel forums.
ducted chiefly within the context of EVALITA. The 2018
edition hosted a shared task on hate speech detection [7] EVALITA corpora The first Italian corpus we use was
based on two corpora, one comprising tweets and one compiled for the first edition of the Hate Speech
DetecFacebook posts. The participating teams experimented tion (HaSpeeDe) shared task, from EVALITA 2018 [7]
with a variety of algorithms, with the top team relying (henceforth HSD-FB), by annotating Facebook posts for
on an SVM and a BiLSTM [8]. The 2020 edition hosted a hate speech. The second one
As regards misogyny in particular, EVALITA 2018 split them 70/30 between training and development sets.
hosted for the first time a shared task on automatic misog- We do not use the test partitions, as we are interested
yny identification (AMI), where the top performing teams in maintaining consistency with the use of the original
used a combination of TF-IDF and SVD for the Italian sce- splits of these corpora.
nario, and TF-IDF with logistic regression for the English
one [14]. EVAL
English-language hate speech detection has been con- We keep the training, development and testing parducted in a variety of ways. Among others, Davidson et al. titions as in the released corpora. IFS-IT is used in its [18] build a corpus of tweets annotated with multi-class entirety solely as a test set, due to the unavailability of labels (“hate speech”, “ofensive”, “neither”) and train lo- additional annotated Italian incel data for training. Said gistic regression and linear SVM models on it. Mathew scarcity prompted us to leverage the available data in et al. [19] build a corpus called HateXplain from Twitter order to conduct cross-lingual experiments for the incel and Gab posts, annotated with multi-class labels based domain, for which Italian is a low-resource language. on whether the post is “ofensive”, expresses “hate”, or is “normal”, which they use to fine-tune a BERT hate speech classifier. Caselli et al. [20] retrain BERT on the MLM task using an unlabelled corpus built from hateful 2https://incels.is (Last access: 11 Aug 2023) 3https://ilforumdeibrutti.forumfree.it (Last access: 11 Aug 2023) 4Refer to Gajo et al. [23] for details on the annotation process.
Table 1 shows the class distribution of all three EVALITA corpora, whereas Table 2 shows the distribution for the incel corpora, where posts are considered hateful if they are either labeled as misogynous or racist.
As can be inferred from the statistics, while misogynous instances comprise around 39% of the instances in both IFS-EN and IFS-IT, the same cannot be said for the racist ones, which are much more prevalent in IFS-EN (13% vs. 0.03%). This shows a clear diference in terms of the hate speech produced by the two incel communities.
We approach the task of identifying hate speech as a binary classification problem, where a post can either be hateful or not. We train each model five times on all possible combinations of the corpora listed in Tables 1 and 2 in order to make our results more reliable and 4. Models diminish the efect of the random initialization of the models. In the monolingual Italian setting we never use With relation to the Italian-only scenario, we use IFS-EN, while it is always included when training the UmBERTo and AlBERTo for our baseline models. We multilingual models in the bilingual setting. We select choose these models because they achieved the best per- the number of epochs based on the convergence of the formance in previous EVALITA shared tasks on hate performance on the validation set, in terms of F1-measure speech [9] and misogyny [15] identification. In order on the positive class. For each corpus combination, the to improve the performance of the two models on the training and validation sets are the union of the individual task of identifying hate speech in Italian incel forums, we training and validation sets of each merged corpus. The train them on the MLM task on posts extracted from Il models are then evaluated on the IFS-IT test set. forum dei brutti. We follow this approach because it has been shown to work in English both for general hateful Monolingual setting Table 3 shows the performance content [20] and incel forums [23]. For training data, in terms of precision, recall and F1-measure for the we use the entirety of the contents of the forum, for a Italian-only models and corpora combinations. The toptotal of 627 posts. The intersection between the un- performing model is Incel AlBERTo, which achieves a labelled incel corpora and the annotated corpora listed test F1 of 0.707 when training solely on HSD-FB. Comin Table 2 is void. That is, none of the data contained pared to AlBERTo, this represents an improvement of in IFS-IT was obtained from the Italian data scraped 2.4 points. To a lesser degree, the same can be observed from Il forum dei brutti and used for MLM pre-training. with regard to Incel UmBERTo and UmBERTo (+0.9 F1 The same is true for IFS-EN and the English MLM pre- points), when using the same combination. In both cases, training data taken from Incels.is. Doing this, we obtain this shows that pre-training AlBERTo and UmBERTo two new models which we refer to as “Incel UmBERTo” using MLM on Italian posts extracted from Il forum dei and “Incel AlBERTo”. brutti is efective in improving their performance.
The MLM pre-training process is carried out in all The worst results are obtained when training solely cases by tokenizing post contents using each model’s on HSD-TW, with Incel AlBERTo and Incel UmBERTo own tokenizer and masking tokens with a probability performing worse than UmBERTo and AlBERTo, showof 15%. We use a batch size of 32 samples and train the ing an opposite trend to the one observed when training models for one epoch on one Tesla P100 GPU with 16 GB on HSD-FB. The validation scores are also lower for of VRAM. HSD-TW combinations, compared to combinations in
As regards the bilingual setting, we use mBERT cluding HSD-FB, showing that the models have a harder as our baseline. We also use an MLM-enhanced version time learning from HSD-TW. This is coherent with of it, “Incel mBERT”,5 obtained by further pre-training the results obtained by teams participating in the two mBERT on 500 posts sampled from Il forum dei HaSpeeDe shared tasks [7, 9] and with the fact that HSD-FB’s messages are “longer and more correct than those in Twitter, allowing systems (and humans too) to ifnd more and more clear indications of the presence of improvements obtained by merging diferent corpora. HS” [7]. The fact that messages in HSD-FB are longer is Therefore, while some improvement can be observed by also coherent with the Italian incel models performing merging diferent corpora, MLM appears to be a more better than the vanilla models when training on HSD-FB, efective strategy for improving the performance of the since Il forum dei brutti on average contains rather long models, although it requires greater computational reposts (∼ 53 avg. tokens),6 unlike Twitter corpora, which sources. were limited to 280 characters per tweet prior to 2023.
Finally, another element which might explain the lower Bilingual setting Table 4 reports the results for the performance when training on HSD-TW is that it con- bilingual setting. Compared to the best combination ustains hate speech against migrants, which might not be ing mBERT, which achieves a test F1 of 0.688, the as relevant when it comes to Il forum dei brutti, since best combination using Incel mBERT achieves a test racism is not all that prevalent in this forum, compared F1 of 0.722 (+3.4 F1 points), which is also the highest to misogyny. score across both language settings. Just like in the
As regards combining diferent Italian corpora, the monolingual setting, mBERT performs better when strategy yields the highest performance for AlBERTo and only training it on HSD-FB (in addition to IFS-EN). ConUmBERTo when training on both HSD-FB and AMI-20. versely, Incel mBERT performs better when training on However, once the models are MLM-trained on Il forum AMI-20 and IFS-EN. This is interesting, since the incordei brutti, the performance decreases for some combina- poration of the AMI-20 corpus lowered the performance tions, with MLM pre-training seemingly nullifying the of all Italian-only models, compared to only training on HSD-FB. Since misogyny is the main way hate speech is expressed in Incels.is (39.44% of the instances in IFS-EN 6Obtained with BertTokenizer: https://huggingface.co/docs/tr ansformers/model_doc/bert#transformers.BertTokenizer are misogynous) and Incel mBERT was pre-trained using corpus combinations out of seven, Incel mBERT’s perposts extracted from this forum, the performance boost formance is higher than all other models. The comcould be due to the fact that the model is better at learn- binations for which Incel mBERT does not beat all ing about misogynous language compared to mBERT the others are HSD-FB+HSD-TW, HSD-FB+AMI-20 and and the Italian-only models. HSD-TW+AMI-20.
On average, the lowest performance is achieved when Since mBERT was originally pre-trained in 104 lantraining separately on IFS-EN and on the Italian corpora guages and AlBERTo and UmBERTo were pre-trained (monolingual rows in Table 4). When using bilingual only on Italian corpora, the fact that Incel mBERT can data, the worst results are obtained when training on outperform them by pre-training on just 1 bilingual inHSD-TW and combinations containing it, coherently stances is rather unexpected. Even more interesting is the with the results in the monolingual settings shown in fact that, although we are testing on an entirely Italian Table 3. corpus, Incel mBERT also outperforms Incel AlBERTo
For almost all combinations of Italian corpora, perfor- and Incel UmBERTo. Therefore, in the approached scemance increases once IFS-EN is added to the training narios, using bilingual instances to pre-train a multidata, i.e. bilingual data leads to better performance. lingual model using MLM yields higher performance than pre-training Italian models only on Italian posts.
Monolingual vs. bilingual The results of our experi- Furthermore, the number of Italian posts used to train ments show that the highest performance is not obtained Incel AlBERTo and Incel UmBERTo is 627, which is by fine-tuning on the Italian-only corpus combinations, greater than the 500 Italian posts used for Incel mBERT. but on the bilingual ones. Indeed, for four bilingual As such, we could arguably conclude that the model is learning to spot hate speech more efectively in IFSIT by learning language-agnostic incel concepts, since Incel mBERT is pre-trained on posts extracted from two incel forums in two diferent languages. Although the two considered incel communities are distinct, the hateful Red Pill ideology has spread internationally and is shared by both. This could explain why Incel mBERT performs better than the Italian-only models: the model might be learning about incel hate speech by paying more attention to the sociological concepts underlying the language, and putting less focus on purely linguistic features, ultimately improving its performance.
brutte, le asiatiche lo sono in media - ma quelle belle lo sono davvero e staccano di misura le cosiddette belle nostrane.
le negre e i trans (più questi ultimi eh).
quindi negri e di quei rari bianchi europei che provengono da altre nazioni europee?
queste merde in massa per farci terronizzare e negrizzare come loro. le 8+ sono davvero rare. tuttavia, dal 5 in su si atteggiano tutte come fossero modelle...
Rac.
Mis.
Performance on misogyny/racism subclasses Table 5 reports the performance of the best model —the one obtained by fine-tuning Incel mBERT on IFS-EN ∪ AMI-20— on the individual misogyny and racism labels of the IFS-IT test set. When looking at the diference between the performance on misogyny vs. racism, we notice a stark diference, with racism having perfect precision and a much higher F1. Expectedly, this also translates into the instances that are both misogynous and racist, with perfect precision and recall. The explanation for the racist instances being much easier to detect is twofold: (i) the number of instances which are only racist is much smaller (8 vs. 187) and (ii) compared to the misogyny expressed by Il forum dei brutti users, the racism is much more explicit and simpler to identify. This can be seen in the examples in Table 6, which display explicit language in the first four instances, which contain racism.
Here, the model can easily detect hate, even though users might even attempt to auto-censor themselves by substituting letters with numbers, as in example #1 (most likely in order to bypass automatic forum filters). Conversely, the misogyny in the last two samples is much more implicit, with the model failing to detect misogyny in sample #5. adapting transformer models to the contents of incel forums boosts their performance when predicting the hatefulness of incel forum posts, both when using Italianonly and multilingual models. The increase in performance obtained through MLM pre-training is particularly high when using bilingual training data with mBERT, which might indicate that the model is learning about incel hate speech by learning language-agnostic incel concepts. We have also shown that for the base Italian models (AlBERTo and UmBERTo) fine-tuning on combinations of diferent Italian corpora can lead to a boost in performance. However, this performance boost is nulliifed after MLM pre-training, which appears to be a more efective strategy for improving the performance of the models. When looking at racism vs. misogyny identification in posts extracted from Il forum dei brutti, the former appears to be much easier to detect. This seems due to the fact that racist language is much more explicit than misogynous language in the scrutinized forum, but further research is needed to ascertain such a supposition.
In future work, we plan to experiment with
diferent resources for MLM pre-training, using corpora in
diferent languages, since it seems multilingual models
such as mBERT are capable of learning about hate
6. Conclusions speech in a language-agnostic way from multiple
languages. In addition, with more computational resources,
In this paper, we have presented an approach to improve larger corpora and more training epochs could be used to
the performance of hate speech detection models in Ital- further improve the performance of the models. Lastly,
ian incel posts. Our experiments show that domain- further experiments can be carried out as regards the
performance of the scrutinized models on the individual [12] M. Polignano, P. Basile, M. De Gemmis, G. Semeraro,
sub-tasks of misogyny and racism identification, respec- V. Basile, AlBERTo: Italian BERT Language
Undertively. standing Model for NLP Challenging Tasks Based
on Tweets, in: R. Bernardi, R. Navigli, G. Semeraro
(Eds.), Proceedings of the Sixth Italian Conference
References on Computational Linguistics (CLiC-it 2019),
volume 2481, CEUR-WS, Bari, Italy, 2019.
[1] F. Alkomah, X. Ma, A Literature Review of Textual [13] L. Parisi, S. Francia, P. Magnani, UmBERTo: An
Hate Speech Detection Methods and Datasets, Infor- Italian Language Model Trained with Whole Word
mation 13 (2022). doi:10.3390/info13060273. Masking, https://github.com/musixmatchresearch/
[2] H. Van, Mitigating Data Scarc
guage Models, 2023. doi:10.48550/arXiv.2302. [14] E. Fersini, D. Nozza, P. Rosso, Overview of the
01806. EVALITA 2018 Task on Automatic Misogyny
Iden[3] A. Nagle, Kill All Normies: Online Culture Wars tification (AMI), in: [25], 2018.
from 4chan and Tumblr to Trump and the Alt- [15] E. Fersini, D. Nozza, P. Rosso, AMI @ EVALITA
Right, Zero Books, W
Learning in Deep Neural Networks at EVALITA [19] B. Mathew, P. Saha, S. M. Yimam, C. Biemann,
2018, in: [25], 2018, pp. 86–95. P. Goyal, A. Mukherjee, HateXplain: A
Bench[9] V. Basile, D. M. Maria, C. Danilo, L. C. Passaro, mark Dataset for Explainable Hate Speech
Deet al., EVAL