In this paper, we present our submission to the Profiling Haters on Twitter shared task at PAN@CLEF2021. The task aims at analyzing Twitter feeds of users in two languages, English and Spanish, in order to determine whether these users spread hate speech on social media. For English, we propose an approach which exploits contextualized word embeddings and a statistical feature extraction method, in order to find words which are used in diferent contexts by haters and non-haters, and we use these words as features to train a classifier. For Spanish, on the other hand, we take advantage of BERT sequence representations, using the average of the sequence representations of all tweets from a user as a feature to train a model for classifying users into haters and non-haters.
The rise of social media in the past decade has undoubtedly changed interaction among people and made the world more inter-connected. It has provided a way for people to keep constantly in contact even when being far apart geographically, united people who have not seen each other for years or who had never met before, helped numerous volunteer associations to gather aid or recruit more volunteers, provided a place for entire communities with common interests to interact with one another and share content, resources and ideas, and the list of benefits continues relentlessly. However, on the flip side of the coin, the growing amounts of usergenerated content online are tied to an increased presence of hateful content on social media. Content moderation online is therefore important to identify and limit the spread of hate speech.
The Profiling Haters on Twitter task [
In this paper, we present our submission to the Profiling Haters on Twitter shared task, which consists of two diferent approaches. First, we propose a novel approach to hate speech detection for the English data set, which derives from the assumption that certain words are used in diferent contexts by haters as opposed to non-haters. The idea is to exploit statistical feature selection techniques in order to find words whose embedding vectors extracted from BERT difer the most between classes, then use these words as features to train a classifier. The Spanish model, on the other hand, is inspired by text classification models, as it allows us to tackle the challenge of having a single representation for long sequences. Therefore, we build the features as though all tweets of a given user were a unique text, without losing information from any tweet. In order to do this, we use a Spanish pre-trained version of BERT for extracting a single vector representation of each tweet by a user. These representations are then averaged and fed into a classifier.
The present work follows the definition of hate speech as described in the overview of this
edition’s shared task [
Most studies carried out on hate speech within natural language processing (NLP) so far have
focused on the detection of hate speech in single messages. The singularity of this shared task
lies in the fact that, instead, it focuses on the quite novel approach of classifying users who
disseminate hateful messages (haters) and users who do no spread any type of hateful messages
(non-haters) on Twitter. To the best of our knowledge, a similar task was proposed only once
[
In the past years, many approaches have been proposed for the detection of hate speech in
single messages extracted from various social media channels, such as Twitter [
As in many other NLP and, more in general, supervised learning methods, feature selection is
one of the most crucial parts of the task. In addition to this, the task of hate speech detection is
particularly complex because messages can involve sarcasm, irony and neutral sentiments that
are challenging for NLP systems to identify. In early models, as Schmidt and Wiegand [19] put
it, simpler surface-level features such as n-gram and character n-grams have been implemented
[
Nobata et al. [20] experiment with features derived from static word embeddings with
annotated data of comments on Yahoo! in three ways. Two of them consist of averaging the
vector representation of all words in a comment derived from two types of word embeddings,
they are the pretrained and word2vec models, both containing 200 dimensions. Their third
approach is based on the representation of paragraph embeddings [
The latest classifiers for hate speech detection take advantage of models such as BERT, RoBERTA and other large multilingual language models [15]. They usually feed the sentence vector representation, the [CLS] token in BERT, into more recent deep learning architectures such as convolutional neural networks, recurrent neural networks and gated recurrent units and they reach very impressive results. In the last SemEval task for detection of ofensive language, the best team reached a F1 score of 0.9204 and the other teams have mostly reached very similar performance in a tight competition.
Our model proposes to work on this line of features because of its potential to capture meaning
beyond a restricted list of words, besides the great number of successful NLP applications that
are based on non-contextualized vector representations of words, for instance GloVe [
Both the English and Spanish training sets are balanced, consisting of 100 haters and 100 nonhaters. The dataset provided by the task organizers contains 200 tweets and a ground truth label for each user. In both datasets provided, user mentions, URLs, and hashtags have been replaced with tags in the form of #HASHTAG#. We remove all hash marks (#) while keeping the accompanying words (USER, URL, and HASHTAG).
We use diferent models to perform the task on English and on Spanish data. Both models exploit contextualized word representations and implement a support vector machine for the task of binary classification.
Both models were tested by the task organizers using the TIRA tool [
The idea underlying our English model is that haters and non-haters might use certain words in diferent contexts. Example (1) below shows a tweet found in the hater class which mentions the word gun in an aggressive circumstance, whereas tweet (2) illustrates an instance from the nonhater class and mentions the same word in a more political context. Thus, the feature selection for the model involves the identification of words and, in this case, of BERT embeddings that significantly vary from one class to another.
1. This money can’t fit in my pockets but I bet that gun fit. 2. New state laws for the new year: California limits gun rights, minimum wages increase #url. . . #url.
To verify whether there are significant diferences in the vector representation of words
between the two classes, we first carry out an experimental coarse analysis with t-SNE [
We first make a list of most frequent tokens by selecting the ones that occur at least 25 times
in both classes. In total there are 788 of them. Note that they are BERT WordPiece tokens [
For the t-SNE analysis, we feed each tweet of a given of the class hater into the BERT model and retrieve the vector representation of a given token () present in the most frequent list. Then, we average all the vectors of of . More formally, let be the token that occurs in a tweet {1, 2,... } of a given . Thus, the vector representation (⃗) of in is: ⃗ [] = ∑︀ =1 [] (1) where N is the number of occurrences of in all tweets by , and ⃗[] is the vector of in . We then repeat the same procedure for the non-hater class and reduce the dimensions. For example, ⃗[gun], which is a matrix of [Nx768], is reduced to a matrix with 2 components [Nx2]. Some of the results can be seen in Figure 1, where each dot represents a ⃗ [].
This coarse evaluation shows that some tokens form well-defined clusters between the two classes such as happy (Figure 1a) and world (Figure 1b). In contrast, others words like amazing (Figure 1c) and indeed even the word gun (Figure 1d) are sprawling and occupy overlapping spaces in both classes, suggesting that they do not have distinguishing vector representations.
Given the results of this coarse analysis with t-SNE, and considering that the reduction of vectors from 768 to 2 dimensions may cause the vector to lose a large amount of relevant information, we turn to a more statistical approach to select the words for our model. 1https://huggingface.co/transformers/main_classes/tokenizer.html (a) Happy (b) World (c) Amazing (d) Gun
Instead of using predefined term lists, we employ a technique called filter approach for
selecting the features (in our case the words) that most diverge between the two classes in
terms of word embeddings. This technique requires two steps. First of all, a statistical test
measures the diference in the vector representation of the tokens, and returns the p-value for
the diference in vector between the two classes for each token. Then, a p-value (our threshold)
is chosen, in order to pick the k most relevant features/tokens. In this study, we analysed
the diference in vectors using the Kolmogorov-Smirnov (K-S) test. Biesiada and Duch [
To start with, we retrieve the same vector representation for each user presented in Equation 1. After that, considering that in this case we want to have a single representation of a token t for each class, we average ⃗[] of all users to get the final representation of , as in: ⃗[] = ∑︀ =1 ⃗[] (2)
In this case N is the number of users that have at least one occurrence of the given and we call the vector ⃗ because it is used for the statistical test. We reach this point with two dictionaries, one for each class, with as key and its corresponding ⃗[] as the value and are ready to run the K-S test. For example, the ⃗[ℎ] from the hater label as variable x and ⃗[− ℎ] as variable y, letting the K-S test be the function − − (x, y).
The results of the K-S test for each token are p-values very close to 1 for most tokens, since they are drawn from the same distribution. However, this is not a problem in our case, because we do not want to know whether they are statistically significant according to the confidence interval. The goal instead is to find out which ⃗[] are lower compared to others, meaning that the ⃗[] between classes are more dissimilar.
Now that we have the p-values for each token in our most frequent list, we must decide on a threshold that will select the number of relevant features to be fed into the classifier. This is done by various runs with diferent thresholds in the model. That is, we pick a p-value, get a single vector representation that is the average of all the ⃗ with t under the p-value of the K-S test, such as:
The embeddings is called feat because it is the feature representation of each user. ⃗[user] is fed to the classifier. The performance is evaluated in terms of accuracy with 5-fold crossvalidation. We finally choose the set of features that results in the model’s best performance. The threshold selected in our submission is 0.998 and the set of tokens consists in a total of 394 tokens/features for the model. As a matter of fact, some of them can be very relevant semantically in the context of hate speech detection such as war, liberal, black, woman, violence, racism, bitch and so forth (all the tokens are presented in Appendix A - List of wordpieces used in the English model).
After having selected the features, we also try diferent layers of representations from BERT’s
outputs given that it is has been observed that each of the 12 layers capture diferent features of
the input text [
As a final step, we add to the features the averaged CLS tokens of each tweet because we
notice that even though the set of tokens is large, there are some users from the test set who do
not contain any of those tokens. Again, we run tests to see which layer of the CLS token is more
advantageous to the model and choose the 12th layer. The experiments are conducted with two
kernels of the support vector machine, the radial basis function (rbf) and the polynomial kernel.
We utilize Bayesian optimization technique [
During the experimental phase, we tested the same approach used in the English data set on Spanish as well. However, in our final submission, we opted for a simpler model, which in our experiments worked better on the Spanish data.
This model follows a more straightforward approach inspired by text classification [
After pre-processing, every tweet {1, 2,... } of a given user is fed into BETO. Then, we extract the vector representation of the CLS token, which encapsulates a single representation of the whole sequence. Lastly, we average these vectors to create the feature representation of each user such as in:
Where N is always 200 given that this is a fixed number of tweets in . The ⃗ is fed into the support vector machine.
We also experimented with summing the CLS tokens and found that results are very similar, given that there is no confound with the frequency of tokens. Similarly to the English model, we test the last three layers of the CLS token in rbf and polynomial functions with Bayesian optimization, and verify that the 11th layer trained on the polynomial kernel (C≈ 7.3588, gamma≈ 0.0285, degree≈ 1.2859) and the 10th layer trained on rbf give the best result in the 5 fold cross-validation, so we submitted both for the shared task, and indeed they have even returned the same labels for the test set.
Table 1 shows that the best accuracy in the training set in English is reached by the model with fewer features (210 tokens) compared to the second place. However, a paired t-test with the results of the 5 fold CV showed that the first and second model are not statistically significantly diferent in the training set (p-value=0.1998). The results of the test test, on the other hand, difer considerably from one another with the 394-token model reaching 4 points better accuracy. One reason for the diference in classification of the test set is that a broader range of tokens included in the features can enhance the performance on unseen data.
2https://github.com/dccuchile/beto
Alternatively to averaging, we try to sum the vectors based on the idea that the frequency with which words occur in the tweets may help the classifier to discriminate better between classes. However, despite having indeed performed better (accuracy 7% higher that our final submission model) in the training set overall, the classification in the test set was overwhelmingly imbalanced with 97 haters out of 100 users and the accuracy was also very imbalanced within the 5 fold cross-validation, showing that it did not generalize well in all folds. It suggests, though, that the classifier learns from the frequency and that there should be a similar number of occurrences for a reasonable performance.
For what concerns the Spanish model, even though we chose the model that extracts the CLS token representation from the 11th layer, the three models actually perform similarly in terms of accuracy as seen in Table 2. The first and second model have even returned the same labels for the classification.
Moreover, we have attempted to apply same approach we used for the English data set, but the results in the training set drop considerably reaching lower performance than with the CLS token approach. One hypothesis for the diference in results could be related to the corpora on which the pre-trained language models are trained. It might be that the language used in the Spanish tweets is more similar to the language in the corpora used for training BETO compared to the English data set and BERT, which would help encompassing the meaning and context of the dataset more easily, therefore prompting better results. Nonetheless, it is dificult to know precisely the reason why they perform so diferently, because of the lack of interpretability of these large language models.
In terms of layer selection, we observed that both models perform quite similarly when trained in each of the three last layers, suggesting that they have very similar representations of tokens.
We present two novel approaches to profile haters on Twitter. The English approach relies on the idea that a set of words can be used in diferent contexts by the hater and non-hater users. The state-of-the-art language model BERT is adopted to capture the contextualized embeddings of tokens. Then, the diference of the vector representation in both classes is measured through the K-S statistical test. And finally, the relevant features are chosen by feeding a set of tokens from diferent thresholds of the test into the support vector machine.
In contrast, we have seen that the same approach does not work as well for the Spanish model. Therefore, inspired by text classification methods, we use the averaged vector representation of all CLS tokens from every tweet of each user as input for the support vector machine. Despite being a simpler model, it yields impressive results considering the amount of training data available.
As a future step, it would be interesting to test the same models with more training data to check whether it boosts their performance, and perhaps replace the SVM approach with a deep learning model. In addition, in this shared task we only process textual information, but in a real scenario other features related to metadata could be included to have more informative and characteristic features, which may improve classification. Lastly, more related to the English model, other types of statistical tests might be experimented as well, in order to distinguish better features for the model. Otherwise, after the statistical test, the model could be trained iteratively with the ablation technique to select best performing features among the ones already selected by the threshold. classification and statistical modeling for policy and decision making, Policy & internet 7 (2015) 223–242. [10] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar, Semeval-2019 task 6: Identifying and categorizing ofensive language in social media (ofenseval), in: Proceedings of the 13th International Workshop on Semantic Evaluation, 2019, pp. 75–86. [11] C. Bosco, F. Dell’Orletta, F. Poletto, M. Sanguinetti, M. Tesconi, Overview of the evalita 2018 hate speech detection task, in: EVALITA@CLiC-it, 2018. [12] M. Sanguinetti, G. Comandini, E. Nuovo, S. Frenda, M. Stranisci, C. Bosco, T. Caselli, V. Patti, I. Russo, Haspeede 2 @ evalita2020: Overview of the evalita 2020 hate speech detection task, 2020. [13] J. Struß, M. Siegel, J. Ruppenhofer, M. Wiegand, M. Klenner, Overview of germeval task 2, 2019 shared task on the identification of ofensive language, 2019. [14] V. Basile, C. Bosco, E. Fersini, D. Nozza, V. Patti, F. M. Rangel Pardo, P. Rosso, M. Sanguinetti, SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter, in: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019, pp. 54–63.
URL: https://www.aclweb.org/anthology/S19-2007. doi:10.18653/v1/S19-2007. [15] M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis, Ç. Çöltekin, SemEval-2020 task 12: Multilingual ofensive language identification in social media (OfensEval 2020), in: Proceedings of the Fourteenth Workshop on Semantic Evaluation, International Committee for Computational Linguistics, 2020, pp. 1425–1447. URL: https://www.aclweb.org/anthology/2020.semeval-1.188. [16] S. Malmasi, M. Zampieri, Challenges in discriminating profanity from hate speech, Journal of Experimental & Theoretical Artificial Intelligence 30 (2018) 187–202. [17] P. Mishra, H. Yannakoudakis, E. Shutova, Neural character-based composition models for abuse detection, arXiv preprint arXiv:1809.00378 (2018). [18] M. Mozafari, R. Farahbakhsh, N. Crespi, A bert-based transfer learning approach for hate speech detection in online social media, in: International Conference on Complex Networks and Their Applications, Springer, 2019, pp. 928–940. [19] A. Schmidt, M. Wiegand, A survey on hate speech detection using natural language processing, in: Proceedings of the fifth international workshop on natural language processing for social media, 2017, pp. 1–10. [20] C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, Y. Chang, Abusive language detection in online user content, in: Proceedings of the 25th international conference on world wide web, 2016, pp. 145–153. [21] J.-M. Xu, K.-S. Jun, X. Zhu, A. Bellmore, Learning from bullying traces in social media, in: Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies, 2012, pp. 656–666. [22] Y. Chen, Y. Zhou, S. Zhu, H. Xu, Detecting ofensive language in social media to protect adolescent online safety, in: 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing, IEEE, 2012, pp. 71–80. [23] N. D. Gitari, Z. Zuping, H. Damien, J. Long, A lexicon-based approach for hate speech detection, International Journal of Multimedia and Ubiquitous Engineering 10 (2015) 215–230.
Appendix A - List of wordpieces used in the English model