Sadly, the presence of toxic messages on social networks, whether in the form of stereotypes, sarcasm, mockery, insult, inappropriate language, aggressiveness, intolerance, or typical of hate speech against immigrants and / or women, among others, is relatively frequent. This presence should not be ignored by the scienti c community, since it is their responsibility to develop tools and systems that allow their automatic detection and elimination. In this paper, we present an exploratory analysis in which di erent deep learning (DL) models for the detection of toxic expressions have been evaluated on the DETOXISIberLEF 2021 challenge using the o cial release of the NewsCom-TOX corpus. Particularly, we compare traditional RNN and state-of-the-art transformer models. Our experiments con rmed that optimum performance can be obtained from transformer models. Speci cally, top performance was achieved by ne tuning a BETO model (the pre-trained BERT model for the Spanish language from the Universidad de Chile) for the toxicity detection tasks. Another contribution of this analysis is the validation of the proposed method for adding task-speci c vocabulary (new tokens) that could help to e ectively extend the original vocabulary of the pre-trained models.
The automatic detection of toxic language, especially in online tweets and comments, is a task that has attracted growing interest from the NLP (Natural Language Processing) community in recent years and has become a tremendously popular and active research area because of its impact on modern society.
In this regard, the DETOXIS challenge is a great opportunity to tackle the hard task of identifying toxic comments in social media. The detection of toxicity is not an easy task at all. It involves much more than just identifying some speci c words or sentences, we must also take the context into account which makes the task even more complex. The present work is well aligned with such interest and its objective is the design and implementation of computational models for toxicity assessment and classi cation of comments in Spanish. Di erent models have been proposed and evaluated for both subtasks of the challenge: { Subtask 1: Toxicity detection task is a binary classi cation task that consists of classifying the content of a comment as toxic (toxic=yes) or not toxic (toxic=no). { Subtask 2: Toxicity level detection task is a more ne grained classi cation task in which the aim is to identify the level of toxicity of a comment (0= not toxic; 1= mildly toxic; 2= toxic and 3: very toxic).
In this paper we will present the di erent toxicity recognition models we
have developed as part of our participation in DETOXIS-Iberlef 2021. Our rst
model is an extension of the system we previously developed for intent detection
and classi cation based on word-embeddings and recurrent neural networks [
Automatic detection of toxic comments to ensure its deletion is mandatory in
a world surrounded by social media. Traditional methods such as RNN models
have provided e cient solutions for similar tasks in multiple elds. Also, RNN
models based on Bi-LSTMs allow to explore the context of the sentence
improving its performance. However, recent advances such as the BERT model have
greatly contributed to enhance natural language processing [
Data provided for the challenge is grouped in NewsCom-Tox dataset which contains around 4357 posts. Most comments are in response of articles from di erent Spanish newspapers. Each comment is classi ed as toxic or not toxic and also in four di erent levels of toxicity. In addition, some other features are included such as argumentation, sarcasm, mockery or insult, for instance [21]. Furthermore, we must consider that classes are not balanced since it contains a higher amount of non toxic comments. 4
As a rst solution for the toxicity analysis we have used a Recurrent Neural Network (RNN), a type of model widely used in the analysis of Twitter messages, for example. RNNs have the ability to process their inputs sequentially, performing the same operation, ht = fW (xt; ht 1), on each of the di erent elements that constitute our input sequence (i.e. words or, to be more exact, their corresponding embeddings), where ht is the hidden state, t the time step, and W the weights of the network.
As it can be observed, the operation is formulated in such a way that the hidden state at each time step depends on the previous hidden states. Hence, the order of the elements in our sequences (i.e. the order of the words) is particularly important. As an immediate consequence, RNNs allow us to handle inputs (i.e. sentences) of variable length, which happens to be an essential feature given the nature of our problem.
Among the di erent possible architectures of this type of networks, we have
opted for the so-called Long Short-Term Memory (LSTM) networks [
Embeddings
(Word) embeddings are vector-type representations obtained for words in
reduceddimensional vector spaces where semantically similar words are always close to
each other. The Fasttext project [
At the time of training, FastText trains by sliding a window over the input
text and either learning the target word from the remaining context (also known
as continuous bag of words, CBOW), or all the context words from the target
word (\Skip-gram"). Learning can be viewed as a series of updates to a neural
network with two layers of weights and three layers of neurons, in which the
outer layer has one neuron for each word in the vocabulary and the hidden
layer has as many neurons as there are dimensions in the embedding space. This
approach is very similar to Word2Vec [14]. However, unlike Word2Vec, fastText
might also learn vectors for sub-parts of words: so-called character n-grams. This
ensures that for instance the words love, loved and beloved all have similar vector
representations, even if they tend to show up in di erent contexts. This feature
enhances learning on heavily in ected languages [
Our approach is based on a 2-layer Bidirectional-LSTM model with a deep
selfattention mechanism which is represented in Figure 1. The model is implemented
in Pytorch [15] and based on the architecture proposed in [
Embedding layer The model is designed to work with sequences of words as inputs, thus allowing us to process any type of sentence. For this, a rst embedding layer is provided that collects the embeddings x1; x2; : : : ; xN corresponding to each of the words w1; w2; : : : ; wN constituing the sentence we want to process, where N is the number of words in our sentence. We initialize the weights of the embedding layer with our pre-trained word embeddings.
Bi-LSTM layer A standard LSTM model behaves in a unidirectional way, that is, the network takes as input the direct sequence of word embeddings and produces the outputs h1; h2; : : : ; hN , where hi is the hidden state of the LSTM cell at time step i, summarizing all the information that the network has accumulated from our sentence up to word wi.
Instead, we have used a bi-directional LSTM (Bi-LSTM) that allows us to collect such information in both directions. In particular, a Bi-LSTM consists of 2 LSTMs, a forward LST M! that allows the analysis of the sentence from w1 to wN , and an inverse or backward LST M which allows a similar analysis to be carried out but in the opposite direction, from wN to w1. To obtain the de nitive outputs of our Bi-LSTM layer, we simply concatenate for each word the outputs obtained from the analysis performed in each speci c direction (see Equation 1 in which jj corresponds to the concatenation operator and L to the size of each LSTM).
hi = !hi jjhi , where hi 2 R2L (1) Attention layer In order to identify the most informative words when determining the polarity of the sentence, the model uses a deep self-attention mechanism. Thus, actual importance and contribution of each word is estimated by means of a multilayer perceptron (MLP) composed of 2 layers with a non-linear activation function (tanh) similar to that proposed in [16].
The MLP learns the attention function g as a probability distribution on the hidden states hi, that allows us to obtain the attention weights ai that each word receives. As the output of the attention layer the model simply computes the convex combination r of the LSTM outputs hi with weights ai, where a convex combination is a linear combination of points where all the coe cients are non-negative and add up to 1.
Output layer Finally, we use r as a feature vector which we feed to a nal task-speci c layer for classi cation. In particular, we use a fully-connected layer, followed by a softmax operation, which outputs the probability distribution over the classes.
Willing to improve the results obtained with the RNN model, we decided to use transformers, a type of model able to process its inputs sequentially as well. For this task we used BERT which stands for Bidirectional Encoder Representations from Transformers allowing to work in both directions. Also, something that make them so powerful is their attention mechanism for identifying which are the keywords in a sentence. These models usually receive sentences as inputs that are divided into single tokens, obtaining a sequence of them.
The way this process is carried out depends on the tokenizer used but BERT's one is based on words and subwords. So, for instance, if a word is not included in the original vocabulary, it will be divided in a sequence of subtokens that all together would form the initial word. However, when we want to netune our model within a speci c eld, it is usual that this occurs with many common words related to the topic, so what we decided to test is whether adding new tokens to our initial vocabulary results in a better toxicity recognition. These new tokens will correspond to the most frequent words in our training dataset that were not already included in the original tokenizer vocabulary. 5.1
Pre-trained NLP models have led to breakthrough performance improvements in
di erent tasks including intent recognition or sentiment analysis, among many
others [17]. However, the adoption of pre-trained language models still must face
two important challenges as their applications expand [20]:
1. First, the need for large training resources; training requires substantial
computation and data, see, e.g., BERTlarge [
Instead of constructing our model with a new vocabulary from scratch, which would require substantial computational resources and training data, or simply adapting the existing pre-trained model on the original vocabulary, which would lead to sub-optimal performance on downstream tasks, we have adopted a simple but e ective approach that addresses both challenges explicitly. Particularly, our method aims at including only a reduced subset of words from the new domain's vocabulary, carefully selected in a rationale way by attending to their actual frequency in our training data, while being able to reuse and adapt the original pre-trained model. This helps reducing required computation and training data while enhancing recognition performance.
To prevent over tting all the experiments have been carried out following a 5-fold
cross-validation scheme. Each setup has been trained for 100 epochs and
'StopEarly' has been adopted as the stopping criterion. No exhaustive exploration of
the hyper-parameters of our models was conducted. Models were trained during
100 epochs using an Adam optimizer [
BERT speci c setup Our BERT model has been implemented and ne-tuned for the toxic comments classi cation tasks using the Simple Transformers library [18]. Although pre-trained tokenizers work at both word and subword levels, the top N new tokens to be added to the vocabulary (i.e. those that happen to be the most frequent in our training data) have been included as word-level units. The rest of new tokens, those connected to more infrequent words, just get split into smaller units to ensure that there are no out-of-vocabulary tokens and all vocabulary units get updated reasonably frequently during training. 6.2
We have evaluated two di erent RNN like models whose main di erence is the
encoder size. Corresponding results are detailed in Table 1. Other values were
also tested though size 200 yielded best performance. Nonetheless, performance
has demonstrated to be signi cantly better for even the most simple version of
our BERT based model: the model ne-tuned from the cased version of BETO,
a BERT model trained on a big Spanish corpus that can be found in [
After con rming the superiority of the BERT based approach, we compared
di erent pre-trained models' performance after ne-tuning them on the rst
downstream subtask: identifying whether a comment is toxic or not, a binary
classi cation problem. As shown in Figure 2, the cased version of BETO clearly
outperforms the other two models: the uncased BETO version and the standard
BERT multilingual base model, a model pre-trained on the top 104 languages
with the largest Wikipedia using a masked language modeling (MLM) objective
[
Fig. 2. BERT models comparison: 5-fold CV results for DETOXIS-Iberlef 2021 Subtask 1.
Then, besides comparing alternative pre-trained models, we also measured the impact of our vocabulary extension method. The results of this analysis have been also presented in Figure 2. Results shown there correspond to independent experiments in which a di erent number of N new tokens (i.e. top N words included as new words) is tested. Results are reported starting from 25 new tokens at rst and up to 100, increasing the amount by 25 on each di erent run.
As it can be deduced from the gure, our vocabulary extension method demonstrates to be e ective achieving a top performance of 76:72% and an improvement over the baseline performance obtained when our model is simply ne-tuned without explicitly adding any new word (i.e. 75:26%, previously reported in Table 1). However, results become worse when the amount of new tokens exceeds a certain small limit, which suggests the importance of nding an adequate balance between the increased complexity of our target model (i.e. the number of new units-embeddings to be learnt) and the available training data.
For the second subtask we have followed exactly the same procedures obtaining similar results as the shown above. In this subtask we are facing a multi-class classi cation problem, where the goal is to identify the toxicity level of every comment in a 0 to 3 scale (i.e. 0: not toxic; 1: mildly toxic; 2: toxic and 3: very toxic). The evaluation results for the adopted experimental setup based on the 5-fold cross-validation scheme have been presented in Figure 3. In this case, only results obtained for our top-performing approach based on the cased version of the BETO pre-trained model have been reported. Besides, and for a proper comparison and analysis, the result corresponding to the case where we simply ne-tune the model while freezing the original tokenizer (i.e. we do not add any new domain-speci c vocabulary) has also been included at the beginning of the series (i.e. the \0" column). Again, we demonstrate that our vocabulary extension method consistently outperforms the prior approach based on general vocabulary. However, once again we con rm that new additive vocabulary can only be introduced to some extent because no further improvement is observed beyond 100 new domain-speci c words (best performance is achieved again by the 25 new tokens con guration).
Analyzing the e ect of di erent data pre-processing methods Adopting
the top-performing approach based on the cased version of the BETO pre-trained
model as a reference, we decided to further explore the use of some of the most
popular text pre-processing techniques to nd out whether they are actually
useful or not. Evaluated techniques include the following:
{ Removing stop words: stop words are removed with the help of the spaCy
library [
Spacy library. { Lemmatization: Spacy lemmatization is applied to generate the root form of the words. { Basic text normalization: special words including emojis, emails, percentages, money, phone numbers, times, dates, urls and/or hashtags are assimilated and replaced by a special tokens, such as MAIL, DATE, URL,... to prevent information from being lost during data representation.
The obtained results have been summarised and sorted in terms of performance in Table 2. As it can be observed, none of the applied techniques that are aimed at removing tokens were found to be e ective. In short, DL methods that do use embedding representations seems to not require the removal of anything. Speci cally, in that n-dimensional vector words like \dogs" and \dog" would already be closer to each other. So, the need to lemmatize becomes unnecessary.
With regards to stopwords, although it could be convenient to remove many of them, we should notice that stopword lists may contain words which shouldn't be removed in certain domains or tasks, as it happens to be the case. Generally speaking, we should not remove anything (e.g., a word or a punctuation mark) that could be useful in some way. Again, DL models working with vector embeddings, similarly to lemmatization, are currently the best methods to handle and lter those irrelevant terms.
Finally, it is interesting to conclude that basic text normalization has proven to be successful (i.e. performance decreases if we omit it). In this case, the process of transforming some words into their single canonical form still helps our model by reducing the number of unique words (i.e. reducing the vocabulary size helps reducing the model complexity and improving its performance).
After carefully analyzing the obtained results that have been previously presented, we decided to submit the runs on both DETOXIS-Iberlef 2021 challenge subtasks for our top-performing approach based on the cased version of the BETO pre-trained model when gradually increasing the amount of new tokens in steps of 25 from 0 to 100, thus resulting in 5 di erent runs and submissions for each task. Details about the challenge and its evaluation are presented in [21].
In both tasks, we are named as GTH-UPM. These rankings are available on
DETOXIS-Iberlef 2021 o cial website [
If we move on to the second subtask, our result in Table 4 corresponds to the run with 75 new tokens, thus also demonstrating the convenience of the proposed extension method. In this case, in spite of the mismatch between the optimization/evaluation parameters (i.e. the subtask aims at CEM evaluation metric while our model was speci cally trained targeting weighted-F1) our model has achieved a signi cantly better result than the best result that we previously obtained. 7
In this paper, we have presented an exploratory analysis in which di erent deep
learning (DL) models for the detection of toxic expressions have been
evaluated on the DETOXIS-IberLEF 2021 challenge using the o cial release of the
NewsCom-TOX corpus. Particularly, we have compared traditional RNN and
state-of-the-art transformer models including standard BERT [
As another important contribution of this work, we have proposed and validated a simple but e ective method for extending our pre-trained models with domain-speci c vocabulary. The method accounts for term frequencies to rank and select speci c words to be added at the word level. As a result, the performance of the extended model can be signi cantly improved. This approach could be particularly attractive to ad-hoc and special-purpose or very speci c domains with unique vocabularies where limited training data is available. Nonetheless, additional work still needs to be done with regards to automatically nding or identifying the exact and optimal amount of new tokens to be added (i.e. the precise value that achieves a good balance between model complexity and available training data).
Furthermore, unlike traditional approaches not using DL nor embedding representations, when using transformer-like models and testing di erent text preprocessing methods, it has been observed that preserving the raw structure of the texts, by not removing anything while simply performing a very basic text normalisation, helps achieving optimal performance.
Funding The work leading to these results has been supported by the Spanish Ministry of Economy, Industry and Competitiveness through the CAVIAR (MINECO, TEC2017-84593-C2-1-R) and AMIC (MINECO, TIN2017-85854-C44-R) projects (AEI/FEDER, UE). Ricardo Kleinlein's research was supported by the Spanish Ministry of Education (FPI grant PRE2018-083225).
Acknowledgments We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for part of this research. 14. Mikolov, T., Chen, K., Corrado, G.S., Dean, J.: E cient estimation of word representations in vector space (2013), http://arxiv.org/abs/1301.3781 15. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic di erentiation in pytorch. In: NIPS 2017 Workshop on Autodi (2017), https://openreview.net/forum?id=BJJsrmfCZ 16. Pavlopoulos, J., Malakasiotis, P., Androutsopoulos, I.: Deep learning for user comment moderation. In: Proceedings of the First Workshop on Abusive Language Online. pp. 25{35. Association for Computational Linguistics, Vancouver, BC, Canada (Aug 2017). https://doi.org/10.18653/v1/W17-3004, https://www.aclweb.org/anthology/W17-3004 17. Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X.: Pre-trained models for natural language processing: A survey. Science in China E: Technological Sciences 63(10), 1872{1897 (Oct 2020). https://doi.org/10.1007/s11431-020-1647-3 18. Rajapakse, T.C.: Simple transformers. https://github.com/ThilinaRajapakse/simpletransformers (2019) 19. Schmidt, A., Wiegand, M.: A survey on hate speech detection using natural language processing. In: Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media. pp. 1{10. Association for Computational Linguistics, Valencia, Spain (Apr 2017). https://doi.org/10.18653/v1/W17-1101, https://www.aclweb.org/anthology/W17-1101 20. Tai, W., Kung, H.T., Dong, X., Comiter, M., Kuo, C.F.: exBERT: Extending pre-trained models with domain-speci c vocabulary under constrained training resources. In: Findings of the Association for Computational Linguistics: EMNLP 2020. pp. 1433{1439. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10.18653/v1/2020. ndings-emnlp.129, https://www.aclweb.org/anthology/2020. ndings-emnlp.129 21. Taule, M., Ariza, A., Nofre, M., Amigo, E., Rosso, P.: Overview of the detoxis task at iberlef-2021: Detection of toxicity in comments in spanish. Procesamiento del Lenguaje Natural 67 (2021) 22. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017), https://proceedings.neurips.cc/paper/2017/ le/3f5ee243547dee91fbd053c1c4a845aaPaper.pdf