This paper describes the system we developed for the task on Aggressive detection in Authorship and aggressiveness analysis in Twitter (MEX-A3T)The task focuses on the detection of aggressive comments in tweets that come from Mexican users. We have analyzed three kinds of models and the proposed system is a combination of them. The rst model is based on Convolutional Neuronal Networks whose outputs feed a LSTM Neural Network. The second one uses the pre-trained Universal Sentence Encoder for encoding sentences into embedding vectors. Finally, the third one consists in a simple Multi-layer Perceptron. The nal results show that our model achieves good results.
Nowadays, the use of social networks is increasing rapidly. Among them, Twitter stands out as a broadcast medium of information. Many users use this social media as one of the main sources for obtaining news. However, many of those users are attacked by tweets with aggressive messages.
This phenomenon constitutes a problem that a ects di erent groups of
people, due to harassment towards immigrants, women or for instance, sexist
comments [
In this work, we propose a system formed by the combination of three strategies. Each of them analyzes the tweet to be classi ed in a di erent way. The rst one is based on Convolutional Neural Networks whose outputs feed a LSTM Neural Network. The second one uses the pre-trained Universal Sentence Encoder for encoding sentences into embedding vectors. The third one consists of a simple Multi-layer Perceptron which gets the TF-IDF representation of the tweet. Then, the strategies are combined in order to build a system that takes into account each of the analysis and predicts whether a given tweet is aggressive or not.
The rest of the paper is organized as follows. Section 2 describes our system. Experimental results are then discussed in Section 3. Finally, we present our conclusions with a summary of our ndings in Section 4. 2 2.1
The rst step for the development of the system is the preprocessing of the texts. In this phase di erent characteristics, typically present in the tweets, and that possibly do not have discriminatory semantic information, are normalized. In this way, the numbers are replaced by the num tag, dates by the date tag, and all the links by the url tag. In addition, user mentions, identi ed by the rst character @, are replaced by user. The hashtags were not processed to avoid losing information that they may contain. 2.2
We propose a system that consists of a combination of di erent strategies as Figure 1 shows. The rst one is a deep learning model (CNN-LSTM) at the word level. For each tweet, CNN-LSTM receives as input the word embbedings, which are processed by a CNN for obtaining a sequence of vectors. These vectors can be seen as the representation of n-grams according with the size of the kernel. In the next section, the details are discussed. Then, the vectors feed a LSTM model for obtaining a prediction. The second model (USE-MLP) takes as input a vector for a tweet. This vector is obtained with the pre-trained Universal Sentence Encoder based on the transformer architecture. Then, a Multi-layer Perceptron is used to get a prediction. Finally, a similar model to the previous one is used in the third one (TFIDF-MLP). The di erence is in the input of the Multi-layer Perceptron. In this case, the vector is the TF-IDF representation of the tweet. In addition, a new component is concatenated to the vector according to a linguistic feature based on a lexicon of obscene and vulgar phrases in the Mexican Spanish. Then, the nal prediction is obtained by majority of votes, given the prediction of each model. In each case, cross entropy is used as the loss function. 2.3
In this rst model, as was mentioned before, the tweets are represented with a sequence of word embeddings. For this, the Word2vec MEX-A3T model provided by the organizers of the competition is used. This has been trained with the MEX-A3T corpus containing 500,000 tokens. The size of the embeddings is 200. The objective of this model is to process bigrams present within a tweet in a sequential manner. The approximation used to obtain the sequence of bigrams vectors is shown in the gure 2. Where 150 lters of 2x200 are used, 2 correspond with the size of the bigrams and 200 correspond with that of the embeddings. The result is a column matrix with depth 150, so that the i-th component taken in depth, can be seen as a high level representation of the i-th bigram. Then, each of these vectors is the input at each time step of the LSTM Recurrent Neural Network which can process them sequentially. Finally, a Softmax layer is used to obtain the prediction. 2.4
The second model takes advantage of the pre-trained Universal Sentence Encoder
[
A problem that frequently occurs with the approaches based on deep learning is
the lack of data to train the models. To solve this problem, a model based on a
traditional approach has been included in the system. For this, each tweet has
been represented as a TF-IDF vector. Additionally, a linguistic feature has been
incorporated into the vector. Basically, this feature corresponds to the number
of aggressive phrases contained in the tweet. The identi cation of these phrases
is based on the study carried out in the work [
The Training set has been divided in the experiments, separating 30% for the Validation set. The results of the F-measure of the aggressive class on that Validation set were 0.64 for USE-MLP, 0.68 for TFIDF-MLP and 0.65 for CNNLSTM, while for the combination of the three models 0.68 was obtained. As can be seen, the best results were achieved with the simplest model, in the same way as in the Test set as shown below.
Table 1 shows the results on the Test set for di erent variants and the result of the best system in the competition (best). The run1 corresponds to the combination of the commented models. On the other hand, run2 and run3 are systems that only take into account the CNN-LSTM and TFIDF-MLP models respectively. Our best result is obtained with the simplest model, which reaches the third position in the competition with a value very close to the rst two in the F-measure of the aggressive class (F1), and in both class (F(P,R)). Our particular results show that the lack of data can a ect the models based on deep learning, with which in this case worse results were obtained. In addition, other problem that may a ect the performance of the deep learning based system is the fact that rare or misspelled words can not be represented with the embeddings. This can badly condition the training, since important information may be lost. 4
We proposed a combination of three di erent models for the MEX-A3T task on aggressive detection in Twitter. The rst one uses a CNN whose outputs feeds to a LSTM model. The second model analyzes the input at the full text level with the Universal Sentences Encoder. The third model is the simplest one that takes a TF-IDF representation of the text, and obtains the prediction with a Multi-layer Perceptron. The best results have been obtained with this last model, instead of the system which combines all the three models. This can be for the lack of data to train deep learning models, or for the problem of rare words that can not be represented with the embeddings. Thus, for future works, it is important dealing with these problems to improve the performance of the system.
Acknowledgments. The work of the second author was partially funded by the the Spanish MICINN under the research project MISMIS-FAKEnHATE on Misinformation and Miscommunication in social media: FAKE news and HATE speech (PGC2018-096212-B-C31).