In this paper we describe our participation for the Aggressiveness Detection Track in the second edition of MEX-A3T. We evaluate di erent strategies for text classi cation, including classi ers such as Support Vector Machines and a Multilayer Perceptron trained on n-grams (words and characters) and word embeddings. We also study the inclusion of features to try to give context to the text messages and explore if people verbally attack di erently depending on their traits and overall environment. Preliminary results show that our strategy is competitive to detect aggression in tweets, ranking in 2nd place with respect to the participants of 2018 and 2019.
Technology has changed the way in which people communicate with each other,
giving rise to new services such as social networks, where a style of informal
communication is used. Such social networks, though, present several challenges to
maintain communication channels open to the free sharing of ideas. The
intolerance and aggressiveness of certain users a ects the experience of other consumers
or people interested in being part of the communities and their conversations.
The fact of not being face to face in the communication channel and even
preserve anonymity, encourages these individuals to express themselves o ensively.
However, the volume of messages that are sent daily, the growth of online
communities, and the respective ease of access to these social networks, make the
moderation of communication channels a di cult task to be dealt with by
conventional means, and as people increasingly communicate online, the need for
high quality automated abusive language classi ers becomes much more
profound[
One of the goals of the second edition of MEX-A3T[
After loading the train and test sets, we strip the tweets from non-alphanumeric characters and only keep some relevant Spanish characters (a,e, ,o,u,n~,and u), all words are then made lowercase and subsequently we noticed that in both sets there exists many di erent terms to express laughter (mainly due to how many times "ja" is repeated when the word "jaja" appears and because of typos) so that led us to replace every word containing "jaja" to "risa" (laugh), with the purpose of decreasing the number of terms that represent this emotion. It is worth mentioning that we also created and conducted experiments on a version of the datasets where emojis were converted to text and hashtags were separated by words (e.g., ":)" would turn into "smiling face", and "#FelizMiercoles" would be "feliz miercoles"), however most hashtags were wrongly separated and the performance of the classi ers decreased by incorporating these steps and were therefore discarded. 2.2
We conducted our research using the following features: Lexical: We use word n-grams (n=1, 2) and char n-grams (n=3, 4) as features, this collection of terms is weighted with its term frequency-inverse document frequency (TF-IDF).
Document Embeddings: The objective was to represent the tweets through
Word Embeddings[
eral strategies to obtain unsupervised author pro les for each document [
Perspicuity score / In esz scale: Based on [
All the extra categorical features mentioned above were concatenated following a One Hot Encoding scheme. 3
The datasets were provided by MEX-A3T Team. Table 1 shows the distribution of training and test partitions for Spanish tweets.
We separated the training set in 67% for training and 33% for validation
to evaluate our experiments with di erent combinations of features discussed
in section 2.2. We started our research by recreating the baselines described in
the overview of the rst edition of MEX-A3T[
We trained Linear Support Vector Machines and a Multilayer Perceptron as
classi ers for this task, and we decided to use the perceptron as the nal system
to submit our predictions since it exhibited the best results in the validation
stage, as shown in Table 2 where we obtained the F1-score macro and the
FMeasure over the aggressive class. We performed all modeling regarding the
creation of tf-idf feature matrices and SVM classi ers using scikit-learn[
As stated before, the Multilayer Perceptron was chosen as nal system, however,
because of time and memory constraints we had to train this model using only
character n-grams of range [
To breakdown our results, we started by getting the 10 most valuable n-grams at character level separated by length, as shown in Table 4. With respect to the aggressive class, our nal con guration had more false positives than false negatives, meaning that it was easier for an aggressive tweet to be missclassi ed as non-aggressive than the other way around. Despite running several experiments and adding new features trying to give context to the tweets, in hopes of improving classi cation in this task, unfortunately these strategies showed, at best, almost unnoticeable changes in the results, and hinder of classi cation at worst. After manual inspection, we observed that this could have happened because: { Occupation and Location predictions did not group the messages in a balanced way, in fact, most tweets would fall under only one out of eight available categories for occupation and six categories for location. { SOM Coordinates would not enhance the classi cation scores before as the clusters were capturing word repetition instead of thematic aspects for each tweet. Later experiments (after submission of results) showed that this behaviour was caused because the clustering was made with n-grams; training the SOM with word embeddings created with the train set of this task (without external resources) solved this issue and did a better job at grouping the tweets by subjects. { There was no relevant pattern by applying a perspicuity score to each tweet, as there were multiple cases of similar scores assigned to both aggressive and non-aggressive messages. In this paper, we describe our strategy to classify aggressive and non-aggressive tweets in Mexican Spanish. In our best performing system, we use only lexical features and our results show a better performance than most results of all participants. This outcome, and the fact that the F-measure for the aggressive class is still low compared to the score on the non-aggressive class, motivates the idea of future work focusing on feature analysis for aggressiveness detection and explore which representations are truly relevant, including word embeddings, Bag of Words and Characters of di erent n-gram ranges, see if these complement each other and if so, how to combine them. We analyzed our clustering strategies, and after changing the way they were trained we could observe slight improvement in classi cation results, motivating us to keep experimenting on ways to try to add context to the text messages. We also believe in the potential that neural networks display for this task, and that more research on how to build and train them properly will certainly improve the current situation of this task. As future work, we look forward to develop new strategies based on deep neural networks, such as Recurrent Neural Networks, which are tools aimed to work with sequential data similar in nature to time series.