The importance of the detection of aggressiveness in social media is due to real e ects of violence provoked by negative behavior online. Indeed, this kind of legal cases are increasing in the last years. For this reason, the necessity of controlling user-generated contents has become one of the priorities for many Internet companies, although current methodologies are far from solving this problem. Therefore, in this work we propose an innovative approach that combines deep learning framework with linguistic features speci c for this issue. This approach has been evaluated and compared with other ones in the framework of the MEX-A3T shared task at IberEval on aggressiveness analysis in Spanish Mexican tweets. In spite of our novel approach, we obtained low results.
The opinions expressed online by users are usually uncontrolled and this lack of
control facilitates and supports negative online behaviors such as cyberbullying,
racism, sexism and any form of hate. In the last few years, governments, social
media platforms, Internet companies and communities of citizens are spending
a growing amount of e orts to monitor and contrast such forms of online
aggressive behaviors and attitudes, with the main aim of limiting it. An example
of governmental dedication about this subject is the campaign No Hate Speech
Movement of the Council of Europe for human rights online. On the academic
side, the research interest about this issue is increasing and the approach is
naturally interdisciplinary. Especially in the natural language processing (NLP) eld,
the attention is supported by international and national workshops or campaigns
of evaluation like the competition proposed in the framework of IberEval 2018 by
the organizers of MEX-A3T4 [
The remainder of the paper is organized as follows. In Section 2 we describe synthetically the previous approaches used until today. In Section 3 we present our proposal followed by the results obtained in the competition (Section 4). Finally, in Section 5 we draw some conclusions. 2
Currently, commercial and simple methods to deal with the automatic detection
of negative online behaviors rely on the use of blacklists , essentially composed
with slurs and swear words. However, ltering the messages in this way does not
provide a su cient remedy because it falls short when user-generated content
is more subtle. Therefore, the research challenges in this eld are oriented at
investigating deeply all dimensions of language and also the communication on
the Web, to envision deeper and more sophisticated solutions exploiting surface
features ([
The common approach to detect aggressiveness online is formulating a prediction
task, and in particular MEX-A3T organizers proposed a classi cation task with
the aim to distinguish aggressive tweet from the non-aggressive [
In the next subsections we describe the set of features provided along with the training data and deep learning architecture operations.
The linguistic features employed aim to cover all the above aspects about the aggressiveness in the context of a tweet. Textual features As textual features we take into account the polarity (positive/negative/neutral) of emoticons5, used especially for giving contextual information to readers.
Style and writing density We consider also stylistic traits of authors, such as: the use of speci c abbreviations used in Mexican tweets (hdp, alv), the number of characters per sentence and word, the use of some elements of punctuation (question, exclamation marks and sequences of dots) and the uppercase characters, inspecting if the user writes all in uppercase or just some letters.
Bag of words In order to understand the importance of some words respect to others, we extract trigrams of words weighted with tf-idf.
Lists of aggressive words Considering the fact that the aggressive text aims to o end, attack, humiliate and hurt an individual or collective target, we created two lists containing speci cally derogatory adjectives and vulgar expressions like profanities and insults (chinga a tu madre, vete a la verga).
Syntactic patterns Another factor involved in aggressive texts is the target, implicit or explicit, to whom the insults or profanities are addressed. Therefore, we examined the syntactic combinations of target explicited with mention (@usuario) or proper name with derogatory adjectives and vulgar expressions.
A ective features Finally, as said above, we take into account the emotions
concerning aggressiveness and we observed that anger and disgust are the
principal emotions that provoke this kind of behavior. For this feature, we used Spanish
Emotion Lexicon (SEL) provided by [
In order to allow our architecture to process these features, we preprocessed
the data deleting symbols and urls that can hinder the process of extraction of
features and pos-tagging the texts using FreeLing [
In this section, we describe the deep learning (DL) framework for detecting
the aggressive tweets. The proposed model is inspired by the deep architecture
proposed in [
Embedding layer: Instead of using any pre-trained word embedding scheme, we have built a vocabulary table which is learned from the training data. The embedding layer works as a lookup table which maps vocabulary word indices into low-dimensional vector representations. As the aggressive tweets are of variable length, we used the zero-padding (i.e., the missing part replaced by zeros) to maintain the input vector to a xed size L.
Features: For Model-1, we integrated the features in the embeddings. We derived a feature set as described in Section 3.1. We combined these features with DL in Model-1. However, we did not combine the features with DL framework in Model-2.
Convolutional layer: Let ti 2 Rk be the k-dimensional vector corresponding to the i-th word in the tweet. A tweet is represented as t1:n = t1 t2 ::: tn, where, the tweet contains the words t1; t2; : : : ; tn and is the concatenation operator.
Also, let tf1:m = tf1 tf2 ::: tfm be the feature set for the tweet t1:n. After combining the feature set tf1:m with the vector representation of the tweet t1:n, the resulting vector is l1:m+n = tf1:m t1:n. Therefore, l1:m+n = l1 l2 ::: lm+n, where either li 2 tf1:m or li 2 t1:n.
Let li:i+j refer to the concatenation of li; li+1; : : : ; li+j . In the convolution operation, the lter w 2 Rhk is applied to a window of h words to produce new features such as feature si is generated from a window of words li:i+h1 by si = f (w:li:i+h 1 + b), where, b 2 R is a bias term and f is a non-linear function. A feature map s = [s1; s2; : : : ; snh+1] (where, s 2 Rn h+1) is produced by applying the aforesaid lter to each possible window of h words (i.e., fl1:h; l2:h+1; : : : ; lnh+1:ng) in the tweet. The max-pooling operation is applied to the feature map s to obtain the maximum value s0 = maxfsg for the particular lter. The objective of the max pooling is to capture the most important feature with the highest value for each feature map. However, the proposed architecture uses multiple lters with varying window sizes to obtain multiple features. Then, these features are passed to the next layer, i.e., a fully-connected layer.
Fully-connected layer: The fully-connected layer is also known as the dense layer. The max-pooling operation selects the best features from each convolutional kernel. Thus, all the resulting features which are selected from the maxpooling are combined in the fully-connected layer. The output of fully connected layer is passed to the output layer.
Output layer: The nal layer (i.e., the output layer) is made of 2 neurons as the given tweets are of 2 target classes (i.e., aggressive and non aggressive). The output layer uses `softmax' as the nonlinear activation function. 4
In the framework of the evaluation campaign, we have submitted two runs: the DL based on CNN with feature engineering (DLF+FE) and a simple DL framework based on CNN (DLF). In order to evaluate the performance of the systems in the competition, the organizers use the F-measure of aggressiveness class. In Table 1, we report the scores obtained along with our position in the ranking for the aggressive tweets prediction. In spite of the novelty of our approach, the results are low and the feature engineering does not outperform the deep learning based model. In this work, we investigate the automatic detection of aggressive texts by incorporating linguistic features into deep learning architecture. Considering the low results, we carry out error analysis that reveals that our systems mainly fail to classify tweets with orthographic errors and sarcastic or ironic utterances, such as: \USUARIO #LOS40MeetAndGreet 9. Por q es una mama luchona que cuida a su bendicion"; \Quiero hablar con el que invento el hecho de \levantarse temprano" Que xxxxx estaba pensando". Therefore, taking into account these observations, we will investigate the use of humorous devices to express negative opinions. Moreover, as future work, in order to make deeper analysis about the impact of feature engineering on deep learning approach, we would like to propose this approach in similar research issues.
The work of Simona Frenda was partially funded by the Spanish MINECO under the research project SomEMBED (TIN2015-71147-C2-1-P).
Frenda et al.