With the use of social networks, the automatic detection of hate speech has become of great importance to prevent people, being protected by anonymity, from feeling free to discriminate against diferent groups. This document describes two approaches taken to detect hate speech by author: the first based on the individual processing of tweets by the author, which establishes a threshold of hate tweets to identify hate speech; the second based in the concatenation of tweets by author for processing.
The corpus for the development phase was divided into two languages (English and Spanish), each language was made up of 200 authors, with 200 tweets each. The labels were at the author level, that is, there were no tags for each tweet. For the test phase, for each language there were 100 authors with 200 tweets each.
Links, user mentions, and hashtags were substitute, for all tweets, with #URL#, #USER# and #HASHTAG#, respectively.
The pre-processing steps for both languages were the same. Links, user mentions, hashtags, and retweets were removed. All non-English and non-Spanish characters, respectively, were removed. The occurrence of more than two consecutive characters (letters) was replaced by only two characters. All punctuation was remove and all numbers were substituted with 0. Emojis (emoticons) were decoded into their text equivalents. Then all the text was lowercase.
Diferent approaches were followed for each language. For English, a manual labeling of the tweets was carried out, so each tweet was considered individually for the training. For Spanish, the original author labeling was used, concatenating all the tweets by author separated by a special token. Each approach is detailed in the next subsection.
A manual labeling of tweets from 45 authors was made by us, based in the original labeling: 23 authors labeled as hate speech and 22 authors as non-hate speech. From these tweets, all tweets labeled as hate speech were taken, 405 tweets; from the rest of the tweets, the same number of tweets was randomly selected in order to have a balanced training corpus. GloVe was used for the embeddings.
A deep neural network was used for the classification 1, the architecture is defined in figure 1. The activation function used for the inner dense layers was relu, for the last dense layer sigmoid was used. To avoid overfitting, two Dropouts 0.8 and 0.5 were used, as can be seen in Figure 1. For the inner dense layers, a kernel regularizer l2 with a value of 0.0001 was set. For the last Dense layer a l2 activity regularizer was set with the same value. The model was configured with binary crossentropy for the loss, an Adam optimizer and using accuracy as metric.
A 10-Fold Cross-Validation was used in the training. Once the training was finished, the rest of the tweets in the corpus were classified. Next, the sum of Hate Speech Tweets (HST) by author was performed. Taking into account the original labeling (by author) and the HST number, a threshold was defined i.e., if an author had more than a certain HST number, this author was classified as a hate speech spreaders. The threshold was set to 30.
All tweet per author were concatenated, using the token <EOT> as separation. For classification a SVM was used2. A linear kernel was used and a max number of iterations of 2000 was set (all the other parameters were left as default). For the training, 10-Fold Cross-Validation was used. Tokenization was perform with CountVectorizer, using the default values.
Accuracy was used as evaluation metric according to the specifications of the organizer of PAN
2021 and TIRA [
Using the manual tweets labeling approach not only flawed the criteria for detecting hate speech but also makes it impossible to recognize the full context by author thus, hate spreaders whose individual tweets do not represent hate speech, but in a general way constantly attack a specific group, cannot be identified with this approach. On the other hand, concatenate all the tweets allows to analyze the complete context by author, but makes processing dificult (200 tweets per author). The results obtained shows that the concatenate approach is better generalizing.
As future work, a more in-depth analysis using deep neural network with tweets concatenation and SVM with labeling of tweets is proposed. In the same way, using the concatenation approach analyzes how the use of mini batches of tweets by author is carried out, maintaining the author’s labels, so that part of the context is preserved and processing is facilitated.
The work was done with support of the Government of Mexico via CONACYT, SNI and the Secretaría de Investigación y Posgrado of the Instituto Politécnico Nacional, Mexico