English. Hate speech is prevalent in social media platforms. Systems that can automatically detect offensive content are of great value to assist human curators with removal of hateful language. In this paper, we present machine learning models developed at UW Tacoma for detection of misogyny, i.e. hate speech against women, in English tweets, and the results obtained with these models in the shared task for Automatic Misogyny Identification (AMI) at EVALITA2018.
Italiano. Commenti offensivi nei confronti di persone con diversa orientazione sessuale o provenienza sociale sono oggigiorno prevalenti nelle piattaforme di social media. A tale fine, sistemi automatici in grado di rilevare contenuti offensivi nei confronti di alcuni gruppi sociali sono importanti per facilitare il lavoro dei moderatori di queste piattaforme a rimuovere ogni commento offensivo usato nei social media. In questo articolo, vi presentiamo sia dei modelli di apprendimento automatico sviluppati all’Universita` di Washington in Tacoma per il rilevamento della misoginia, ovvero discorsi offensivi usati nei tweet in lingua inglese contro le donne, sia i risultati ottenuti con questi modelli nel processo per l’identificazione automatica della misoginia in EVALITA2018.
Inappropriate user generated content is of great
concern to social media platforms. Although
social media sites such as Twitter generally
prohibit hate speech1, it thrives online due to lack
of accountability and insufficient supervision.
Although social media companies hire employees to
moderate content
In this paper, we focus on the automatic
detection of misogyny, i.e. hate speech against women,
in tweets that are written in English. We present
machine learning (ML) models trained for the
tasks posed in the competition for Automatic
Misogyny Identification (AMI) at EVALITA2018
Interest in the use of ML for automatic
detection of online harassment and hate speech is
fairly recent
For the AMI@EVALITA competition, which is
the focus of the current paper, we experimented
with the extraction of lexical features based on
dedicated lexicons as in
The training data consists of 4,000 labeled tweets that were made available to participants in the AMI@EVALITA competition. As Table 1 shows, the distribution of the tweets over the various labels is imbalanced; the large majority of misogynistic tweets in the training data for instance belong to the category “Discredit”. In addition, the distribution of tweets in the test data differs from that in the training data. As the ground truth labels for the test data were only revealed after the competition, we constructed and evaluated the ML models described below using 5-fold crossvalidation on the training data. 2.1
Text Preprocessing. We used NLTK2 to tokenize the tweets and to remove English stopwords. Feature Extraction. We extracted three kinds of features from the tweets:
Bag of Word Features. We turned the
preprocessed tweets into BOW vectors by counting the
occurrences of token unigrams in tweets,
normalizing the counts and using them as weights.
Lexical Features. Inspired by the work of
Model Training. We trained 3 EoC models for designating a tweet as misogynous or not (Task A). The EoC models differ in the kind of features they consume as well as in the kinds of classifiers that they contain internally.
EoC with BOW (resham.c.run2)5: an ensemble consisting of a Random Forest classifier (RF), a Logistic Regression classifier (LR), a Stochastic Gradient Descent (SGD) classifier, and a Gradient Boosting (GB) classifier, each of them trained on the BOW features.
EoC with BOW and sentiment scores (resham.c.run1): an ensemble consisting of the same 4 kinds of classifiers as above, each of them trained on the BOW and sentiment score features.
EoC with BOW, sentiment scores, and lexical
features (resham.c.run3): an ensemble
consisting of
– RF on the BOW and sentiment score features
– SVM on the lexical features
– GB on the lexical features
– LR on the lexical features.
– GB on the BOW and sentiment features
All the ensembles use hard voting. For training
the classifiers we used scikit-learn
For Task B, our winning system himani.c.run3 consists of a pipeline of two classifiers: the first classifier (step 1) in the pipeline labels a tweet as misogynous or not, while the second classifier (step 2) assigns the tweets that were labeled misogynous to their proper category and target.
For Step 1 we trained a deep neural network
that consists of a word embedding layer, followed
by a bi-directional LSTM layer with 50 cells,
a hidden dense layer with 50 cells with relu
5Here ’resham.c.run2’ refers to the second run of the data
submitted by the author in connection with the competition.
Similar citations that follow have a corresponding meaning.
activation, and an output layer with sigmoid
activation. For the embedding layer we used the
pretrained Twitter Embedding from the GloVe
package
Next we describe how we trained the models used in Step 2 in himani.c.run3.
Text Preprocessing. We used the same text preprocessing as in Section 2.1. In addition we removed words occurring in more than 60 percent of the tweets along with those that had a word frequency less than 4.
Feature Extraction. We turned the preprocessed tweets into Bag of N-Gram vectors by counting the occurrences of token unigrams, bigrams and trigrams in tweets, normalizing the counts and using them as weights. For simplicity, we keep referring to this as a BOW representation.
Model Training. For category and target
identification, himani.c.run3 uses an EoC approach
where all classifiers are trained on the BOW
features mentioned above. The EoC models for
category identification on one hand, and target
detection on the other hand, differ in the classifiers they
contain internally, and in the values of the
hyperparameters. Below we list parameter values that
differ from the default values in scikit-learn
EoC for Category Identification: – LR: inverse of regularization strength C is 0.7; norm used in the penalization is L1; optimization algorithm is ‘saga’. – RF: number of trees is 250; splitting attributes are chosen based on information gain. – SGD: loss function is ‘modified huber’; constant that multiplies the regularization term is 0.01; maximum number of passes over the training data is 5. – Multinomial Naive Bayes: all set to defaults. – XGBoost: maximum depth of tree is 25; number of trees is 200.
EoC for Target Identification:
Approach majority baseline resham.c.run1 resham.c.run2 resham.c.run3 himani.c.run3
– LR: inverse of regularization strength C is 0.5; norm used in the penalization is L1; optimization algorithm is ‘saga’. – RF: number of trees is 200; splitting attributes are chosen based on information gain.
For completeness we mention that himani.c.run2 consisted of a two-step approach very similar to the one outlined above. In Step 1 of himani.c.run2 tweets are labeled as misogynous or not with an EoC model (RF, XGBoost) trained on the Bag of N-Gram features. In Step 2, a category and target label are assigned with respectively an LR, XGBoost-EoC model and an LR, RF-EoC model in which all classifiers are trained on the Bag of N-Gram features as well. Since this approach is highly similar to the himani.c.run3 approach described above and did not give better results, we do not go into further detail. 3 3.1
yny categories are characterized by their own, particular language, and that during training our binary classifiers have simply become good at flagging misogynous tweets from categories that occur most often in the training data, leaving them under-prepared to detect tweets from other categories.
Regardless, one can see that the ensembles benefit from having more features available. Recall that resham.c.run2 was trained on BOW features, resham.c.run1 on BOW features and sentiment scores, and resham.c.run3 on BOW features, sentiment scores, and lexical features. As is clear from Table 2, the addition of each feature set increases the accuracy. As already mentioned in Section 2.2, the accuracy of himani.c.run3, which is a bidirectional LSTM that takes tweets as strings of words as its input, is lower than that of the resham models, which involve explicit feature extraction. Table 3 contains the results of our models for Task B in terms of F1-scores. Following the approach used on the AMI@EVALITA scoreboard, both subtasks are evaluated as multiclass classification problems. For Category detection, there are 6 possible class labels, namely the label ‘nonmisogynous’ and each of the 5 category labels. Similarly, for Target detection, there are 3 possible class labels, namely ‘non-misogynous’, ‘Active’, and ‘Passive’.
When singling out a specific class c as the “positive” class, the corresponding F1-score for that class is defined as usual as the harmonic mean of the precision and recall for that class. These values are computed treating all tweets with ground truth label c as positive examples, and all other tweets as negative examples. For example, when computing the F1-score for the label “Sexual harassment” in the task of Category detection, all tweets with ground truth label “Sexual harassment” are treated as positive examples, while the tweets from the other 4 categories of misogyny and the nonmisogynous tweets are considered to be negative examples. The average of the F1-scores computed in this way for the 5 categories of misogyny is reported in the columns F1 (Category) in Table 3, while the average of the F1-scores for ‘Active’ and ‘Passive’ is reported in the columns F1 (Target) in Table 3. The first columns contain results obtained with 5-fold CV over the training data with 4,000 tweets from Table 1, while the last columns contain results for models trained on the entire training data of 4,000 tweets and subsequently applied to the test data. The latter correspond to the results on the competition scoreboard.
As a simple baseline model, we include an algorithm that labels every tweet as misogynous and subsequently assigns it to the most frequently occurring Category and Target from the training data, i.e. ‘Discredit’ and ‘Active’. This model has a very low precision, which explains why its F1scores are so low. The best results on the test data are obtained with himani.c.run3, which is an EoC approach using a BOW representation of extracted word unigrams, bigrams, and trigrams as features. This was the best performing model for Task B in the AMI@EVALITA competition.
Recall that himani.c.run3 uses a two step approach where tweets are initially labeled as misogynous or not (Step 1) and then assigned to a Category and Target (Step 2). Given that for the task in Step 1, the binary classifier of himani.c.run3 was outperformed in terms of accuracy by the binary classifier of resham.c.run3 (see Table 2), an obvious question is whether higher F1-scores for Task B could be obtained by combining the binary classifier for misogyny detection from resham.c.run3 with the EoC models for Category and Target identification from himani.c.run3. As the last row in Table 3 shows, this is not the case. To give more insight into where the differences in predictive performance in the last two rows of Table 3 stem from, Table 4 contains more detailed results about the precision, recall, and F1-scores for Task B: Target Identification on the train as well as the test data, while Table 5 and 6 contain corresponding confusion matrices on the test data. These tables reveal that the drop in F1-scores in the last row in Table 3 is due to a substantial drop in recall. As can be seen in Table 4, replacing the binary classifier in Step 1 by the method from resham.c.run3, causes the recall for ‘Active’ tweets in the test data to drop from 0.75 to 0.45, and for ‘Passive’ tweets from 0.61 to 0.19. The slight increase in precision is not sufficient to compensate for the loss in recall. As can be inferred from Table 5 and 6, the recall of misogynous tweets overall with himani.c.run3 is (301 + 60 + 15 + 36)=460 0:896 while with resham.c.run3 it is only (182 + 18 + 10 + 11)=460 0:480. 4
In this paper we presented machine learning models developed at UW Tacoma for detection of hate speech against women in English language tweets, and the results obtained with these models in the shared task for Automatic Misogyny Identification (AMI) at EVALITA2018. For the binary classification task of distinguishing between misogynous and non-misogynous tweets, we obtained our best results (2nd best team) with an Ensemble of Classifiers (EoC) approach trained on 3 kinds of features: bag of words, sentiment scores, and lexical features. For the multiclass classification tasks of Category and Target Identification, we obtained our best results (winning team) with an EoC approach trained on a bag of words representation containing unigrams, bigrams, and trigrams. All EoC models contain traditional machine learning classifiers, such as logistic regression and tree ensemble models.
Thus far, the success of our deep learning models has been modest. This could be due to the limited size of the dataset and/or the limited length of tweets. Regarding the latter, an interesting direction to explore next is training neural networks that can consume the tweets at character level instead of at word level, as we did in this paper.