For this study, we used the Doc2Vec embedding approach for feature extraction, with the context window size of 2, minimum word frequency of 2, sampling rate of 0.001, learning rate of 0.025, minimum learning rate of 1.0E-4, 200 layers, batch size of 10000 and 40 epochs. Distributed Memory (DM) is used as the embedding learning algorithm with the negative sampling rate of 5.0. Before feature extraction, all the tweets were pre-processed by converting the characters to their lower case, removing stop words, numbers, punctuations and words that contain no more than 3 characters as well as stemming all the kept words by Snowball Stemmer. Additionally, three classi ers are trained by using SVM with a linear kernel, random forests (RF) and gradient boosted trees (GBT). In the testing stage, the same way of text pre-processing and feature extraction is applied to test instances separately, and each pair of two out of the three trained classi ers (SVM+RF, SVM+GBT and RF+GBT) are fused by combining the probabilities for each class by averaging.
Social media platforms have provided users the ability to freely express
themselves, however, it has also resulted to increase in cyberhate such as bullying,
threats and abuse. A study has shown how 67% of teenagers between the ages
of 15 to 18-year olds have been exposed to hate materials on social media, with
21% becoming victims of such materials [
Misogyny is de ned as a particular type of hate speech that is targeted
towards women [
Therefore, due to increasing evidence showing that cyberhate is increasingly becoming a threat to the society, it has become necessary to implement techniques that can be automated to classify cyberhate so as to reduce the burden on those responsible for public safety. Hence, the aim of this study is to: 1. Identify and distinguish between misogynous and non-misogynous contents; 2. Classify misogynistic behaviour into several behavioural types; and 3. Identify if the target of the misogynistic behaviour is active or passive. 2
The experiment was carried out using misogynous text collected from Twitter [
Due to the noisy nature of social media data [
Figure 1 shows the experimental processes for this study, while subsequent sections provide a detailed description of the experiment with the results of the classi cation. set contain only English language text with a total of 3, 977 instances as shown in Table 1, which also contains a brief description of the features. Table 2 shows a detailed description of the training set labels with the number of instances for each category.
Dataset
Training
Test
Total
The training and test datasets were separately pre-processed and ltered using
standard text pre-processing features. User names, URLs and non-ascii
characters were removed. The tweets were converted to lower case, and the stop words,
numbers and punctuation characters were ltered out. Words that contain less
than 3 characters were removed and all the remaining words were stemmed using
the Snowball Stemmer in Knime [
In addition to the standard pre-processing, features containing these four labels: Id, Misogynous, Misogynous-category and Target were extracted individually using the Doc2Vec Learner. The learner had a context window size and 4 minimum word frequency of 2, while the sampling rate of 0.001, learning rate of 0.025, minimum learning rate of 1.0E-4, 200 layers, batch size of 10000 and 40 epochs. Also, the Distributed Memory (DM) is used as the embedding learning algorithm with the negative sampling rate of 5.0. The extracted labels were exported as tables for classi cation. 2.3
Classi cation The pre-processed training set was used to train ve classi ers using the following machine learning algorithms: SVM with a linear kernel, random forests (RF), gradient boosted trees (GBT), Decision Tree (DT) and Nave Bayes (NB). The 10-fold cross validation approach was used for evaluation and the results of this experiment was used to determine the algorithms that had the best performance among the ve machine learning algorithms used.
To obtain the labels for the test set, the highest three performing trained classi ers (determined as described in the previous paragraph) were used: SVM with a linear kernal, random forest (RF) and gradient boosted trees (GBT). These classi ers were paired and each pair of two out of the three trained classi ers, i.e. SVM+RF, SVM+GBT and RF+GBT, were fused using algebraic fusion to combine the probabilities for each class by averaging them. This experiment was executed three times. 3
In this section, the results of the experiment for the three runs will be discussed
and compared to the results published in [
In this study, text containing both misogynous and non-misogynous contents were extracted from Twitter. The extracted training set was pre-processed and used to train three machine classi ers. With this text, we were able to achieve an accuracy of 0.624 (misogyny identi cation), 0.247 (misogynistic behaviour classi cation) and 0.623 (misogynistic target classi cation). On the test set, the performance was lower, which we believe is due to the incompatibility between the features extracted from the training set and the ones extracted from the test set - we will further investigate this issue. Additionally, we strongly believe it is empirical to further improve the identi cation and classi cation performance for misogynous contents on social media, speci cally Twitter, as this can potentially safe lives.