Online misogyny has become a major cause of concern for Arab women who face gender-based online abuse regularly. Misogyny is a form of hate speech that denigrates a person or a group that identifies as feminine; it is generally described as hatred or contempt towards women. Arab women exposed to many forms of online misogyny, which sadly reinforces and justifies gender inequality, inferior social standing, sexual assault, violence, maltreatment, and underestimating. This paper proposes three methods to identify and classify misogyny behavior from Arabic tweets: (i) BERT, (ii) Ensemble-based model, and (iii) Dense Neural Network-based model. The suggested approach performs admirably on both tasks. The BERT model outperformed the other two suggested methods for misogyny identification, with an accuracy of 0.883, while the ensemble-based approach outperformed the other two suggested methods for misogyny behavior classification task, with an accuracy of 0.764.
Online social platforms like Facebook, Twitter, and Instagram are among the popular platforms
for spreading the news, sharing the achievement, and connecting with the worldwide community
[
On the internet, misogyny has evolved into a worldwide issue that has spread across several
social media platforms [
To identify misogyny and its behavior from the tweets, this paper proposes three diferent models: (i) Ensemble-based model (Support Vector Machine (SVM) + Logistic Regression (LR)), (ii) Dense Neural Network (DNN) model, and (iii) BERT (bert-base-arabic) [23]. The proposed models are validated with the dataset released in ArMI 2021 (a subtrack of HASOC FIRE2021)1. Two sub-tasks were shared by the organizer: (i) Misogyny Content Identification and (ii) Misogyny Behavior Identification. The dataset consists of 7,866 Misogyny tweets labeled into various categories like dominance, damning, harassment, and others. The dataset was developed by collecting the tweets from Twitter during January 2019-2021 and annotated into eight classes. The organizer has proposed two tasks [24] on this topic: (i) Task 1 is a binary classification task where tweets have categorized either misogyny or not, and (ii) Task 2 is a multi-class classification where tweets are labeled into eight categories- seven misogyny categories and the last one is from none misogyny category. The categories are (i) Damning (Damn): Tweets containing cursing content fall under this category, (ii) Derailing (Der): This category includes tweets that justify women’s violence or mistreatment, (iii) Discredit (Disc): Slurs and insulting words directed towards women can be found in tweets in this category. (iv) Dominance (Dom): Tweets in this category imply that men are superior to women, (v) Sexual Harassment (Harass): Sexual approaches and sexual nature abuse are discussed in tweets in this category, (vi) Stereotyping Objectification (Obj): Tweets in this category promote a stereotypical picture of women or describe their physical attractiveness, (vii) Threat of Violence (Vio): The content of tweets in this category is frightening, with threats of physical violence, (viii) None: if no misogynistic behaviors exist.
The rest of the paper is organized as follows: Section 2 discusses the proposed methodology in detail, the findings of the proposed system are listed in Section 3 and finally, the paper is concluded in Section 4.
The systematic diagram for the proposed model for misogyny comment and behavior identiifcation can be seen in Figure 1. Three diferent models were proposed: (i) Ensemble-based model (Support Vector Machine (SVM) + Logistic Regression (LR)), (ii) Dense Neural Network (DNN) model, and (iii) BERT (bert-base-arabic) [23]. The models were trained with the dataset published on the HASOC-ArMI 2021 track2. The overall statistic of the dataset can be seen in Table 1.
1http://fire.irsi.res.in/fire/2021/hasoc 2https://sites.google.com/view/armi2021/
t n eon tion t C ac y if ny it n g e iso Id M DNN
r o i eahv tion a B ic ny tif y n isog Ied M t n eon tion t C ac y if yn it n g e iso Id M
r o i eahv tion a B ic ny itf y n isog Ied M
Extensive experiments were carried out to determine the best-suited n-gram range for the ensemble-based model by varying the character n-gram range from one to six. Among diferent combinations of the n-gram range, we found that one to six character N-grams TF-IDF features performed best. Therefore, in the proposed ensemble-based model, one to six character N-grams TF-IDF features were used by SVM and LR classifiers to predict the probabilities for each of the output classes. The class-wise output probabilities of both the models were averaged. Finally, the higher average probability is used to decide the final class label. The overall flow diagram of the ensemble-based model can be seen in Figure 1.
The proposed dense neural network-based model uses one to six character N-grams TF-IDF features as an input to the network. The TF-IDF features are then passed through a four-layered dense network containing 4,096, 512, 64, and 2-neurons. As the performance of deep learning models is very sensitive to the chosen hyper-parameters, extensive experiments were performed by varying learning rate, batch size, dropout rate, optimizer, loss function, and the number of epochs. The best-suited hyper-parameters were found with a learning rate of 0.001, batch size of 20, the dropout rate of 0.2, Adam as the optimizer, binary cross-entropy for misogyny identification, and categorical cross-entropy for misogyny behavior classification. The model was trained for 50 epochs for the final prediction. The detailed flow diagram of the proposed dense neural network-based model can be seen in Figure 1.
The BERT (bert-base-arabic) pretrained on approx 8.2 Billion Arabic words [23]. The corpus and vocabulary set are not limited to Modern Standard Arabic; dialectical Arabic is included as well. To train BERT (bert-base-arabic), the pretraining method is similar to that of BERT, with the following exceptions: Instead of 1M training steps with a batch size of 256, 3M training steps with a batch size of 128 were trained. To fine-tune the BERT (bert-base-arabic) on our dataset, we set the maximum length of tweets to 30 because we observed that most tweets are less than 30 words in length. It indicates that tweets with less than 30 words were padded with zero, while tweets with more than 30 words were curtailed out. The experiment was performed with a learning rate of 2e-5, a batch size of 32, and it is trained for 50 epochs.
The performance of the proposed models is measured in terms of accuracy, macro precision, recall, and 1-score. The experimentation was first performed by taking 10% data samples from the provided training dataset. The results of diferent models with the validation dataset for misogyny identification are listed in Table 2. For the validation dataset, BERT (bert-base-arabic) model performed best with an accuracy of 0.88, macro precision, recall, and 1-score of 0.88 (as can be seen in Table 2). The results of the diferent models for misogyny behavior classification are listed in Table 3. Again, the proposed BERT(bert-base-arabic) model performed best with an accuracy of 0.77, macro precision of 0.55, recall of 0.51, and 1-score of 0.52, respectively.
In the HASOC-ArMI-2021 track, we had submitted three models (i) Ensemble-based model (SVM + LR), (ii) Dense Neural Network (DNN), and (iii) BERT (bert-base-arabic) for the final evaluation. The result of these models for misogyny identification and behavior classification are listed in Table 4. The submitted models performed significantly well for both tasks. The BERT (bert-base-arabic) performed best for misogyny identification among all the three submitted models with an accuracy of 0.883, macro precision of 0.878, recall of 0.876, and 1-score of 0.877. The ensemble-based model achieved an accuracy of 0.873, macro precision of 0.868, recall of 0.865, and 1-score of 0.866. The Dense neural network-based model achieved an accuracy of 0.854, macro precision of 0.846, recall of 0.850, and 1-score of 0.848.
For misogyny behavior classification, the proposed ensemble-based (SVM + LR) performed best among all the submitted models with an accuracy of 0.764, macro precision of 0.676, recall of 0.480, and 1-score of 0.531. The dense neural network-based model achieved an accuracy of 0.745, macro precision of 0.559, recall of 0.508, 1-score of 0.526. The BERT (bert-base-arabic) model achieved an accuracy of 0.780, macro precision of 0.549, recall of 0.503, and 1-score of 0.519 (as can be seen in Table 4).
Misogynistic language on social media sites such as Facebook and Twitter has been highlighted as a global issue that has increased over the last decade. In this paper, we proposed three diferent models such as BERT, ensemble-based model, and dense neural network-based model for the identification of misogyny and classification of misogyny behavior. We explored the role of character-level features and found that the use of character-level features from Arabic tweets shows promising performance in the misogyny identification and misogyny behavior classification from the tweets. The proposed fine-tuned BERT model performed best with an accuracy of 0.883 for the misogyny identification task whereas the proposed ensemble-based model form best with an accuracy of 0.764 for the misogyny behavior classification task. [17] M. Ferrier, N. Garud-Patkar, Trollbusters: Fighting online harassment of women journalists, in: Mediating Misogyny, Springer, 2018, pp. 311–332. [18] M. Amjad, N. Ashraf, A. Zhila, G. Sidorov, A. Zubiaga, A. Gelbukh, Threatening language detection and target identification in Urdu tweets, IEEE Access 9 (2021) 128302–128313. [19] M. Amjad, G. Sidorov, A. Zhila, A. F. Gelbukh, P. Rosso, Overview of the shared task on fake news detection in Urdu at FIRE 2020., in: FIRE (Working Notes), 2020, pp. 434–446. [20] H. Mulki, B. Ghanem, Let-mi: An arabic levantine twitter dataset for misogynistic language, in: Proceedings of the Sixth Arabic Natural Language Processing Workshop, 2021, pp. 154–163. [21] M. Amjad, N. Ashraf, G. Sidorov, A. Zhila, L. Chanona-Hernandez, A. Gelbukh, Automatic abusive language detection in Urdu tweets, ACTA POLYTECHNICA HUNGARICA (2021). [22] M. Amjad, A. Zhila, G. Sidorov, A. Labunets, S. Butt, H. I. Amjad, O. Vitman, A. Gelbukh, UrduThreat@ FIRE2021: Shared track on abusive threat identification in Urdu, in: FIRE 2021: Forum for Information Retrieval Evaluation, December 13-17, 2021, India, ACM, 2021. [23] A. Safaya, M. Abdullatif, D. Yuret, KUISAIL at SemEval-2020 task 12: BERT-CNN for ofensive speech identification in social media, in: Proceedings of the Fourteenth Workshop on Semantic Evaluation, International Committee for Computational Linguistics, Barcelona (online), 2020, pp. 2054–2059. URL: https://www.aclweb.org/anthology/2020.semeval-1.271. [24] H. Mulki, B. Ghanem, ArMI at FIRE2021: Overview of the First Shared Task on Arabic Misogyny Identification, in: Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, CEUR, 2021.