In this paper, we describe our eforts on the shared task of Arabic Misogyny Identification (ArMI) [ 1]. We tackled the Misogyny Content Identification subtask (Subtask-1). Our experiments were based on preprocessing the given data, then fine-tuning pretrained MARBERT language model on the Misogyny Identification downstream task. Experimental results performed only on Subtask-1, show that keeping emojis in text can influence the model.
preprocessing strategy.
The rest of the paper is organized as follows: in Section 2 we introduce ArMI dataset. In Section 3, we describe our approach in tackling the problem. Then, in Section 4 we provide and discuss the results of the proposed method on Subtask-1. Section 5 concludes our work throughout this shared task.
The released train dataset [
• None: if no misogynistic behaviors exist.
This section describes the various data preparation procedures and models utilized in the experiments.
To prepare the dataset for preprocessing, we completed 4 main stages as follows:
• Cleaning: we removed all of the diacritics such as (damma,tashdid, fatha, kasra, etc.),
English words and numbers, English and Arabic punctuations, URLS and USER mention
tokens.
• Elongation removal: any repeated character for more than twice was removed. For
example, the word “دددديكا ” becomes “ديكا” after the preprocessing.
• Letter normalisation: Arabic characters that appeared in a variety of forms were
combined into a single form. For example, letter like آ , إ, and أ are replaced with a ا
• Extract hashtag keywords: To extract intelligible key phrases, we deleted the hash
symbol ”#” and replaced the underscore ”_” within a hashtag with a white space. For
exemple ” دمحأ_ اي_ لغتشتب# ” becomes ” دمحأ اي لغتشتب.
3.2. Model
In a complex NLP task like misogyny identification, we need context-aware embedding tools.
BERT [
Recently, some works focused on Arabic language and dialects. We cite AraBERT [
After preprocessing the ArMI dataset as described above, we split the dataset into 90% for training and 10% for validation. We have trained MARBERT on the 90% split with a batch size of 32 and a sequence length of 128 for 5 epochs.
In this section, we present and discuss the results of our experiments on Subtask-1.
The evaluation metric used to test our system on Subtask-1 is the accuracy. This metric was specified by the competition organizers. Table 2 lists the results of the classifier built by finetuning MARBERT on the training set and tested on the validation set.
Table 3 lists the results performed by our model on the test set as provided by the competition organizers. We observe a large drop in model performance on the test set, which is most likely
We performed extra error analysis for our proposed model results. This analysis aims to find where the model failed to correctly categorize the tweets and tries to discover the causes of this misclassification.
We examine a sample of 50 random misclassified tweets. We discovered various reasons why sarcastic tweets are classified as not misogyny and vice versa. These reasons are summarized as follows: • Human annotation is not perfect since the diversity of annotators’ cultures and backgrounds may not be taken into account throughout the annotation process. • The absence of context: In some tweets, the context is missing, making it impossible for our algorithm to grasp the context and accurately forecast the label. Actually, Arabs use some highly ofensive words not with the intention to spread hate but by sarcasm. • Emojis are not processed: since we left the emojis in tweets without any kind of preprocessing, we discovered that some emojis had an efect on the categorization process.
Actually, 14% of the tweets contain at least one emoji.
To identify misogyny on Dialectal Arabic tweets we first proposed a preprocessing strategy. Secondly, we built a classification model based on MARABERT language model which was selected for the final submission. We observed that considering emojis in the preprocessing step is crucial to the classification performance.