This research paper discusses topics related to specific content that may be sensitive or offensive, which some readers may find distressing. The intent is to analyze and understand the research work.
The commence and growth of internet have a profound impact on our social structure, the way we communicate, our relationships through the development of various social platforms. Twitter, one of such social platforms, has developed into a lively forum for sharing idea and discourse because of its succinct format and emphasis on real-time updates, with attractive features like hashtags, tags, etc. While such advancements of social platforms promotes connectivity and facilitates the spread of information it also tempts people to gain fame thorough views and likes, and throw inappropriate contents and comments in disguise of freedom of speech lacking empathy towards race, gender, religion [1]. Evidently sexism exists in Twitter in the form of sexist tweets sometimes intentionally while at other times unintentionally. These tweets and contents can range from blatant objectification and insults to more implicit bias and prejudice. Sexist tweets can create a hostile environment online, especially for women and other targeted groups. Identifying sexism in online platforms is crucial for a various number of reasons. Recognizing sexist content is essential to advancing equality and averting social harm. The propagation of detrimental stereotypes and biases through such information contributes to gender inequality and has a detrimental effect on people's mental health and self-esteem [2]. We can promote an inclusive and respectful culture by addressing sexist content, making sure that everyone feels respected and secure.
The sheer volume of such sexist tweets makes it difficult to identify sexism manually, requiring the immediate need for automated solutions. Natural Language Processing (NLP) can emerge as a successful tool in recognizing such harmful contents and filtering them out [3]. It can prevent people to post contents that goes against community standards creating a safer and more acceptable platform. Using natural language processing (NLP) techniques, we can create automated systems that can recognize sexist tweets with accuracy. Text content can be analyzed by these technologies, which can also spot bias and discriminatory language patterns and flag tweets for moderation or additional review.
EXIST aims to capture sexism in a broad sense, from explicit misogyny to other subtle expressions that involve implicit sexist behaviours. EXIST is a series of scientific events and shared tasks on sexism identification in social networks [4], [5]. The EXIST 2024 Tweets Dataset contains more than 10,000 labeled tweets, both in English and Spanish. Based on the labeled data, the tasks were to identify sexist tweets among them, the underlying intention of the author, and if it contained sexism at multiple degrees. The challenge lies in the nuanced and context-dependent nature of language on social media. Tweets often use slang, sarcasm, or coded language that can obscure sexist intent, while URLs can lead to external content that may contain sexist material not immediately evident in the tweet itself. Mentions and hashtags can complicate analysis by linking tweets to broader conversations or by being used to target specific individuals. Emojis add another layer of complexity, as their meanings can vary widely depending on context and cultural interpretation. These factors make automated detection systems prone to errors, requiring sophisticated algorithms and often human oversight to accurately identify and address sexism in tweets.
Our approach involved training different Machine Learning (ML) models to capture the sexist pattern in the contents and choose the one that performs the best on the validation dataset. The machine learning models included TF-IDF+SVM [6], TF-IDF+XGB [7], TF-IDF+RF [8], LSTM [9], LSTM+Attention [10], XLM-Roberta [11], mBert [12], BETO [13]. We yielded results for different ways of preprocessing the data and further finetuned the models based on best results. Additionally, we attempted to find out the provoking words that contributed in the semantic meaning of sexism reflected through the attention weights of those words. Taking into accounts the attention weights of the sentence representation helped the same model to yield the best result for Task 1, Task 2 and Task 3. The results show that the Bert based models are well trained to capture the pattern of sexism when considering the attention layer. The performance metrics used were accuracy and F1 score and we could obtain an accuracy of 84.80%, 72.06% and 88.25% and F1 score of 84.76%, 51.43% and 54.77% for Task 1, Task 2 and Task 3, respectively. To do better modeling and include different explainability results, we adopt different training and explainable experiments from EDAL, ITPT, HateXplain [14,15,16] papers. In the remainder of this paper, we present the Problem Description in Section 2, Methodology in Section 3, Experimental Setup in Section 4, Result Analysis in Section 5, Error Analysis in Section 6 and Conclusion in Section 7.
In this work, we have addressed tasks 1, 2, and 3. The task descriptions are listed as per the guidelines.
1. Task 1 (Sexism Identification in Tweets): Given a tweet, this subtask aimed to classify whether a tweet contains sexist expressions or behaviors. The tweet can be sexist itself, describe a sexist situation or critic sexist behavior. It is a binary classification problem. We needed to label 'YES' or 'NO' to the tweets.
2. Task 2 (Source Intention in Tweets): This subtask aims to classify each tweet according to the intention of the person who wrote it. This is a multi-class classification problem. The classes are as follows:
• DIRECT: The intention is to write a message that is sexist itself.
• REPORTED: The author intends to report or describe a sexist situation or event suffered by a woman or women in the first or third person. • JUDGEMENTAL: The author intends to be judgemental since the tweet describes sexist situations or behaviors to condemn them. • NO: The tweet is detected as not sexist in subtask 1.
3. Task 3: This subtask categorizes the tweets according to the type of sexism. This is a multi-label classification problem with 5 labels. So more than one class can be assigned to each tweet. The labels are as follows:
• IDEOLOGICAL-INEQUALITY: The tweet discredits the feminist movement, rejects inequality between men and women, or presents men as victims of gender-based oppression. • STEREOTYPING-DOMINANCE: The tweet expresses false ideas about women that suggest they are more suitable to fulfill certain roles (mother, wife, family caregiver, faithful, tender, loving, submissive, etc.), or inappropriate for certain tasks (driving, hard work, etc.), or claims that men are somehow superior to women. • OBJECTIFICATION: The tweet presents women as objects apart from their dignity and personal aspects or assumes or describes certain physical qualities that women must have to fulfill traditional gender roles (compliance with beauty standards, hypersexualization of female attributes, women's bodies at the disposal of men, etc.). • SEXUAL-VIOLENCE: The tweet includes or describes sexual suggestions, requests for sexual favors, or harassment of a sexual nature (rape or sexual assault). • MISOGYNY-NON-SEXUAL-VIOLENCE: The tweet expresses hatred and violence towards women, different from that with sexual connotations. • NO: When none of the 5 labels are assigned to the tweet.
The dataset includes over 10,000 tweets both in Spanish and English. The train, dev, and test sets contain 6064, 934, and 2076 tweets respectively. These numbers are after discarding the non-labeled samples in the gold standard. The distribution between both languages has been somewhat balanced to tackle the issue of biases. The exact figures are in Table 1. Task 1 is a binary classification problem with 'YES' and 'NO' labels. Task 2 is a multi-class classification problem with 3 sexism sources and one 'NO' label. Task 3 is a multi-label classification problem with 5 different labels and a 'NO' label when none of the labels are assigned. The distribution is not balanced. Each tweet is labeled by six different annotators. The gold label is the average of the labels. The exact figures of the distribution is in Table 2. They are then passed to the model which gives the encoded format of those tokens. Encoded tokens except CLS are passed to the attention module whereas CLS is passed to the MLP layer which then is aggregated using various methods and then this representation is used for classification. The logits are yield by the classifier.
In this section, we will outline our methodology. Given the multilingual nature of the dataset encompassing both English and Spanish texts, we employ a multilingual pretrained model. We further refine this model through fine-tuning on the dataset. Our model architecture comprises five key components: i) Pretrained model backbone, ii) Sentence Representation from CLS token, iii) Attention-based Context Vector, iv) Feature Aggregation Module, and v) Classifier Head
An input sentence 𝑆 is passed into the pretrained multilingual tokenizer to obtain the tokens of the sentence 𝑆 = {𝑡 [𝐶𝐿𝑆] , 𝑡
To get the representation for the entire sentence, we use the representation of [CLS] token ℎ [𝐶𝐿𝑆] . This representation is passed into a Single Layer Perception to get the enhanced representation.
As the tasks are natural language understanding tasks where individual words hold distinct predictive significance, we integrate an attention-based network to ascertain word importance. By accounting for the significance of each word, we construct a context vector for the sentence. We only consider the representations of the words and exclude the representations of the special tokens.
Once contextual representations 𝐻 are obtained for a sentence 𝑆, an additional attention layer is added to compute learnable attention scores 𝛼 𝑖 for each token 𝑡 𝑖 in 𝐻, and its calculation is as follows:
This results in a set of attention_scores = {𝛼 1 , 𝛼 2 , . . . , 𝛼 𝑛 } corresponding to the tokens in sentence 𝑆. These attention scores collectively represent the overall attention distribution across the sentence, indicating the relative importance or relevance of each token to the context of the entire sentence. After finding attention scores for each token, we find the context vector for the sentence 𝑆 by multiplying the contextual representations of token 𝑡 𝑖 with its attention score 𝛼 𝑖 .
In this section, we consolidate various representations of the sentence 𝑆. We obtain two distinct representations: one based on the CLS token ℎ
After finding the aggregated feature representation 𝑧, it is fed into a classification layer. The representation is the logits 𝑧 is employed for the classification process by the following:
Finally, we calculate the Cross-Entropy (CE) loss based on 𝑧 ′ with the ground truth.
For the tasks, our initial approach involved a thorough exploration of various model types to determine the most appropriate one. Our investigation led us to examine three distinct categories: Machine Learning (ML) Models, Deep Learning Models, and Transformer-based Pretrained Models. For machine learning models, we considered Support Vector Machine (SVM), Random Forest (RF), and XGBoost.
Employing TF-IDF as our feature extractor, we processed each sentence through this mechanism before inputting it into the ML models for classification. For task 3, we consider Logistic Regression instead of XGBoost.
In our deep learning methodologies, we leverage both LSTM [17] and LSTM + Attention models. As for transformer-based approaches, we've explored XLM-RoBERTa [18], mBERT [19], and BETO [20]. The outcomes are detailed in Table 3 for the development set. These model selections were driven by the presence of two distinct languages within the dataset. The total unique words for TF-IDF experiments were 39613 after deleting the punctuations, auxiliaries, and spaces. For LSTM experiments, we have used embedding dim = 50, hidden units = 64 of the LSTM layer, and output dim = 256 of a fully connected layer with a learning rate of 0.001. The experiments were done in 20 epochs. For the transformer-based models, we fine-tuned the pre-trained models. We use 10 epochs with a batch size of 32 Table 3 presents the results of different models for Task 1 and Task 2 datasets, while Table 4 displays the model performance specifically for Task 3 on the development set. Analysis of Table 3 reveals that classical ML models exhibit competitiveness against deep learning counterparts. Notably, employing TF+IDF with XGBoost yields superior results to deep learning models. Incorporating the attention module leads to a notable 3% enhancement in accuracy for Task 1 and 1% improvement Task 2. Additionally, fine-tuning pretrained models demonstrates substantial improvements ranging from 6-9%. Among these, XLM-Roberta demonstrates optimal performance for both Task 1 and Task 2, thereby being selected as the final model for further experimentation. Turning to Task 3, as indicated in Table 4, mBERT marginally outperforms XLM-RoBERTa. Consequently, mBERT is chosen as the final model for Task 3.
The datasets retrieved from Twitter contain additional information such as usernames and URLs alongside the original posts and comments. To gauge their impact, we conducted experiments using various preprocessing methods. These techniques included removing usernames, URLs, punctuation, and emojis. The outcomes of these experiments are presented in Table 5. From the table, we can see that removing url from the tweets improves the model performance for task 1 and task 3. For task 2, no preprocessing is helpful. those results, we set a learning rate of 2e-5, random seed = 42 and batch size 32 = for tasks 1 & 3 and 16 for task 3 for our model. We use AdamW optimizer in our experiments with betas = (0.9, 0.99). All experiments were conducted using Python (version 3.12) and PyTorch, leveraging the free NVIDIA Tesla P100 GPU provided by Kaggle. For the transformer based models we consider Huggingface transformers library. All the transformer based models were run for 10 epochs.
When assessing the effectiveness of the models, we consider different performance metrics. Mainly we focus on Accuracy, Macro-F1, and ICM for our performance as our evaluation metircs.
Accuracy measures the proportion of correctly classified instances among all instances. It is calculated by dividing the sum of true positives (correctly predicted positive instances) and true negatives (correctly predicted negative instances) by the total number of instances.
𝑇 𝑃 + 𝑇 𝑁 𝑇 𝑃 + 𝑇 𝑁 + 𝐹 𝑃 + 𝐹 𝑁
The F1 score is the harmonic mean of precision and recall. In the macro F1 score, each class is given equal weight, and the mean of these F1 scores across all classes is calculated. Macro F1 Score is calculated as follows:
The ICM metric functions as a similarity measure employed to assess the likeness between modelgenerated outputs and the actual ground truth in classification tasks. It does so by comaring the Information Content of catrgories presented through its features with the help of a Similarity function.
It extends the principles of Pointwise Mutual Information, a common metric used for evaluating relationships between words. When all parameters in the ICM formula are set to 1, it becomes equivalent to PMI. This means ICM can capture similar relationships as PMI but with more flexibility. ICM helps evaluate how well a system's output aligns with the ground truth by comparing the information content of the categories they represent. (ICM) is adopted for all tasks and evaluation types (hard-hard, hard-soft, soft-soft). ICM-soft is an extenstion of ICM for evaluating hierarchical multi-label classification tasks where there might be disagreements in the ground truth. It can can handle both soft system outputs and soft ground truth.
Table 9 exhibits the results of various sentence representation extractors employing different aggregation techniques across three tasks on the validation dataset. Regarding task 1, employing the CLS tokenbased representation yields an accuracy of 84.90% and an F1 score of 83.90%. Transitioning to context vector representation through an attention feature extractor results in a slight increase in the F1 score. Furthermore, combining both types of representation through addition-based aggregation shows a 2% enhancement in performance across the board. If we combine them using concat based aggregation techniques we see a small improvement in F1 score but not like the addition based aggregation one For task 2, we experience similar trend in the performance where attention based sentence representation performs better than CLS token based sentence extractor. If we combine both CLS and context vector getting from attention through addition based aggregation techniques, 1% improvement in accuracy and 2% improvement in F1 score. While we consider concat based aggregation techniques for combining both sentence level representation, the performance is further improved by 1.2% in accuracy and around 3% improvement in F1 score. For task 3, we also get better result than the CLS based baseline while we aggregate both sentence representations using concatenation. The top-performing model is chosen to generate predictions for the test dataset across all three tasks. The performance results on the test dataset are detailed in Table 10, showcasing the ICM-Hard, ICM-Hard Norm, and Macro-F1 scores as our performance metrics. Our best-performing model achieves an F1 score of 75.01 for all tweets, with 72.09 and 77.33 F1 scores recorded for English and Spanish tweets, respectively. For task 2, the score is 48.56, while task 3 reaches 43.79 across all tweet data. The table shows that our model can predict Spanish tweets more effectively than English tweets.
The evaluation of our approach for Task 1 is done on the development dataset. The confusion matrix shown in 2(a) represents the performance of the classification. The number of True Positive is the number of correctly predicted positive cases. The model correctly predicted 388 cases as "Yes" (True Yes). True Negative cases is the number of correctly predicted negative cases. The model correctly predicted 418 cases as "NO" (True No). False Positive is the number of incorrectly predicted positive cases. The model incorrectly predicted 61 cases as "YES" when they were actually "NO" (False YES). False Negative (FN) is the number of incorrectly predicted negative cases. The model incorrectly predicted 67 cases as "NO" when they were actually "YES" (False NO). The model produced 806 correct classification as opposed to 128 misclassification.
Similarly, the confusion matrix is shown in 2(b) shows the result of the evaluation done on the development dataset for Task 2. As shown in the figure, out of 479 true NO labels, 420 cases were correctly predicted as NO, 150 were correctly predicted as DIRECT out of 204 true DIRECT cases, 36 were correctly predicted as REPORTED out of 65 true REPORTED cases and 18 were correctly predicted as JUDGEMENTAL out of 83 true JUDGEMENTAL cases. The model produced 624 correct classification as opposed to 217 misclassification.
We have calculated the Layer Integrated Gradient attributions for few specific input tweets using Captum [21]. The attributions explain how each input element of a tweet contributes to the model's prediction for the target class. It means that the attention provided by the model to each tokens. So we name it attention heatmap. The darker the color in the cell of a token, the higher its attribution value and the higher its contribution to the target. We have considered both English and Spanish tweet for the experiment and creating the heatmap. The corresponding labels are also listed in the Figure 3. We can infer that for 'NO' label the most attentive tokens among the 4 tweets are last, abuse, ada, in and for 'YES' label the most attentive tokens among the 4 tweets are economy, s, a, LAS. This experiment was done considering the best performing model in the task 1.
We also extracted the most attentive tokens for each category in the validation dataset for further analysis. We determine these tokens by extracting the highest average self-attention scores for each token across the entire category-wise validation dataset. Transformer-based models compute attention scores to gauge the relationship between 𝑤𝑜𝑟𝑑 𝑖 and 𝑤𝑜𝑟𝑑 𝑗 across various layers and heads. Considering a transformer-based model with 𝐿 layers and 𝐻 heads per layer, and a sequence length of 𝑛 for a given sentence 𝑆, the resulting attention matrix has dimensions of 𝐿 × 𝐻 × 𝑛 × 𝑛. To compute the average attention score for token 𝑖 within sentence 𝑆, we follow this calculation:
To find category-wise average attention score for token 𝑖 on the validation dataset, we take the average of Avg_Attn_Score 𝑖 𝑆 for those sentences where token 𝑖 appears in. After calculating the token attention scores, we arrange them in descending order and select the top K words. In Table 11, we present the category-wise most attentive tokens predicted by the top-performing model for both Task 1 and Task 2.
List of most attentive tokens for tasks 1 and 2 for each class with the best performing configuration for each task. The table presents a detailed breakdown of the most attentive tokens categorized by the topperforming model across two distinct tasks. In Task 1, where the classification pertains to identifying sexist content, the model identifies tokens such as "volant," "slut," and "cum" as prominent indicators for non-sexist content, while tokens like "penis, " "forever, " and "gangbang" are highlighted for identifying sexist content. Task 2, which involves various categories like "NO," "DIRECT," "REPORTED," and "JUDGEMENTAL, " exhibits a diverse range of most attentive tokens. For instance, in the "NO" category, tokens like "pun," "wall," and "question" stand out, while "DIRECT" category tokens include "where," "GPS, " and "tomorrow. " Additionally, tokens such as "wenr, " "school, " and "saben" are emphasized in the "REPORTED" and "JUDGEMENTAL" categories, respectively. This comprehensive analysis sheds light on the model's attention mechanism and its discernment of different types of content within each task.
The EXIST challenge is designed to promote research on automated sexism detection and modeling in online environments, with a particular focus on Twitter. In this paper, we performed extensive research and conducted thorough experiments to achieve this objective, employing advanced techniques in natural language understanding and machine learning. Specifically, we enhanced existing machine learning models by incorporating an attention layer and a CLS token, which emphasize the words that contribute to the context of sexism. Our findings demonstrate the effectiveness of our approach and models on both the training and validation datasets.
However, as the training is done on Spanish and English language the model might not be proficient at identifying sexist content in other languages. This can lead to misclassification of sexist Tweets.
This project has been sponsored by Penta Global Limited Bangladesh. We would like to express our deepest gratitude to Penta Global for for their financial support.