Social media platforms contains a vast user base and ofers ease with which information can be shared. This can adversely facilitate the spread of sexist content which is infeasible for human monitoring and filtering. This paper investigates the automated detection of sexism in tweets using Natural Language Processing (NLP) techniques. Sexist tweets can create a hostile online environment and perpetuate harmful stereotypes. Manual identification is impractical due to the vast amount of data. The research proposes a system utilizing machine learning models to analyze text, identify bias and discriminatory language patterns, and flag tweets for moderation. The fourth edition of EXIST shared task 2024 Tweets Dataset, containing labeled English and Spanish tweets, is used to train and evaluate the models. The system explores various approaches, including TF-IDF with diferent classifiers (SVM, XGB, RF), Long Short-Term Memory (LSTM) networks with and without attention mechanisms, and pre-trained transformer models (XLM-Roberta, mBERT, BETO). The efectiveness of diferent preprocessing techniques and the role of attention weights in identifying sexism are also explored. The paper outlines the methodology, experimental setup, and analysis of results, paving the way for further discussion on error analysis and conclusions in subsequent sections.
This research paper discusses topics related to specific content that may be sensitive or ofensive, which some readers may find distressing. The intent is to analyze and understand the research work.
The commence and growth of internet have a profound impact on our social structure, the way we
communicate, our relationships through the development of various social platforms. Twitter, one
of such social platforms, has developed into a lively forum for sharing idea and discourse because of
its succinct format and emphasis on real-time updates, with attractive features like hashtags, tags,
etc. While such advancements of social platforms promotes connectivity and facilitates the spread of
information it also tempts people to gain fame thorough views and likes, and throw inappropriate
contents and comments in disguise of freedom of speech lacking empathy towards race, gender, religion
[
The sheer volume of such sexist tweets makes it dificult to identify sexism manually, requiring
the immediate need for automated solutions. Natural Language Processing (NLP) can emerge as a
successful tool in recognizing such harmful contents and filtering them out [
EXIST aims to capture sexism in a broad sense, from explicit misogyny to other subtle expressions
that involve implicit sexist behaviours. EXIST is a series of scientific events and shared tasks on sexism
identification in social networks [
Our approach involved training diferent Machine Learning (ML) models to capture the sexist pattern
in the contents and choose the one that performs the best on the validation dataset. The machine learning
models included TF-IDF+SVM [
In this work, we have addressed tasks 1, 2, and 3. The task descriptions are listed as per the guidelines. 1. Task 1 (Sexism Identification in Tweets): Given a tweet, this subtask aimed to classify whether a tweet contains sexist expressions or behaviors. The tweet can be sexist itself, describe a sexist situation or critic sexist behavior. It is a binary classification problem. We needed to label ’YES’ or ’NO’ to the tweets. 2. Task 2 (Source Intention in Tweets): This subtask aims to classify each tweet according to the intention of the person who wrote it. This is a multi-class classification problem. The classes are as follows: • DIRECT: The intention is to write a message that is sexist itself. • REPORTED: The author intends to report or describe a sexist situation or event sufered by a woman or women in the first or third person. • JUDGEMENTAL: The author intends to be judgemental since the tweet describes sexist situations or behaviors to condemn them.
• NO: The tweet is detected as not sexist in subtask 1. 3. Task 3: This subtask categorizes the tweets according to the type of sexism. This is a multi-label classification problem with 5 labels. So more than one class can be assigned to each tweet. The labels are as follows: • IDEOLOGICAL-INEQUALITY: The tweet discredits the feminist movement, rejects inequality between men and women, or presents men as victims of gender-based oppression. • STEREOTYPING-DOMINANCE: The tweet expresses false ideas about women that suggest they are more suitable to fulfill certain roles (mother, wife, family caregiver, faithful, tender, loving, submissive, etc.), or inappropriate for certain tasks (driving, hard work, etc.), or claims that men are somehow superior to women. • OBJECTIFICATION: The tweet presents women as objects apart from their dignity and personal aspects or assumes or describes certain physical qualities that women must have to fulfill traditional gender roles (compliance with beauty standards, hypersexualization of female attributes, women’s bodies at the disposal of men, etc.). • SEXUAL-VIOLENCE: The tweet includes or describes sexual suggestions, requests for sexual favors, or harassment of a sexual nature (rape or sexual assault). • MISOGYNY-NON-SEXUAL-VIOLENCE: The tweet expresses hatred and violence towards women, diferent from that with sexual connotations.
• NO: When none of the 5 labels are assigned to the tweet.
The dataset includes over 10,000 tweets both in Spanish and English. The train, dev, and test sets contain 6064, 934, and 2076 tweets respectively. These numbers are after discarding the non-labeled samples in the gold standard. The distribution between both languages has been somewhat balanced to tackle the issue of biases. The exact figures are in Table 1.
Task 1 is a binary classification problem with ’YES’ and ’NO’ labels. Task 2 is a multi-class classification problem with 3 sexism sources and one ’NO’ label. Task 3 is a multi-label classification problem with 5 diferent labels and a ’NO’ label when none of the labels are assigned. The distribution is not balanced. Each tweet is labeled by six diferent annotators. The gold label is the average of the labels. The exact ifgures of the distribution is in Table 2. NO DIRECT REPORTED JUDGEMENTAL IDEOLOGICAL-INEQUALITY STEREOTYPING-DOMINANCE OBJECTIFICATION SEXUAL-VIOLENCE MISOGYNY-NON-SEXUAL-VIOLENCE NO l e d o d ien laM a u -ter ign rP illt u M
In this section, we will outline our methodology. Given the multilingual nature of the dataset encompassing both English and Spanish texts, we employ a multilingual pretrained model. We further refine this model through fine-tuning on the dataset. Our model architecture comprises five key components: i) Pretrained model backbone, ii) Sentence Representation from CLS token, iii) Attention-based Context Vector, iv) Feature Aggregation Module, and v) Classifier Head
An input sentence is passed into the pretrained multilingual tokenizer to obtain the tokens of the sentence = {[], 1, 2, . . . , , [ ]}, where represents the -th token and [] & [ ] are the special tokens. The tokens are passed into the pretrained multilingual model to achieve contextual representations for each token , denoted as = {ℎ[], ℎ1, ℎ2, . . . , ℎ, ℎ[ ]}, where ℎ represents the contextual representation of token . Specifically, we extract the last layer hidden representations of the pretrained model, which are further fine-tuned during the dataset training.
To get the representation for the entire sentence, we use the representation of [CLS] token ℎ[]. This representation is passed into a Single Layer Perception to get the enhanced representation.
′ ℎ = · ℎCLS +
As the tasks are natural language understanding tasks where individual words hold distinct predictive significance, we integrate an attention-based network to ascertain word importance. By accounting for the significance of each word, we construct a context vector for the sentence. We only consider the representations of the words and exclude the representations of the special tokens.
Once contextual representations are obtained for a sentence , an additional attention layer is added to compute learnable attention scores for each token in , and its calculation is as follows: = softmax( · ℎ + ),
= 1, 2, . . . ,
This results in a set of attention_scores = { 1, 2, . . . , } corresponding to the tokens in sentence . These attention scores collectively represent the overall attention distribution across the sentence, indicating the relative importance or relevance of each token to the context of the entire sentence. After ifnding attention scores for each token, we find the context vector for the sentence by multiplying the contextual representations of token with its attention score .
= ∑︁ · ℎ
=1
In this section, we consolidate various representations of the sentence . We obtain two distinct ′ representations: one based on the CLS token ℎ and the other based on attention-based context vector . The aggregation of these representations is performed using two distinct techniques considering the two following techniques: • Concat-based Aggregation: For concat-based aggregation, we just simply concatenate ℎ′ and to get the aggregated representation as follows:
_concat = concat[ℎ′ , ] Then _concat is passed into single layer MLP layer to get the final combined representation where = MLP(_concat) • Element wise Addition based Aggregation: In this aggregation, we combine ℎ′ and by summing them element wise as follows
= ℎ′ + Then _suumed is passed into single layer MLP layer to get the final combined representation where = MLP(_suumed)
After finding the aggregated feature representation , it is fed into a classification layer. The representation is the logits is employed for the classification process by the following:
′ = · +
Finally, we calculate the Cross-Entropy (CE) loss based on ′ with the ground truth.
For the tasks, our initial approach involved a thorough exploration of various model types to determine the most appropriate one. Our investigation led us to examine three distinct categories: Machine Learning (ML) Models, Deep Learning Models, and Transformer-based Pretrained Models. For machine learning models, we considered Support Vector Machine (SVM), Random Forest (RF), and XGBoost. Employing TF-IDF as our feature extractor, we processed each sentence through this mechanism before inputting it into the ML models for classification. For task 3, we consider Logistic Regression instead of XGBoost.
In our deep learning methodologies, we leverage both LSTM [
Table 3 presents the results of diferent models for Task 1 and Task 2 datasets, while Table 4 displays the model performance specifically for Task 3 on the development set. Analysis of Table 3 reveals that classical ML models exhibit competitiveness against deep learning counterparts. Notably, employing TF+IDF with XGBoost yields superior results to deep learning models. Incorporating the attention module leads to a notable 3% enhancement in accuracy for Task 1 and 1% improvement Task 2. Additionally, fine-tuning pretrained models demonstrates substantial improvements ranging from 6-9%. Among these, XLM-Roberta demonstrates optimal performance for both Task 1 and Task 2, thereby being selected as the final model for further experimentation. Turning to Task 3, as indicated in Table 4, mBERT marginally outperforms XLM-RoBERTa. Consequently, mBERT is chosen as the final model for Task 3.
The datasets retrieved from Twitter contain additional information such as usernames and URLs alongside the original posts and comments. To gauge their impact, we conducted experiments using various preprocessing methods. These techniques included removing usernames, URLs, punctuation, and emojis. The outcomes of these experiments are presented in Table 5. From the table, we can see that removing url from the tweets improves the model performance for task 1 and task 3. For task 2, no preprocessing is helpful.
For the hyper parameter settings, we also investigated with diferent values of them. We did ablation
studies on learning rate with two diferent values [1e-5, 2e-5], on batch size with values [
All experiments were conducted using Python (version 3.12) and PyTorch, leveraging the free NVIDIA Tesla P100 GPU provided by Kaggle. For the transformer based models we consider Huggingface transformers library. All the transformer based models were run for 10 epochs.
When assessing the efectiveness of the models, we consider diferent performance metrics. Mainly we focus on Accuracy, Macro-F1, and ICM for our performance as our evaluation metircs.
Accuracy measures the proportion of correctly classified instances among all instances. It is calculated by dividing the sum of true positives (correctly predicted positive instances) and true negatives (correctly predicted negative instances) by the total number of instances.
=
+ + + +
The F1 score is the harmonic mean of precision and recall. In the macro F1 score, each class is given equal weight, and the mean of these F1 scores across all classes is calculated. Macro F1 Score is calculated as follows: 2 F1-Score =
1 1 +
Macro-F1 = 1 ∑︁ F1-Score
=1 4.4.3. ICM The ICM metric functions as a similarity measure employed to assess the likeness between modelgenerated outputs and the actual ground truth in classification tasks. It does so by comaring the Information Content of catrgories presented through its features with the help of a Similarity function. It extends the principles of Pointwise Mutual Information, a common metric used for evaluating relationships between words. When all parameters in the ICM formula are set to 1, it becomes equivalent to PMI. This means ICM can capture similar relationships as PMI but with more flexibility. ICM helps evaluate how well a system’s output aligns with the ground truth by comparing the information content of the categories they represent. (ICM) is adopted for all tasks and evaluation types (hard-hard, hard-soft, soft-soft). ICM-soft is an extenstion of ICM for evaluating hierarchical multi-label classification tasks where there might be disagreements in the ground truth. It can can handle both soft system outputs and soft ground truth.
Table 9 exhibits the results of various sentence representation extractors employing diferent aggregation techniques across three tasks on the validation dataset. Regarding task 1, employing the CLS tokenbased representation yields an accuracy of 84.90% and an F1 score of 83.90%. Transitioning to context vector representation through an attention feature extractor results in a slight increase in the F1 score. Furthermore, combining both types of representation through addition-based aggregation shows a 2% enhancement in performance across the board. If we combine them using concat based aggregation techniques we see a small improvement in F1 score but not like the addition based aggregation one
For task 2, we experience similar trend in the performance where attention based sentence representation performs better than CLS token based sentence extractor. If we combine both CLS and context vector getting from attention through addition based aggregation techniques, 1% improvement in accuracy and 2% improvement in F1 score. While we consider concat based aggregation techniques for combining both sentence level representation, the performance is further improved by 1.2% in accuracy and around 3% improvement in F1 score. For task 3, we also get better result than the CLS based baseline while we aggregate both sentence representations using concatenation.
The top-performing model is chosen to generate predictions for the test dataset across all three tasks. The performance results on the test dataset are detailed in Table 10, showcasing the ICM-Hard, ICM-Hard Norm, and Macro-F1 scores as our performance metrics. Our best-performing model achieves an F1 score of 75.01 for all tweets, with 72.09 and 77.33 F1 scores recorded for English and Spanish tweets, respectively. For task 2, the score is 48.56, while task 3 reaches 43.79 across all tweet data. The table shows that our model can predict Spanish tweets more efectively than English tweets.
The evaluation of our approach for Task 1 is done on the development dataset. The confusion matrix shown in 2(a) represents the performance of the classification. The number of True Positive is the number of correctly predicted positive cases. The model correctly predicted 388 cases as "Yes" (True Yes). True Negative cases is the number of correctly predicted negative cases. The model correctly predicted 418 cases as "NO" (True No). False Positive is the number of incorrectly predicted positive cases. The model incorrectly predicted 61 cases as "YES" when they were actually "NO" (False YES). False Negative (FN) is the number of incorrectly predicted negative cases. The model incorrectly predicted 67 cases as "NO" when they were actually "YES" (False NO). The model produced 806 correct classification as opposed to 128 misclassification.
Similarly, the confusion matrix is shown in 2(b) shows the result of the evaluation done on the development dataset for Task 2. As shown in the figure, out of 479 true NO labels, 420 cases were correctly predicted as NO, 150 were correctly predicted as DIRECT out of 204 true DIRECT cases, 36 were correctly predicted as REPORTED out of 65 true REPORTED cases and 18 were correctly predicted as JUDGEMENTAL out of 83 true JUDGEMENTAL cases. The model produced 624 correct classification as opposed to 217 misclassification.
We have calculated the Layer Integrated Gradient attributions for few specific input tweets using Captum [21]. The attributions explain how each input element of a tweet contributes to the model’s prediction for the target class. It means that the attention provided by the model to each tokens. So we name it attention heatmap. The darker the color in the cell of a token, the higher its attribution value (a) Task 1 (b) Task 2 and the higher its contribution to the target. We have considered both English and Spanish tweet for the experiment and creating the heatmap. The corresponding labels are also listed in the Figure 3. We can infer that for ’NO’ label the most attentive tokens among the 4 tweets are last, abuse, ada, in and for ’YES’ label the most attentive tokens among the 4 tweets are economy, s, a, LAS. This experiment was done considering the best performing model in the task 1.
Attention heatmap for tokens of a sample tweet Lang
Label EN EN ES ES EN EN ES ES
NO NO NO NO YES YES YES YES
We also extracted the most attentive tokens for each category in the validation dataset for further analysis. We determine these tokens by extracting the highest average self-attention scores for each token across the entire category-wise validation dataset. Transformer-based models compute attention scores to gauge the relationship between and across various layers and heads. Considering a transformer-based model with layers and heads per layer, and a sequence length of for a given sentence , the resulting attention matrix has dimensions of × × × . To compute the average attention score for token within sentence , we follow this calculation:
Avg_Attn_Score =
1 · · =1 ℎ=1 =1,̸= ∑︁ ∑︁ ∑︁
Attention,,ℎ,
To find category-wise average attention score for token on the validation dataset, we take the average of Avg_Attn_Score for those sentences where token appears in. After calculating the token attention scores, we arrange them in descending order and select the top K words. In Table 11, we present the category-wise most attentive tokens predicted by the top-performing model for both Task 1 and Task 2.
The table presents a detailed breakdown of the most attentive tokens categorized by the topperforming model across two distinct tasks. In Task 1, where the classification pertains to identifying sexist content, the model identifies tokens such as "volant," "slut," and "cum" as prominent indicators for non-sexist content, while tokens like "penis," "forever," and "gangbang" are highlighted for identifying sexist content. Task 2, which involves various categories like "NO," "DIRECT," "REPORTED," and "JUDGEMENTAL," exhibits a diverse range of most attentive tokens. For instance, in the "NO" category, tokens like "pun," "wall," and "question" stand out, while "DIRECT" category tokens include "where," "GPS," and "tomorrow." Additionally, tokens such as "wenr," "school," and "saben" are emphasized in the "REPORTED" and "JUDGEMENTAL" categories, respectively. This comprehensive analysis sheds light on the model’s attention mechanism and its discernment of diferent types of content within each task.
The EXIST challenge is designed to promote research on automated sexism detection and modeling in online environments, with a particular focus on Twitter. In this paper, we performed extensive research and conducted thorough experiments to achieve this objective, employing advanced techniques in natural language understanding and machine learning. Specifically, we enhanced existing machine learning models by incorporating an attention layer and a CLS token, which emphasize the words that contribute to the context of sexism. Our findings demonstrate the efectiveness of our approach and models on both the training and validation datasets.
However, as the training is done on Spanish and English language the model might not be proficient at identifying sexist content in other languages. This can lead to misclassification of sexist Tweets.
This project has been sponsored by Penta Global Limited Bangladesh. We would like to express our deepest gratitude to Penta Global for for their financial support. [20] J. Cañete, G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, J. Pérez, Spanish pre-trained bert model and evaluation data, in: PML4DC at ICLR 2020, 2020. [21] N. Kokhlikyan, V. Miglani, M. Martin, E. Wang, B. Alsallakh, J. Reynolds, A. Melnikov, N. Kliushkina, C. Araya, S. Yan, O. Reblitz-Richardson, Captum: A unified and generic model interpretability library for pytorch, CoRR abs/2009.07896 (2020). URL: https://arxiv.org/abs/2009.07896. arXiv:2009.07896.