Sexism is an ancient evil that society struggles with till date. The freedom of speech and ease of anonymity granted by social media has also resulted in incitement to hatred. This presents the need for automatic detection of sexist posts or tweets on social media. In this paper, we present the machine learning models that can detect sexism. Speci cally, we describe the model submitted for the shared task on sEXism Identi cation in Social network at IberLeF 2021. The problem concentrates on sexism detection in two languages: English and Spanish. The challenge is divided into two tasks of di erent granularity: (1) a binary classi cation problem to discover the instances of sexism in the post and (2) to predict one the ve types of sexism present. Overall, our performance is good but it needs some improvement, our scores are encouraging enough to work for better results in future.
The digitalization of the world also means the digitalization of sexism. Sexism can be referred to as individuals' attitudes, beliefs, behaviors, organizational, institutional and cultural practices that either re ect negative evaluations of individuals based on their gender or support unequal status of women and men. With rapid advancement of the internet and social media, such beliefs can spread rapidly. Thus, it is very essential to detect such behaviour. The amount of data generated on social media sites can be estimated from the fact that, every second, on average, around 6,000 tweets are generated. Content moderation of such a huge data is di cult to achieve exclusively through man power. Social networking sites are struggling with content moderation. Arti cial Intelligence and di erent Machine Learning techniques can be exploited for Sexism Detection.
In this paper, we present the models to detect sexism, speci cally the system we submitted for EXIST 2021 shared task 3. Sexism can be expressed directly, 3 IberLEF 2021, September 2021, Malaga, Spain.
Copyright ' 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
implicitly and can be hidden in the form of humor. Automatically detecting the
subtle instances of sexism is much harder than discovering the misogyny and
violence aimed towards women. We try to identify the instances of both implicit
and explicit sexim and future classify the sexism into one of the ve categories
which are as follow: 1. Ideological and Inequality, Stereotyping and Dominance,
Objecti cation, Sexual Violence, Misogyny and Non-sexual Violence.
As per requirement of EXIST 2021 [
The organization of the rest of the paper is as follows. Section 2 describes about Related Work. In Section 3, we describe about Task & Dataset. Section 4, describes about System Description i.e. Test Preprocessing, Feature Extraction and Machine Learning Models. In Section 5, we discuss about our Results. Lastly, Conclusion and future direction of our research work is presented in section 6. 2
Several works have been proposed to detect misogyny and hate speech against
women across social platforms. Resham Ahluwalia et al. [
The dataset we used, misogynist tweets are considered as a sub-category of sexist tweets. As most of these works focus on misogyny and hate speech detection but not the subtle instances of sexism. We try to identify these subtle sexism in tweets. 3
In this section, we describe the automatic sexism detection shared task and the
dataset provided to the participants [
EXIST 2021 shared task was divided into two subtasks (Task1 and Task2). Task1 aims at nding the instances of sexism while the goal of Task2 is to categorize the sexism into one of the following 5 categories (shown in Table 1): 1. Ideological and inequality : Discrediting of the feminist movement, rejecting the precence of inequality between men and women, or presenting man as the victim of gender bias. 2. Stereotyping and dominance : Expressing the false idea about women like they are more suitable for certain tasks and not suitable for certain others. 3. Objecti cation : Presenting women as objects. 4. Sexual Violence : Sexual suggestions, requesting sexual favours or sexual harrasment threats. 5. Misogyny and Non-sexual Violence : Indicating voilence and hate toward women of non-sexual nature.
We used the dataset obtained from the shared task at IberLEF 2021. The dataset contains 11,000 short texts i.e. tweets and gab posts in English and Spanish languages. The dataset consists of 6,977 tweets for training and 3,386 tweets for testing with balanced distribution in both the languages. In addition the test data contains 492 gabs in English and 490 in Spanish from the uncensored social network Gab (gab.com). The data was labeled manually by human annotators. Data was annotated at two di erent levels of granularity. First, each text was labelled as sexist or non-sexist. Secondly, the instances labelled as sexist are further divided as ideological-inequality, stereotyping-dominance, objecti cation, sexual-violence, misogyny-non-sexual-violence. The EXIST dataset focuses on covering sexism in a broad sense, i.e, both explicit misogyny to implicit subtle sexism. 4
We splitted the annotated data into a training set with 85% of the annotated data and validation set with the remaining 15% of the instances. We built similar systems for both English and Spanish, and then results obtained by these two systems were combined and submitted as a run. Same splitting was used for both task1 and task2. 4.1
We removed punctuation symbols, links and numbers. Also, we removed stop words. We used lemmatization for the English text and stemming for the spanish text to ping together the di erent in ected forms of a word. In particular, we used nltk wordnet for lemmatization and nltk snowball for stemming. The same preprocessing is used for both Task1 and Task2. 4.2
We extracted the following features from the text:
Upper Case Count : Number of uppercase letters present in the text Hashtag Count : Number of hashtags present in the text Link Count : Number of hyperlinks in present in the text Tweet Length : Total number of characters in the text
Slang Words Count : used in the English language These features were extracted before preprocessing as hashtags and links were removed after preprocessing. We used the Slang word count feature only for the English text for both Task1 and Task2. All the rest features were used for both languages and both tasks. To convert the text into numerical features we used Tf-idf vectorizer. For LSTM, we used the tokenizer by keras library and for SVM and Random Forest we used Tf-idf vectorizer from scikit-learn library. Tokenizer by keras converts the text into either a sequence of integers or into a vector where the coe cient for each token could be binary, based on word count, based on tf-idf etc. 4.3
For Task1, we submitted three runs based on three di erent algorithms, namelySVM, Random forest and LSTM. We used the scikit-learn library for SVM and Random forest based models and Keras for LSTM. For SVM, we experimented with three kernels - linear, rbf (radial basis function) and polynomial. The nal run we submitted was the linear kernel SVM. We used the following values of the parameter in scikit-learn: 1. For SVM, kernel coe cient gamma is 0.01, and penalty parameter is 5 (default parameters). 2. For Random Forest, maximum depth is 150, and number of forest is 1000. 3. For LSTM, we used mean squared error for loss,and batch size as 64.
For Task2, we submitted two runs based on -SVM and Random Forest. We treated Task2 as a multi-classi cation problem with 6 categories ( ve categories of sexism and one category as non-sexist) and not as an extension of Task1. That is, in case of Task2, we used the whole dataset for training and not just the text classi ed as sexist in Task1. The parameter values were the same as mentioned above. And SVM was implemented with a linear kernel. 5
The results of Task1 are represented in terms of accuracy (shown in Table 2), while the results of Task2 are in terms of macro F1 score (shown in Table 3). The best score as accuracy, we get from Task1 is 0.7115. For Task2 we get best score as F1- measure is 0.4504. Table 2 and 3 shows the ranking of our submissions based on shared task o cial ranking. Our best system was ranked 38 in Task1 and 43 for task2. Also, baseline is provided by the organizers for both the task. These rankings indicate the combined results in English and Spanish.
For Task1, the Random forest system performed better than LSTM and SVM. But for Task2, SVM performed better than Random forest. The accuracy and F1 score obtained in Task2 was lower than that of Task1. The same trend could be found in results of all the teams. This can be attributed to the fact that classi cation of the sexist text into ner granularity is a much more di cult task than detecting instances of sexism. Also, as Task2 was multi-class classi cation problem it's F1 score was lower as compared to the binary classi cation problem in Task1 .Features like number of slang words, number of hashtags present, hyperlink count resulted in improved accuracy. We evaluated the performance of di erent classi cation algorithms for sexism this year’s shared task. The results, we achieved were average as compared to other submissions obtained in the EXIST 2021 shared task. We found that extracted features like number of slangs and hyperlink counts can result in better performance. We look forward to experimenting with di erent features. Also, ne tuning the parameters of the algorithm can help in improvement of the overall performance. And the results of more than one system can be combined to generate an overall better score. We shall be exploring these tasks in the coming days.