For the EXIST 2021 sexism detection task, a novel approach is proposed. In previous work we see that deep learning models capture biases related to race, gender, religion, etc. There are numerous works that seek to lower the bias of these models to make them fairer and more equitable. We will use the opposite approach to these works, instead of reducing the bias of the models we will seek to increase it. So we will train 3 times the same base model, generating 3 versions of the model. A normal version, a sexist version and an anti-sexist version. Belonging to these models will mean marking or not the tweet as sexist.
The EXIST 2021 workshop deals with the identi cation and classi cation of
sexist messages coming from the websites gab.com and twitter.com. Sexism
detection is not a novel task, we could consider it a specialization of the hate
detection task, in which the hate has an impact on one of the genders. Although
it is not a novel task, it is a topical task. There are already systems in production
that use hate detection as an automatic content moderation tool, for example
twitter [
This is not a trivial task since in many cases it is the social context itself that causes a message to undervalue the role of one of the genders or even feed negative stereotypes. This document explains the approach followed for the participation in the rst version of the EXIST 2021 workshop along with the data and tools used.
EXIST is proposed as a workshop embedded as part of IberLEF. It provides a set of tweets and gabs in Spanish and English and proposes two tasks. The rst one is to detect whether each of the messages is sexist or not. The second task proposes to classify the type of sexism present in the tweet for a set of proposed categories.
The task considers that a message is sexist as one that produces prejudice, inequality or stereotype following The Oxford English Dictionary de nition. They also provide a set of tweets and gabs labeled in the proposed tasks. 1.2
PreviousnRelated Work
There are many previous works [
We will try a novel approach, instead of using a classi er we will train three models of each language (English and Spanish) and we will score how likely are the tweets to belong to each model. One model base model will be trained with random data, the stereotyped model will be trained with toxic/stereotyped comments and the anti-stereotype will be trained with data sample that contradicts stereotypes. Given a sentence, it will be considered toxic/sexist if the likelihood of been generated from a stereotyped model is greater than that of any of the other two models (anti-stereotyped and neutral). 2.1
Previous work demonstrates that neural network based models capture certain
associations present in the data that may be sexist, racist, or have some type
of unwanted bias. The study of bias in neural network based models shows
how recent models such as BERT or even GPT-3 capture these biases. Studies
such as [
One of the most common examples is that Word embeddings models captures the association man - woman ! king - queen correctly, however it also captures associations like man - woman ! computer scientist - homemaker
Using a list of professions it is obvious that there are professions strongly associated with men and others strongly associated with women. Despite the fact that in certain applications this bias may be detrimental, we will take it as an advantage. 2.2
Civil Comments/Jigsaw corpora This dataset is the one proposed by Google's
Jigsaw initiative for the [
StereoSet corpora It is a dataset in English labeled to re ect the stereotypes
present in the data. To do this, a dataset is built[
Multiple versions of BERT are available, we have used bert-base-uncased.
Spanish BERT model (BETO) [
For working with the models the HuggingFace libraries were used. Some scripting and data transformation work was done using custom nodejs/python scripts. 2.4
We have taken the English and Spanish models described above and ne-tuned them using the two stereotyped corpora introduced before. The ne-tune task was the same as the common task used to train these two models from scratch: masked language modeling. Training process was performed on a machine equipped with x2 Intel(R) Xeon(R) Silver 4208 CPUs at 2.10GHz 192 GB of RAM and x2 NVIDIA Tesla V100 (32GB, Volta). The implementation was based on the PyTorch versions of these models provided by the Transformers library. It took about 10 minutes for training the Spanish version of the three models (stereotype, anti-stereotyped and neutral) and about 20 minutes for the three English ones. 2.5
We calculate the probability of a sentence belonging to the model by using the chain rule of probability. To get the normalized probabilities we apply the softmax function to the model output logits which are the raw predictions of the last layer of the network.
P (X1; X2; ; Xn) = P (X1) P (X2jX1) P (X3jX1; X2) P (XnjXn 1; Xn 2;
Qn k=1 P (XkjX1k 1) ; X1) = (1)
It has been veri ed with some sentences from the training set that the model does indeed capture some structures in the form of probabilities. For example for the sentence "The schoolgirl is walking down the street, She is a target for child predators", the probabilities are:
AntiStereotype Other Stereotype the 0.00894868653267622 0.032950423657894135 0.005774959456175566 schoolgirl 0.0002689003486011643 0.00017878220387501642 0.002795979264192283 is 0.006179201416671276 0.03133246302604675 0.00566072016954422 walking 6.194096204126254e-05 3.609031409723684e-05 0.0002682682534214109 down 9.467829659115523e-05 2.7176944058737718e-05 6.732833571732044e-05 the 0.026873258873820305 0.02360343001782894 0.023960916325449944 street 0.005731010489128607 0.0007263075825676424 0.1331725064192142 she 0.001979067223146558 0.0008687127847224474 0.0036194631829857826 is 0.0073087140917778015 0.02685435861349106 0.08509364724159241 a 0.0025236799847334623 0.0009333917987532914 0.0032958367373794317 target 1.7575850506545976e-05 1.2269218530036596e-07 1.707993391164564e-07 for 0.00017388597188983113 3.896695488947444e-05 8.425339183304459e-05 child 0.022296445444226265 0.0001155364152509719 0.011923404410481453 predators 0.004697749387711762 0.0002539085783155315 0.001255747340716577 Total 1.5692968748328783e-40 9.478321051397374e-48 1.1521866603985196e-39
This phrase is e ectively classi ed as stereotypical. However, the di erence with respect to anti-stereotyped, which is the opposite class, is not large enough. This may be due to the fact that the probability calculation method does not treat BERT as bidirectional. 4
We have presented a proposal for the classi cation of sexist tweets based on bias Although we see that there are di erences between the models, there is still a lot of work to be done for this approach to give good results. On the one hand, it will be necessary to nd concrete structures that are representative of sexist or toxic content. Once the structures have been found, it will be necessary to improve the training method to emphasize the structures and thus ne-tune the model by focusing on the characteristics of the problem.
Another point where improvement can be made is in the calculation of model membership, and other metrics can be applied to obtain a more reliable result.
One aspect that would be interesting to explore is the automatic extraction of social media content to train the model. Is it feasible to characterize as toxic or non-toxic non-annotated texts from social networks using these networks' own metrics? For example, characterizing as toxic those with a large proportion of negative votes and as non-toxic those with a proportionally large number of positive votes. This can allow the model to mature, having the same information that ts the reality.