The ever-increasing number of applications based on semantic text analysis is making natural language understanding a fundamental task. Language models are used for a variety of tasks, such as parsing CVs or improving web search results. At the same time, concern is growing around embedding-based language models, which often exhibit social bias and lack of transparency, despite their popularity and widespread use. Word embeddings in particular exhibit a large amount of gender bias, and they have been shown to reflect social stereotypes. Recently, sentence embeddings have been introduced as a novel and powerful technique to represent entire sentences as vectors. However, traditional methods for estimating gender bias cannot be applied to sentence representations, because gender-neutral entities cannot be easily identified and listed. We propose a new metric to estimate gender bias in sentence embeddings, named bias score. Our solution, leveraging the semantic importance of individual words and previous research on gender bias in word embeddings, is able to discern between correct and biased gender information at sentence level. Experiments on a real-world dataset demonstrates that our novel metric identifies gender stereotyped sentences.
Language models are used for a variety of downstream applications, such as CV parsing for a
job position, or detecting sexist comments on social networks. Recently, a big step forward in
the field of natural language processing (NLP) was the introduction of language models based
on word embeddings, i.e. representations of words as vectors in a multi-dimensional space.
These models translate the semantics of words into geometric properties, so that terms with
similar meanings tend to have their vectors close to each other, and the diference between two
embeddings represents the relationship between their respective words [
Word embeddings boosted results in many NLP tasks, like sentiment analysis and question
answering. However, despite the growing hype around them, these models have been shown
to reflect the stereotypes of the Western society, even when the training phase is performed
over text corpora written by professionals, such as news articles. For instance, they return
sexist analogies like : = : ℎ [
Lately, sentence embeddings – vector representations of sentences based on word embeddings
– are also increasing in popularity, improving results in many language understanding tasks, such
as semantic similarity or sentiment prediction [
This work expands research on social bias in embedding-based models, focusing on gender
bias in sentence representations. We propose a method to estimate gender bias in sentence
embeddings and perform our experiments on InferSent, a sentence encoder designed by Facebook
AI [
Although language models are successfully used in a variety of applications, bias and fairness in
NLP have received relatively little consideration until recent times, running the risk of favouring
prejudice and strengthening stereotypes [
Static word embeddings were the first to be analysed. In 2016, they have been shown to exhibit
the so-called gender bias, defined as the cosine of the angle between the word embedding of a
gender-neutral word, and a one-dimensional subspace representing gender [
Research is quite lacking regarding sentence-level representations. WEAT was extended to
measure bias in sentence embedding encoders: the Sentence Encoder Association Test (SEAT)
is again based on the evaluation of implicit associations and showed that modern sentence
embeddings also exhibit social bias [
As already mentioned, gender bias in word embeddings is estimated using the cosine similarity
between word vectors and a gender direction identified in the vector space [
⃗ · ⃗ ‖⃗‖ ‖⃗‖ , where is the angle between ⃗ and ⃗. The more cos( ) approaches 1, the higher is the semantic similarity between ⃗ and ⃗. In word embedding models, similarity with respect to the gender direction means that a word vector contains information about gender. Since only genderneutral words can be biased, gendered words like man or woman are assumed to contain correct gender information.
When it comes to sentence representations, the main problem is that gender-neutral sentences cannot be easily identified and listed like words, because sentences are infinite in number. Moreover, they may contain gender bias despite their being gendered. Consider the sentence my mother is a nurse: the word mother contains correct gender semantics, but the word nurse is female stereotyped. Table 1 shows that the gender-neutral sentence someone is a nurse still contains a lot of gender information due to the bias associated with the word nurse.
Therefore, it is important to distinguish between the amount of encoded gender information
coming from gendered words, and the amount coming from biased words. For this reason, we
adopt a more dynamic approach: we keep working at the word level, using the cosine similarity
between neutral word representations, and the gender direction to estimate word-level gender
bias. Then, we sum the bias of all the words in the sentence, adjusted according to the length of
the sentence and to the contextualised importance of each word. This decision is grounded on
two observations: first, the semantics of a sentence depends largely on the semantics of the
words contained in it; second, sentence embedding encoders are based on previously defined
word embedding models [
To estimate gender bias in sentence representations, we consider four elements: • cos(⃗, ⃗): cosine similarity between two word vectors ⃗ and ⃗, • : gender direction identified in the vector space, • : list of gendered words in English, • : a percentage estimating the semantic importance of a word in the sentence. Our metric, named bias score, takes a sentence as input, and returns two indicators corresponding to the amount of female and male bias at sentence level. Respectively, they are a positive and a negative value, obtained from the sum of the gender bias of all words, estimated from cosine similarity with respect to the gender direction. Since gender bias is a characteristic of genderneutral words, gendered terms are excluded from the computation and instead their bias is always set to zero. In detail, for each neutral word in the sentence we compute its gender bias as the cosine similarity between its word vector and the gender direction , and then we multiply it by the word importance . In particular, for a given sentence : () = () = ∑︁ cos(, ) × ∈∈/ ⏟ >0⏞ ∑︁ cos(, ) × ∈∈/ ⏟ <0⏞ Notice that, for each word that is gender-neutral, ∈/ . Also, word importance is always a positive number, and the cosine similarity can be either positive or negative. Therefore, bias score keeps the estimations of gender bias towards the male and female directions separated. In the following sections, we go into more detail by illustrating how we derive , and .
The first step of our method is to identify in the vector space a single dimension comprising the majority of the gender semantics of the model. The resulting dimension, named gender direction, serves as the first term in the cosine similarity function, to establish the amount of gender semantics encoded in a vector for a given word, according to the model.
GloVe [
As shown in Fig. 1, the top component resulting from the analysis is significantly more important than the other components, explaining almost 60% of the variance. We use this top component as gender direction, and we observe that embeddings of female words have a positive cosine with respect to it, whereas for male words we have a negative cosine.
A list of gendered words is fundamental to estimate gender bias, because only gender-neutral entities can be biased. Since the number of elements in the subset of gender-neutral words in the vocabulary of a language is very big, while the subset of gendered words is relatively small (especially in the case of the English language), we derive as the diference between the complete vocabulary of the language and the subset of gendered words: = ∖ . To achieve this, we define a list of words containing as many of the elements of the subset as possible. Therefore, gender bias is estimated for all elements in the subset (neutral words), whereas for all elements in the subset (gendered words) the gender bias is always set to zero: ∀ ∈ , () ̸= 0 ∀ ∈ , () = 0 For this reason, all the elements from are not considered when estimating gender bias. As a matter of fact, we consider the gender information encoded in their word embeddings to be always correct. Examples of gendered words include he, she, sister, girl, father, man.
Our list contains a total of 6562 gendered nouns, of which 409 and 388 are respectively
lower-cased and capitalised common nouns taken from [
Following the approach in [
3https://www.kaggle.com/datagov/usa-names 15 % 10 5 0 0 2 4 6
8
We consider both the absolute importance of each word, and the percentage with respect to the total absolute importance of all the words in the sentence. For instance, in the example of Fig. 2, the absolute importance of the word saxophone is 1106, meaning that its vector representation is selected by the max-pooling procedure for 1106 dimensions out of the total 4096 dimensions of the sentence embeddings computed by InferSent. The percentage importance is 14100966 ≈ 0.27, meaning that the word counts for around 27% of the semantics of the sentence. In particular, the percentage importance is also independent of the length of the sentence, despite the fact that very long sentences generally have a more distributed semantics. For this reason, we use the percentage importance to compute bias score.
Bias score enables to discern gender bias towards the female and male directions. However, we can also take the absolute value of each word-level bias to derive a single estimation of gender bias at sentence level: -() = ∑︁ | cos(, ) × | ∈∈/ ⏟ word-leve⏞l bias
Table 2 illustrate an example of gender bias estimation via bias score, showing that gender
stereotyped concepts like wearing pink dresses are heavily internalised in the final sentence
representation. Additionally, we used bias score to estimate gender bias for sentences contained in
the Stanford Natural Language Inference (SNLI) corpus, a large collection of human-written
English sentences for classification training [
Football players scoring touchdowns -0.149844 Football players playing defense. -0.140169 A defensive player almost intercepted the football from the quarterback. -0.139420 Baseball players -0.138058 the highest absolute bias score include sentences with a high bias score in either the female or male direction, like football players scoring touchdowns or the bikini is pink. Additionally, sexualised sentences like the pregnant sexy volleyball player is hitting the ball are also present.
In this paper we proposed an algorithm to estimate gender bias in sentence embeddings, based on
a novel metric named bias score. We discern between gender bias and correct gender information
encoded in a sentence embedding, and weigh bias on the basis of semantic importance of each
word. We tested our solution on InferSent [
Future work will include adapting the proposed solution to diferent language models and diferent kinds of social bias. Additionally, since bias score allows to identify stereotypical entries in natural language corpora used for training language models, removing or substituting such entries may improve the fairness of the corpus. Thus, future work also includes re-training language models using text corpora made fairer with such procedure, and a comparison with the original models both from the quality and the fairness point of view. We are grateful to our mentor Letizia Tanca for her advice in the writing phase of this work.