In this work we study gender bias in Italian word embeddings (WEs), evaluating whether they encode gender stereotypes studied in social psychology or present in the labor market. We find strong associations with gender in job-related WEs. Weaker gender stereotypes are present in other domains where grammatical gender plays a significant role.
In the literature, the study of gender bias in word
embeddings (WEs) is of interest for two main
reasons: (i) WEs, as components of automatic
decision systems (e.g. job search tools), may
contribute to harm some user groups
A common taxonomy of bias in algorithms
concentrates on the types of harm that they may cause
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). clear example of this situation. Query completion suggestions for “why are italian . . . ” associate diverse concepts to the country and its inhabitants. Italians contribute very little to these results as they are unlikely to search information about themselves in English.
Italian WEs have been developed
Gender stereotypes are representational harms
which influence the lives of women and men both
descriptively and prescriptively, shaping the
qualities, priorities and needs that members of each
gender are expected to possess
To identify a vectorial subspace which encodes information about gender, we follow Bolukbasi et al. (2016) by building a list of gender definitional pairs: [lui (he), lei (she)], [uomo (man), donna (woman)], [padre (father), madre (mother)], [marito (husband), moglie (wife)], [fratello (brother), sorella (sister)], [maschio (male), femmina (female)].
These pairs are built so that the second word denotes a female entity and the first word is, semantically, its male counterpart. Moreover, given we are interested in capturing semantic information about gender, while avoiding entanglement with grammatical gender, we ensure that the words in a pair do not derive from the same root via inflection. An example of pair discarded due to this criterion is [figlio (son), figlia (daughter)].
Principal Component Analysis (PCA) on the six vector differences resulting from each gender definitional pair. The first eigenvalue dominates the remaining ones, with the first PC explaining 57% of variance. We normalize the first PC and consider it the main gender direction, denoted by gPCA.
This is an established procedure to isolate the
direction that captures most of the information
about gender
Vector differences. To evaluate the robustness of this approach and highlight potential anomalies, we also consider each vector difference on its own, defining six unit length gender directions gdiffi : gdiff0 = l!ui gdiff1 = uom!o gdiff2 = pad!re ! lei don!na mad!re gdiff3 = mari!to gdiff4 = fratel!lo gdiff5 = masch!io mog!lie sorel!la femmi!na
us indicate with w its corresponding word vector. Let us consider any of the gender directions g defined above. We call gender score the normalized projection of w onto the direction g, defined as sg(w) = w g=(jwjjgj): (1) This scalar captures associations of w along gendered lines. Informally, a highly positive value means that w is closer to the male terms of the pairs than to the female ones, while a strongly negative value entails the opposite. 3.1.2 WEAT The Implicit Association Test (IAT - Greenwald et al. (1998)) is an assessment developed in cognitive psychology to measure subconscious associations between categories and concepts. It is commonly employed to assess implicit stereotypes in people. The Word Embedding Association Test (WEAT - Caliskan et al. (2017)) is a technique inspired by the IAT to measure associations between concepts in WEs. Let X and Y be two equal-sized sets of target words and A and B two sets of attribute words, e.g., X = fprogrammer; engineerg, Y = fnurse; teacherg, A = fman; maleg, B = fwoman; femaleg. Let cos(a; b) be the cosine similarity between the word vectors a and b. The differential association of a word w (taken from X or Y ) with the attribute sets A and B is measured as c(w; A; B) = meana2A cos(w; a)
meanb2B cos(w; b): (2) The normalized differential association between targets and attributes is defined as d = meanx2X c(x; A; B) meany2Y c(y; A; B) std-devw2X[Y c(w; A; B) (3)
This is called effect size in statistics, and summarizes how different the quantity c(w; A; B) is, when evaluated on elements of target set X as opposed to target set Y . It is computed as a difference of means within each set, divided by overall standard deviation.
Gender score and WEAT. It is worth noting that,
when jAj = jBj = 1, WEAT is almost equivalent
to the gender score defined in Section 3.1.1. Let
A = fa0g and B = fb0g be the sets of attribute
words. Since we are using normalized vectors and
the distributive property holds for the dot product,
then
:
c(w; A; B) = cos(w; a0)
cos(w; b0)
= w (a0
b0) = w g = sg(w):
(4)
Italian is a gendered language, wherein
grammatical gender is assigned to all nouns. Within a
sentence, each word is surrounded by other words of
agreeing grammatical gender. This phenomenon,
called grammatical gender agreement, in
conjunction with the distributional hypothesis
Mean gender score. To compute the gender score (Equation 1) for gendered words that have both a feminine and a masculine version, we propose the following approach. Let us indicate with wf and wm the feminine and masculine version of a gendered word w. We define their gender score as smeang (w) = (sg(wf ) + sg(wm))=2: (5)
Averaging the masculine and feminine version with equal weights corresponds to giving both versions of the word the same importance. Different approaches, based for instance on word frequency, may be applicable in other contexts.
Orthogonal projection. Some nouns cannot be inflected into the opposite grammatical gender, making the above approach impractical. An example is ufficio (office). In this context, we propose to mitigate the effect of grammatical gender by reembedding every word through an orthogonal projection. We build a list of 138 inflected word pairs. Each pair consists of the feminine and masculine inflections of the same root, such as cara and caro (dear), which only differ in grammatical gender. We take the embedding of both words in a pair and compute their difference.
We perform PCA on these vector differences. The resulting PCs span a subspace U that contains most of the variance due to grammatical gender. To reduce the influence of grammatical gender, we re-embed vectors by projecting them on the orthogonal complement of U . In other words, given a word embedding w, let us call projU w its orthogonal projection onto the “grammatical gender subspace” U . We propose re-emdedding every word vector w to w0 = w projU w : (6)
jw projU wj By means of this procedure, we obtain a new set of WEs. By construction, in this new embedding space, grammatical gender should have a lower influence on the geometry of word vectors. 4
To study gender bias we use WEs trained on two
different datasets for the Italian language, both
made available by FastText
We compare our results from the analyses on Italian WEs with results on their English counterpart. To this end, we also download two sets of FastText WEs trained on the English version of the same corpora, i.e. the English counterparts of wiki and wiki-cc. Given Wikipedia is a more curated source, we expect to find weaker stereotypes in wiki than in wiki-cc for both languages. As a pre-processing step we normalize every word vector to unit length.
Census data about the labor market is required
to analyse the correlation between the gender gap
in professions and the gender score of the
respective WEs. The statistics on the American
occupation and gender representation are readily
available
Finally, in order to perform the Word Embedding Association Tests (WEAT), we need sets of 1The authors provide no information about which Wikipedia dump they use.
2Common Crawl is a corpus of web pages, aimed at
representing “a copy of the internet” at a given time. The authors
train WEs on pages written in Italian, exploiting language
identification as preliminary step for their pipeline.
3The detailed list of sources is available upon request.
target and attribute words in Italian. The sets of
target words for the gender-science WEAT
(Section 5.2) are derived from the Italian version of
IAT;4 those for the gender-career WEAT (Section
5.3) were unavailable and have been translated by
the authors of this work from the original IAT
This experiment investigates gender
representation for different jobs in Italy and their association
with gender-realted information in WEs,
following studies on the English language
We compute Pearson’s correlation r between the gender scores and the percentage of women employed in each profession. The same analyses are carried out on English WEs, restricting them to the same set of occupations considered in Italian. Results are summarized in Table 1 and Figure 1, showing that Italian WEs consistently capture information about different gender representation in jobs. Informally, this mean that ordering jobs by percentage of women and by projection on a gender direction yields similar results. The right pane of Figure 1 demonstrates the significant effect of grammatical gender. 5.2
In this WEAT, the sets of target words for Science and Arts, taken from the Italian version of the IAT, are: X = fbiologia (biology), fisica (physics), chimica (chemistry), matematica (mathematics), geologia (geology), astronomia (astronomy), ingegneria (engineering)g, Y = ffilosofia (philosophy), umanesimo (humanism), arte (arts), letteratura (literature), italiano (italian), musica (music), storia (history)g. The sets of male and female attributes are taken from the gender definitional pairs (Section 3.1.1): A = flui, uomo, padre, marito, fratello, maschiog, B = flei, donna, madre, 4https://implicit.harvard.edu/ implicit/italy/takeatest.html IT gPCA gdiff0 gdiff1 gdiff2 gdiff3 gdiff4 gdiff5 EN gPCA IT all moglie, sorella, femminag. We compute the effect size d and the p-value using the whole attribute sets A and B, and label this analysis “all”. Moreover, we also perform the WEAT test over single word pairs, e.g. A = fluig, B = fleig. Results are reported in Table 2. We find no stereotypical association in the expected direction. We hypothesize that this is due to the feminine grammatical gender of all science-related target words, deferring a more detailed analysis to Section 5.4. In essence, the Career and Family WEAT is very similar to the Science and Arts WEAT; the only difference is in the sets of target words. The target sets are translated into Italian from the original English IAT as follows: X = fesecutivo (executive), management (management), professionale (professional), azienda (corporation), stipendio (salary), ufficio (office)g, Y = fcasa (home), ungendered mean gendered rrscoeend eg 0.1 00..01 ((tceeraulleemctcktitrordi(cnicccraiiiiossvaltztbenaaobr()l)alaaeggirorr)(ioonetngneoegcmginnniiecseoetr)re) co(amcmco(enunronoc(tdttidaaaaeerlinnioy(nst(gta))tgatieasriersctocotahlh)oliaoit(gmt(g(eldgejbihoitsooecoetoutctdoarr)t()rtngeivlnocaaalvraoiwlte)o(ilopsicyrcsr)tha(eateawfmi)rtill)ooaoesistrooeiepfror)h(evveeert)t(eepprrririinnneaacsriri((dip(ypzobp(zbea)orpaforaoloaho)rorflrmfaeiolmesorsgatsgmisaainnssooc(oaf()trnieh)rcisegu)ryitsmiar(getsbbi)ineeieoii)norsleolitosagtg(()ipdsodsitpie)yestctiicichsoi(otaiclanblon(aotsag)rgeedeoiagsagctnni)hvateeenrrt))(eo(bmotsesattaeeecttshrrtieicrcorioa)n) 40 % female 60 0 20 80 100 0 20 80 100 mantic association with gender, as measured objectively by the percentage of women in each profession and (ii) a grammatical association with gender, as half of those words admit a feminine and a masculine version.
We measure the relative strength of semantic and grammatical associations in the proposed gender directions as follows. Let us denote by Sg, the set of job-related words which admit a feminine and a masculine version and by w = sg(wm) sg(wf ) the difference in their gender scores.5 We compute the average influence of grammatical gender on direction g (based on set Sg) as g = 1
X jSgj w2Sg w: (7) Visually, this corresponds to the average (signed) length of the vertical lines, in the right pane of Figure 1, connecting the feminine and masculine version of a job-related word.
Moreover, let us denote by S the complete set of job-related words wj and by xj the percentage of women in job wj . Let us indicate with sg(wj ) the respective gender score, computed according to Equation 1 or 5, depending on whether wj admits different masculine and feminine inflections. We define maxS (x) (minS (x)) as the maximum (minimum) percentage of women in a job from set 5For the sake of brevity, we concentrate on gPCA; the remaining gender directions (gdiffi ) yield similar results. genitori (parents), bambini (children), famiglia (family), cugini (cousins), matrimonio (marriage), nozze (wedding), parenti (relatives)g. Results are summarized in Table 3. Stereotypical associations for wiki-cc WEs are present but weak, whereas they are more significant for wiki.
In this section we quantify the extent to which the semantic gender information (Section 3.1) is influenced by grammatical gender, and test one approach designed to mitigate its influence (Section 3.2).
The dataset used in the experiment about jobrelated WEs (Section 5.1) is suitable for this analysis, as it consists of words which have (i) a seS. Furthermore, let us call m the angular coefficient computed by (linearly) regressing sg(wj ) onto xj over set S. We compute the full-scale influence of semantic gender on direction g (based on set S) as s = jm(maxS (x) minS (x))j: (8) Visually, this corresponds to the vertical component of the blue regression line in Figure 1, clipped between minS (x) and maxS (x).
Finally, we compute the relative strength of semantic and grammatical associations in the proposed gender direction as the ratio k = g : s (9)
The first three rows of Table 4 report g, s and k for wiki-cc (first column) on the job dataset described in Section 4. The second column concentrates on a set of word embeddings derived from wiki-cc by removing information about grammatical gender from every word, via Equation 6.6 We label this new set of word embbeddings wiki-cc?. In going from wiki-cc to wiki-cc?, g is reduced by over 40% while s decreases by less than 10%. This indicates that the orthogonal projection procedure reduces the influence of grammatical gender while retaining semantic information which is present in the original version of the WEs, hence the value of k decreases.
The final three rows of Table 4 report summary
statistics for stereotypical associations described
in Sections 5.1-5.3. Interestingly, the significance
of each association is larger for wiki-cc? than
for wiki-cc. In particular, the effect size for the
Science-Arts WEAT becomes positive, in
accordance with the stereotype. We interpret these
results as evidence for the hypothesis that
grammatical gender confounds and outweighs
stereotypical associations in Italian WEs, in line with prior
work on gendered languages
We successfully replicated prior analyses about gender-stereotypical associations in English WEs, finding them to be consistently stronger when computed on WEs trained on a weakly curated 6In this experiment, the grammatical gender subspace U is spanned by the first PC. wiki-cc
wiki-cc?
Occupations (gPCA) corpus. To the best of our knowledge, this is a novel result.
For Italian WEs, the picture is more nuanced
and tied to grammatical gender. WEs for
occupations, which are ungendered or admit a dual
form, are robustly associated with gender along
a stereotypical direction. Compared against the
other stereotypes analysed in this work, this is
the strongest association, confirming results from
prior work on English WEs
In Italian WEs, we find that wiki embeddings contain stronger stereotypical associations than wiki-cc embeddings for the Career-Family WEAT. This disconfirms our hypothesis that WEs trained on a less curated corpus (wiki-cc) would encode stereotypes more strongly. Finally, we find no consistent property connected to specific gender directions gdiffi . Across different corpora and stereotypes, the aggregated analyses (labelled “all” and gPCA) provide a reasonable summary of the stereotypical associations encoded in the single gender directions gdiffi . 7
Overall, we have analyzed gender bias in Italian
WEs, adapting existing techniques and gathering
data where required. We looked for stereotypical
associations with gender-imbalanced professions,
Career and Family, Science and Arts, finding
significant associations in 2 out of 3. As expected
from prior work
We draw the following preliminary conclusions: (i) Italian WEs seem to have less potential than their English counterparts to systematically reinforce the tested gender stereotypes, mostly due to grammatical gender. However, (ii) the influence of grammatical gender on WEs may cause different harms. As an example, in the context of job search, masculine is likely to be the default choice for queries of recruiters (male as norm - e.g. “psicologo” [psychologist]). Those queries would likely be closer to male candidates’ CVs than equivalent female ones, in some embedded text representations, potentially putting women at a systematic disadvantage. Both points above require further analysis of text retrieval/classification systems based on Italian WEs. Finally, (iii) isolating stereotypical concepts and gendered associations in Italian WEs along a single direction is challenging. The tested WEs show little promise as a reliable measurement tool for gender-stereotypical associations, unless combined with approaches to mitigate the influence of grammatical gender.