Language models significantly impact society. They are ubiquitous in applications ranging from search engines to hiring systems. State-of-the-art models like GPT-4 [1] and LLama2 [2] dominate current research due to their high performance. However, earlier models, such as classic pre-trained embeddings (Word2Vec [3], GloVe [4]) and smaller-scale language models (BERT [5]), remain in industrial use for their cost-effectiveness due to fast computation and memory efficiency [6].

All these pre-trained representations present a significant issue: they encode concepts of gender derived from training data that mirror existing patterns of inequality and discrimination [7]. These biased representations can reinforce discriminatory patterns through generated language or influence hiring decisions that perpetuate gender imbalances based on stereotypes [8]. To build trustworthy and fair language technologies, we must therefore ensure that the training language is fair from the outset.

One approach to achieving this is through training with gender-inclusive language, which has three main aims: (1) Avoiding the use of masculine terms generically to refer to people of unknown gender or groups of mixed gender (e.g. mankind→humankind, to man→to staff ), (2) eliminating irrelevant gender distinctions such as in headmistress/headmaster→headteacher [9], and (3) establishing a trans-inclusive model of gender that includes references beyond binary categories, such as the use of singular they or neopronouns [10]. Gender-inclusive language has a long research tradition in feminist linguistics [11,12,13] and has recently become a focus in research on gender bias in NLP. Examples include gender-neutral rewriting models [14,15] and gender-inclusive language as a means of combating misgendering in translation [16] or as a fine-tuning strategy for reducing gender stereotyping in Large Language Models (LLMs) [17,18].

Most of these works, however, make an assumption that equality-promoting effects of genderinclusive language, such as reduction in gender stereotyping and discrimination, can be directly picked

As a fireman, Zachary is always ready to help people, but since his parents' relationship was marked by conflict, he is opposed to commitments.

As a firefighter, Zachary is always ready to help people, but since their parents' relationship was marked by conflict, they are opposed to commitments.

Example sentence from Wikipedia subcorpus before and after gender-neutral rewriting up by LLMs through fine-tuning [17]. However, fine-tuning an LLM necessarily invites interference from the pre-trained model, which might obscure conclusions on how gender-inclusive language is incorporated into model representations of gender. This work therefore presents a foundation-level proof-of-concept study with classic pre-trained embeddings. These allow us to train a model with gender-neutral text from scratch. By contrast, training an LLM from scratch goes beyond our and many other institution's resources [8]. Further, word embedding models might still be used in small-scale industry settings due to their low computational costs, which makes them relevant [6].

We train two Word2Vec embedding models [3] on unchanged vs. gender-neutral English text, additionally comparing against a common post-hoc debiasing technique [19]. The code for our experiments is openly accessible 1 . In the experiments we find that the use of gender-neutral terminology reduces gender stereotyping as measured by the Word Embedding Association Test [20] and the Embedding Coherence Test [21] as well as reducing latent gender information in the embeddings of gender-neutral words. These results demonstrate how incorporating principles of gender-inclusive language, which were designed to help people avoid bias or discrimination in how they speak or write, can have the same effect on how gender is represented in word embedding models.

Our experiments were conducted on a corpus introduced by [22], the Small Heap. The corpus is made up of random subsections of three popular LLM training corpora: OpenWebText2 [23] (50%), CC-News [24] (30%) and English Wikipedia (20%). The final sub-corpus contains ~250 million tokens, or 1.5 GB of text.

The corpus was edited using the NeuTralRewriter [14]. This involved replacing gender-specific pronouns (he, she, him etc.) with the corresponding variant of the gender-neutral pronoun they. Additionally, 91 gender-specific nouns (headmaster, mankind, etc.) including plural and spelling variants were replaced by neutral versions (principal, humankind, etc.; for full set see 14). Table 1 shows an example of rewritten text.

There are two implementations of the NeuTral Rewriter, a rule-based version and a neural, machine translation-based model. While the neural model performed better in the original experiments [14], it proved to be very susceptible to noise in our data (email addresses, digits, etc.) as well as low-frequency words, often translating them into unintelligible text. We therefore used the rule-based implementation, which uses a combination of word, part-of-speech and dependency information to derive the correct replacement of pronouns.

In order to evaluate the effects of gender-neutral language on representations of gender within the corpus, we built three different Word2Vec models [3]. The first was trained on the original and the second on the rewritten corpus. Each Word2Vec model was trained using the Continuous Bag of Words (CBOW) algorithm with the default hyperparameters of the gensim library's Word2Vec class [25]. The third model was created by performing hard debiasing [19] on our original model in order to compare our method to an existing, model-based debiasing method. Hard debiasing modifies embeddings in such a way that gender-neutral words (e.g. babysit) are equidistant to gendered word pairs (e.g. grandfather -grandmother). Additionally, the gendered component of embeddings of gender-neutral words, as defined by what is termed the 'gender subspace', is set to zero.

The three trained embedding models were analyzed for underlying gender bias using three methods. Previous research found that bias measures are not always consistent [26,27]. Using a composition of metrics therefore allows for a more comprehensive evaluation.

The Word Embedding Association Test (WEAT) is one of the most commonly applied bias measures for word embeddings [20]. The test is modelled after a psychological assessment, the Implicit Associations Test [28], and measures bias by computing the mean association between two sets of target and attribute words. We used a WEAT implementation by Lauscher et al. [29]. Each WEAT test (i.e. the specific combination of target and attribute terms) is identified as W𝑁 with 𝑁 corresponding to its position in the original WEAT paper. W9 and W10 were added by Lauscher et al. [29]. We additionally added two tests using attribute words related to male-and female-stereotypical professions (W𝐴) as well as words related to computer science and childcare (W𝐵, cf. Table 4).

Clustering and Classification into two groups was used by Gonen and Goldberg [26] to show that embedding spaces retain gender information despite application of debiasing. We measured cluster integrity after K-Means clustering (averaged over 50 runs) as well as classification accuracy with an SVM (trained for 20 epochs) in order to find out how well gender information can be recovered from the embedding space. We use the original model's 500 most male-/female-biased words according to their similarity to the element-wise mean of the male/female attribute embeddings 𝐴 1 and 𝐴 2 of W8 (cf. Table 5).

The Embedding Coherence Test (ECT) calculates distances between two sets of gendered target words 𝑇 1 ,𝑇 2 and a set of attribute words 𝐴 that relate to a societal gender imbalance (e.g. captain, football) [21]. Instead of relying on the absolute distances, the ECT calculates the Spearman coefficient between the ranked distances for 𝑇 1 and 𝐴 vs. 𝑇 2 and 𝐴. A high coefficient indicates similar ranks between the two gendered sets, signifying reduced bias. We used the ECT implementation by [29].

Semantic Quality is evaluated following Lauscher et al. [29], by using the similarity benchmarks SimLex-999 [30] and WordSim-353 [31] and computing the Pearson and Spearman correlation coefficients between the benchmark term-pair similarities and cosine similarities of the corresponding embedding pairs of the respective models.

We will discuss the results for our chosen gender bias metrics: WEAT [20], ECT [21] and Clustering and Classification [26]. Finally, we contextualize these findings with the performance of our models on two semantic quality benchmarks.

WEAT: Results indicated a reduction in gender bias related to stereotypical associations of women with arts, domestic work, and childcare, and men with (computer) science, maths, and careers, respectively. All five tests measuring gender bias, W6, W7, W8, W𝐴, and W𝐵, showed a reduction in the statistic after rewriting (Table 2). W𝐴 additionally shows that there is a reduction in the association of feminine/masculine words with traditionally gendered professions. Comparing the WEAT scores after rewriting to the hard debiased embeddings, one can see that the scores for the hard debiased embeddings are approaching zero, which indicates equal association of male/female attributes with the respective targets. Thus, on one hand, training with gender-neutral language generally leads to a reduction in WEAT bias scores, indicating that this change in the language can lead to a reduction in associations based on stereotypes. On the other hand, stereotyped associations can be specifically targeted and mostly removed post-hoc. ECT: A reduction in bias was demonstrated in relation to gendered associations with professions. Table 3a shows that for arts vs science and profession attributes, the ECT scores increase, both after rewriting and hard debiasing. However, the ECT scores show a higher increase for the hard debiased model, suggesting an advantage of this method over neutral rewriting. For the computer science vs. childcare attributes however, both methods show a reduction in ECT scores, which could indicate that neither neutral rewriting nor hard debiasing are sufficiently affecting words in these semantic fields.

Clustering and Classification: Our results demonstrated that while the embeddings clearly encode gender information that is very salient to a binary classifier, rewriting with gender-neutral terminology has a more comprehensive effect than focusing on removing a limited 'gender subspace', as done by hard debiasing. In fact, hard debiasing improves the clustering accuracy, indicating that gender in word embeddings is more intricately encoded than can be captured by a gender subspace. These results mirror the findings of Gonen and Goldberg [26]. Both clustering and classification accuracy marginally decline in the model trained on gender-neutral text, as shown in Table 3a. The SVM shows a very high accuracy of 98% in separating words with male and female direct bias in the unchanged and hard-debiased model, which is reduced to 96% in the gender-neutral model.

Semantic Quality: The semantic quality of the word embeddings as measured by the SimLex 999 [30] and WordSim 353 [31] benchmarks dropped only minimally by at most 0.01 points after rewriting (Table 3b). Overall, these results fall only slightly behind larger embedding models. According to the SimLex 999 leaderboard, a Word2Vec model trained on one billion words of Wikipedia text reached a Spearman correlation of 0.372 , which is similar to our model.

The first limitation pertains to the size of the data used. Our corpus contains 250 million tokens, which is less than 25% of the training data size for a common embedding model [3]. We will explore in future work whether our findings hold for larger datasets and whether the measured reduction in gender stereotyping in the embedding model can translate to LLMs if fine-tuned on gender-inclusive text. Secondly, our research is focused on gender-inclusive language in English and not directly applicable to other languages. The NeuTralRewriter [14] was specifically developed for English, and since the specific characteristics of gender-neutral terminology are language-dependent, applying the method to other languages would require the development of a language-specific version of the Rewriter. We leave this to future research. A third limitation of our research lies in the erasure of word embeddings for he/she pronouns due to the replacement with they. However, since we are presenting a proof-of-concept study and the measurement of gender stereotyping is not dependent on these pronouns, we accepted this. Future work could rewrite pronouns only in a percentage of cases or only in cases where masculine/feminine pronouns are used generically. Lastly, our research is limited by a narrow focus on binary male and female genders when assessing model bias. There is a significant gap in NLP research regarding the incorporation of non-binary gender identities in both measuring and mitigating bias [32]. Due to the nature of this proof-of-concept study, we adhered to commonly employed binary metrics. Future work will need to examine progress made regarding the integration of non-binary gender identities in embedding models through inclusive terminology.

This research explored the effects of gender-neutral language on gender stereotyping and latent gender information in classic embedding models. We found that training on text with gender-neutral singular pronouns and role nouns effected a reduction in stereotyping as measured by WEAT [20] and ECT [21]. These reductions do not surpass those that can be achieved by targeted, post-hoc debiasing [19]. However, gender-neutral training data showed an advantage when measuring latent gender information in embeddings through classification and clustering. This demonstrates a more comprehensive effect of gender-neutral language in the removal of unnecessarily gendered associations, which is in line with the aims of gender-inclusive language.

While future work will need to investigate whether our results hold at scale and can be transferred to LLMs, our exploratory findings suggest that adjusting training data to be more gender-inclusive can improve gender representations in pre-trained models toward a more equitable conceptualization of gender. This research presents a promising approach to the incorporation of principles of genderinclusive language to ensure fairness and inclusivity in AI systems.