This study delves into the representation of gender in French literature from 1800 to the present, aiming to assess the prevalence of gender stereotypes in the description of 昀椀ctional characters. By employing an annotated corpus and statistical modeling techniques, the research explores how authors perpetuated gender biases while shaping characters and narratives. The 昀椀ndings reveal signi昀椀cant linguistic patterns that reinforce gender norms, with women being characterized by emotional and physical attributes, while men are associated with action and agency.
Consequently, this research employs the masculine/feminine dichotomy to capture prevalent gender constructs in French literature.
Gender bias in 昀椀ctional characterization refers to the systemic tendency to portray female characters in a derogatory or disadvantageous manner within works of 昀椀ction. This bias can manifest through various means, such as reinforcing stereotypes, limiting agency or complexity, or presenting characters in a demeaning or objectifying light based on their gender. In the seminal study conducted by Jockers and Kirillo1昀[0], the authors explored these biases by investigating the association between verbs and gendered pronouns and nouns in characterizations. They concluded that gender serves as a confounding factor in character agency and behavioral representations. Based on these 昀椀ndings, Underwood, Bamman, and Lee17[] revealed a disproportionate representation of male characters by male authors, whereas female authors tended to provide a more equal representation of both genders. The key 昀椀nding indicated a decline in gender-based distinctiveness of 昀椀ctional characters over time, suggesting a blurring of gender boundaries in contemporary 昀椀ction.
Following this line of research, the aim of this study is to identify consistent patterns of
negative portrayals of female characters or the perpetuation of stereotypes, which may indicate
gender bias. In alignment with Naguib, Delaborde, Andrault, Bekolo, and Semin1c5k],[we will
concentrate on character space detection1[
The 昀椀rst part of this paper is about the creation of a large annotated corpus using the predictions of a classi昀椀er. On this large annotated corpus, three main experiments were made to determine whether characters are described di昀erently depending on their gender. First, we focused on agency using linguistic features: we tried to 昀椀nd if women were more o昀琀en grammatical objects of the sentence than men. The results are presented in sectio3.n2. The second experiment, detailed in section3.1, was to use lemmatized words linked to a character to predict its gender: if it works, it would mean that there are ’gendered’ words that help the classi昀椀cation. Finally, we explored the vocabulary in more details to 昀椀nd out which words were more associated to women or to men in sectio3n.3.
We used the Chapitres1 corpus for our analysis. It is comprised of 2942 French novels in XMLTEI2 (Text Encoding Initiative) encoding from 1811 to 2020, put together in the context of the
2TEI Consortium, eds. TEI P5: Guidelines for Electronic Text Encoding and Interchange. Version 1.0. TEI Consortium. http://www.tei-c.org/Guidelines/P5/. french ANR3 project ”Chapitres”4. An overview of the ”Chapitres” corpus is given in tabl1e. Sentences and tokens were computed using the Spacy 8[] fr_dep_news_trf mode5l.
A 昀椀rst part of this work was to manually annotate the gender of the 10 main characters from 100 novels picked randomly in the corpus Chapitres. The four classes are: • ’m’ for masculine characters • ’f’ for feminine characters • ’p’ for plural characters • ’u’ (unde昀椀ned) when there were not enough clues
A 1000 lines (one for each character) tsv 昀椀le was created. To each character, the most frequent mentions, and related adjectives and verbs were associated. (see sectio2n.3 for the extraction of related adjectives and verbs). A mention of a character is a linguistic item referring to this given character, notably proper names and pronouns.
Annotation was performed by three annotators who are native speakers. The gender was
attributed depending on the mentions (pronouns, names, ...), and the gender of the adjectives.
The latter are useful for 昀椀rst person narrators, as mentions do not carry gender marks. The
dataset was split evenly across all annotators to ensure that every character would be annotated
twice. This allowed us to compute an inter annotator agreement (IAA) using NLTK1’s3][
implementation of Cohen’s [
For each character, the goal was to obtain all its mentions in the book, and all adjectives and verbs having a dependency link with a mention of this character. A 昀椀rst step was to associate 3FNA, French National Agency 4Website of the the project:https://chapitres.hypotheses.org 5The model is described at the following URhLt:tps://spacy.io/models/fr#fr_dep_news_trf each token with 2 informations: is this token a character, and what gender is this character. To store these informations, we created a tsv format with the columns presented in ta4b.le
By using an NLP pipeline speci昀椀cally tuned for novels, (fr-BookNLP, part of the multilingual
BookNLP project [
This NLP pipeline allowed us to associate column 7 (character id) to the 昀椀rst column. The 10
most important characters of each book were considered, thus column 7 contains character ids
from 1 to 10 when the token is a mention of a character. Then, all sentences from our corpus
were parsed with Spacy 8[
In table4, a sentence fromClaudine en ménage by Colette is presented. The character with id 1 was annotated as a female and the one with id 4 as a male, as can be seen in the last two columns. ’Renaud’ is the subject of the sentence. Character one’s 昀椀rst mention is ’me’, which is the object of the sentence (see 6th column).
In a second step, the information given by Spacy was used to de昀椀ne the lists of related adjectives and verbs. For each character, we retrieve all tokens that are linked to a mention of this character (each token whose index correspond to the head that governs the mention), if these tokens are adjectives or verbs.
To represent the features (the adjectives, verbs or mentions related to characters) as vectors, we decided to use a Bag of Words (BoW) representation. Thus, each line corresponds to a character, and each column represents a word. In each cell is found the number of occurrences of the word corresponding to the column for the character given by the line, divided by the total number of words associated to this character. This allows to have the relative frequency for each word. Because of the over-representation of masculine characters in our corpus (see table3), we decided to balance the classes.
We trained a classi昀椀er to predict the gender of characters and used these predictions as
annotations for our experiments described below. We employed a Random Forest classi7昀椀e]r, [
implemented in scikit-learn1[
Using a Bag of Words representation of the most frequent character mentions as input, the classi昀椀er proved e昀ective in predicting gender based on crucial words such as ’il’ (he) and ’elle’ (she), as depicted in Figure1.
We then inferred gender across all our corpora, which encompasses a vast array of 29,490 characters, holds signi昀椀cant importance in unveiling broader patterns and trends in gender representations within the literary landscape. By examining a wide range of characters from various genres, time periods, and cultural backgrounds, we aim to discern recurrent patterns and subtle nuances in character depictions, shedding light on the implicit gender constructs prevalent throughout literary history.
Following Underwood, Bamman, and Lee17[], we tried to reproduce on our corpus of French novels some of their 昀椀ndings. For this experiment, we use a similar method to the one used to annotate the corpus, described in2.4. We adapted the features used to predict gender: instead of using mentions, we used adjectives and verbs.
A major di昀erence between French and English that we had to take into account for this experiment is the gender mark on certain adjectives and participles. For example, in the French sentence below there are two signs that the speaker is feminine, while there are none in the English equivalent: ”Je suis heureuse d’y être alleé” ”I’m glad I went there”
If a classi昀椀er is able to correctly predict whether a character is feminine or masculine due to these marks, there is no evidence that gender stereotypes helped the classi昀椀cation. Therefore, we chose to use lemmas instead of word forms. Thus, in the example above, the gendered words would become ’heureux’ and ’aller’. This way, if the classi昀椀er achieves an accuracy which is signi昀椀cantly higher than the baseline, it will be because it used sociological features instead of grammatical ones.
m’ (me) mmmee(M(mres))
mon (my) du (ø / some) ses (his / her) une (a / one)
ma (my) son (his / her)
je (I) sa (his / her) madame (Madam / Missus)
cette (this) lui (him / her) le (the) la (the) il (he) elle (she) 0
50 feature weight (×103) 100
In a literary setting, character agency refers to the capacity of a character to take intentional and autonomous actions within the narrative. It is the degree to which a character is portrayed as an active agent who drives the events and makes choices that impact the story’s progression. Characters with low agency may be more passive or reactive, in昀氀uenced by external forces or events rather than actively driving the story. They might have limited control over their circumstances or be more prone to being acted upon rather than taking independent action. To evaluate a character’s agency, we compared the number of mentions of this character that are subject of the sentence, and the ones that are object. To retrieve this information, we used Spacy as dependency parser.
To compare the degree of agency of di昀erent characters, we created a metric. Thaegency score, between -1 and 1, corresponds to:
With nsubj the number of occurrences of a character or group of characters as subject of the sentence, and obj the number of occurrences of a character or group of characters as object of the sentence.
The higher the score, the more o昀琀en the character (or group of characters) holds the position of subject of the phrase, and thus has an active role.
3.1.1. For the corpus annotation Resulting accuracies range from 60.25% to 91.3%, as described in Tab5l.eThe classi昀椀er demonstrated a good accuracy rate when using the mentions as features, achieving 91.3%. The errors are mostly due to inaccuracies in the identi昀椀cation of characters by BookNLP: it happens sometimes that a character has both feminine and masculine mentions because two di昀erent characters were mixed up. However, we considered that 91.3% accuracy was enough to annotate the whole corpus with this method. 3.1.2. For the bias analysis In our 昀椀rst experiment (second line in the table), adjectives are used to predict the gender of the characters they describe. We ran the same classi昀椀er using lemmas instead of words for comparison (third line of the table). The accuracy using words is 0.782, which is higher than the 0.602 accuracy obtained with lemmas This may indicate that the classi昀椀er relied on grammatical gender marks when adjectives were not lemmatized.
It is also interesting to notice that the results are better when using only adjectives when word forms are used (0.782 with adjectives against 0.74 with adjectives and verbs), whereas it is the contrary when using lemmas (0.602 with adjectives against 0.612 with adjectives and verbs). A possible explanation is that adjectives bear a gender mark more than verbs. This would mean that the classi昀椀er relies a lot on these gender marks when it can.
Now, the results of the classi昀椀er using lemmas are interpretable. In tab6leis presented the confusion matrix obtained with the BoW representation containing lemmatized adjectives and verbs as input to a Random Forest. Because the two classems ale and female were balanced, the baseline is 0.5. The classi昀椀er is able to predict the gender of a character given the adjectives and verbs related to it with an accuracy of 0.612. The features that were most useful for the classi昀椀er to classify the characters are presented in 昀椀gure2, and will be discussed later. Eventually, this result indicates that male and female characters are not characterized in the same way.
The results of our computations show that masculine characters tend to be a little bit more ’agent’ than feminine ones, but the di昀erence is not striking. Indeed, we can see in table7 that men are more o昀琀en subjects than women (85% of the time against 82%) while they are less o昀琀en objects, but their agency scores are close. However, the small di昀erence in the agency scores might imply that the descriptions of male characters exhibit slightly higher agency than those of female characters.
Using the method described in section2.3, we were able to obtain the verbs and adjectives having a dependency link with a given character. From this vocabulary, our goal was to determine whether some words were associated to one gender more than to the other.
As seen in 昀椀gure 4, we can see that a common stereotype is to associate women to love and passion, and men to action. We wanted to investigate about such well-known stereotypes, to see if they also appear through words in the corpus. A simple and signi昀椀cant example is the verb ’aimer’ (to love): we found that this verb occurs at least once for 44% of the female characters, against 25% for the male characters.
To visualize the results, plots were made with, for each verb, whether it is more associated to women or to men in the corpus. More precisely, the values correspond to: − +
With masc the proportion of masculine characters to which the given verb is related in the corpus, andfem the proportion of feminine characters. Thus, the more positive the values are, the more the word is associated to men, and negative with women. To avoid imprecision due to lack of data, verbs occurring for less than 1% of the characters were not considered. 3.3.1. Gendered di昀erentiation through action verbs This experiment was, among others, used to have another approach of agency, more focused on the lexicon than on the dependencies. Two kinds of verbs were used: action verbs (昀椀gure 3), and emotion verbs (昀椀gure 4). The lists are quite arbitrary: the categories ’action verb’ and ’emotion verb’ are not 昀椀xed. However, the list of action verbs contains verbs that show an in昀氀uence on the narration. We considered that the more a character or a gender was related to these verbs, the more he is an agent of the narration and plays an active role in it. Conversely, emotion verbs are more static and focus on the mind.
When comparing 昀椀gures 3 and 4, it appears that most of the ’action verbs’ are considered as more masculine, and most of the ’emotion verbs’ occur more for feminine characters than for masculine ones. Among these verbs conveying an emotion, those related to love are clearly more feminine: ’aimer’ (to love), ’adorer’ (to really like, to adore).
The mean of the proportion of masculine and feminine characters linked to a certain category of verbs or adjectives was computed to compare more precisely the two genders. The results, presented in table8, correspond (for each category of adjectives or verbs) to: ∑#= 1 %
#
Action verbs occur in average for 3.66% of masculine characters, against 3.45% of feminine characters. The di昀erence is slight; however, it is bigger when looking at the emotion verbs: 9.11% of men are, in average, linked to an emotion verb, against 12.26% of women. 0.2
0 −0.2 3.3.2. Other comparisons We created a non-exhaustive speech verbs list to see which speech verbs are more associated to a gender than to the other. Figure5 presents the results. Some interesting observations can be made: 昀椀rst, the verb ’crier’ (to shout) is more masculine, as opposed to ’murmurer’ (to whisper), which is more feminine. This seems to refer to the stereotype associating women to quietness and men to loudness.
Similarly, gender-related adjectives also contribute to reinforcing stereotypes in French literature. In 昀椀gure 6, adjectives such as ”joli” (pretty) and ”doux” (gentle) are commonly used to describe female characters, emphasizing their appearance and nurturing qualities. While adjectives such as ”vieux” (old) and ”gros” (fat) are more frequently applied to male characters, their antonyms ”jeune” (young) and ”昀椀n” (thin) are more associated to women. This opposition seems to refer to the valorization of youth and thinness for women.
Considering the model that uses lemmas to predict the gender (third model in ta5b) lseo it cannot use gender evidence from word, we want to discuss here the possibility that these features show gender biases. The discriminant features of our model, presented in 昀椀gu2r,eare the features that helped the most our classi昀椀er to predict the gender of a character.
Some of these lemmas are particularly associated to women or to men, as showed in section 3.3. On the 昀椀rst hand, the verb ’aimer’ (to love) is much more associated to feminine characters, jolib(eplr(ebtetdayuo)utixfu(ls)wlaeiedt)(puâglleyp)(eptaitle(s)mdujaerlul()nhear(dy)oufning)g(mtrhaainnidg)r(eta(sllk)ingnryo)sv(ifeautx)(old) and is also the fourth most useful feature to determine if a character is feminine or masculine (see 昀椀gure 2). Verbs commonly associated with female characters include ”pleurer” (to cry), ”reprocher” (to reproach), ”murmurer” (to murmur), and ”rire” (to laugh). These verbs tend to emphasize emotional expressions and portray women as more emotive and sensitive.
On the other hand, verbs associated with male characters include ”tirer” (to pull), ”découvrir” (to discover), and ”expliquer” (to explain). These verbs o昀琀en depict actions and intellectual pursuits, implying that male characters are more active and analytical in their roles.
Our approach is subject to the inherent accuracy limitations of many NLP and machine learning algorithms used, including fr-BookNLP, Spacy, and our Random Forest, all of which are prone to making errors.
As mentioned, the classi昀椀er used to predict the gender of the 29,490 characters achieves a 0.91 accuracy on the set of 1000 annotated characters. This means that approximately 9% of the characters on which the experiments were made do not have the correct label.
Most of the time, the errors are certainly due to the imprecision of the retrieved characters. Indeed, when doing the manual annotation, 9% of the characters were classi昀椀ed as ’unde昀椀ned’ because the mentions referred in fact to di昀erent characters, as showed in tabl2e.
A limitation of this study lies in the fact that the temporal dri昀琀 of gender bias has not been explored. Our intention was to develop methods to delve into this question using a large corpus and identify the associated challenges. Future research will speci昀椀cally focus on this aspect, aiming to assess the resistance of these stereotypes to temporal shi昀琀s and explore their evolution within 昀椀ction.
In conclusion, this article focuses on the representation of gender in French 昀椀ction, from 1800 to the present day. By examining gender bias prevalent in French literature, the study sheds light on how authors perpetuated gender stereotypes while shaping their characters and narratives. The study highlights the importance of critically examining such representations to foster a more inclusive and diverse literary landscape.
Our statistical modeling reach from 61% to 91% accuracy, depending on the features given to the classi昀椀er (with or without obvious gender marks). We employed it to infer the gender of 29490 characters, on which we can run some experiments. This allowed us to study how each gender is portrayed in French literature.
The analysis of French literature revealed certain gender stereotypes that are re昀氀ected in the language used to describe female and male characters. These linguistic patterns in the portrayal of characters in French literature re昀氀ect underlying gender bias and perpetuate traditional gender norms and roles. For example, we found that women get characterized with far more physical and emotional adjectives and verbs than men. These last were more linked with action verbs, indicating a stronger agency for male characters.
This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19P3IA0001 (PRAIRIE 3IA Institute). Jean Barré’s PhD is supported by the EUR (Ecole Universitaire de Recherche) Translitterae (programme “Investissements d’avenir” ANR-10- IDEX-000102 PSL and ANR-17-EURE-0025).
Figure 7 shows the distribution of novels over time broken out by author’s gender. The Chapitres corpus spans from 1810 to 2020, with a notable concentration of novels in the latter half of the 19th century. The proportion of female writers remains between 15 and 20% of the literary production, with a slight increase towards the end of the corpus, reaching about 30%. The corpus includes 714 authors, with 149 (21%) being female writers, contributing to a total of 626 novels (27%). 250 200 150 100 50 0 male writers female writers
We evaluated Fr-BookNLP coreference pipeline using three metrics used in state of the art coreference evaluation. All three evaluate the correctness of coreference chains and how well
None of them are really relevant for coreference at the novel’s scale, so we follo3w]ed [ coreference evaluation, averaging those three metrics. This version of Fr-BookNLP is ten points behind English evaluation. tion Evaluation of the fr-BookNLP pipeline, scores for character mentions detection and coreference resolu
Metrics 43,5
Coreference resolution is still hard for long texts. An issue is that characters can get duplicated, that is, the same character is detected in multiple coreference chains. While problematic at 昀椀rst glance, our primary objective is not to retrieve unique and distinct characters but a proxy for characterization as a whole. The focus is more on character’s gender representation and the overall impact of these characters on the literary landscape, rather than identifying separate and non-repeating characters.