Introduction

This paper asks whether (i) female persons are less likely to be mentioned by newspaper journalists than male persons, and (ii) whether the propensity to mention female persons differs between female and male journalists. To answer these questions we analyse the entire article output of the Austrian daily newspaper Der Standard (https://www.derstandard.at) from the years 2021 and 2022. These articles are publicly available online. We additionally obtained the full names of the authors of all articles from Der Standard; this authorship information is nonpublic for most of the articles. Der Standard is the fourth-largest daily newspaper in Austria, having more than 500,000 daily readers [29].

This study employs a binary notion of gender, as does the language policy of Der Standard, which prescribes the sole use of feminine and masculine pronouns, effectively prohibiting the use of neopronouns. Therefore this study does not contribute to the abolishment of a binary notion of gender, an aim prominently championed within the digital humanities by Laura Mandell [20].

We employ pretrained models for natural language processing to automatically identify and enumerate female and male persons mentioned in article texts. In particular, we use named entity recognition and automated gender-assignment to first names. The respective methods are susceptible to bias. We therefore estimate the error profile of the gender assignment to first names and verify that there is no significant bias that might have distorted our analysis.

We statistically estimate the probability that a journalist mentions a female person when mentioning a person. We find that females are less likely to be mentioned by journalists. This finding holds for male as well as female authors, but we find it to be less pronounced in female journalists. These effects are present to different degrees in different editorial departments, with some departments not exhibiting the imbalance at all. The causal pathways leading to these effects could include statistical 'self-selection', whereby female authors and male authors have a differing propensity to report on certain issues, or female authors could tend to highlight the roles of female persons more than their male colleagues, even when reporting on the same issue.

Literature review. The issue of gender representation and gender inequality is lively discussed in the digital humanities, including computational linguistics [12], digital film studies [3,32], game studies [31], and computational literary studies [4,5,8,13,19,30,32]. Newspapers and magazines in particular have been studied with respect to gender representation and possible gender bias. Yun et al. found that women were given more space in online than in print journals and that, in a significant fraction of cases, women were portrayed in stereotypical ways [16]. Kian et al. analysed tennis news, finding that female reporters did not write more often about female athletes than their male colleagues but that female reporters tended to use more stereotypical descriptions [17]. Kozlowski et al. [18] analyse the magazines from an Argentinian publisher from 2008 to 2018 using topic modelling and find that the prevalence of thematic areas differs between magazines that target female readers and those that target male ones. They find that this gap is diminishing with respect to certain topics, such as 'family' and 'children', whereas it remains large in others, such as 'fashion' and 'horoscope'. The largescale analysis of Shor et al. [27], in which more than 20,000 prominent personalities of male and female gender and different (but matching) professions were searched in about 2,000 Englishlanguage newspapers, came to the conclusion that the reduced media coverage of women is not in line with the readers' interest, which does not favour prominent men over prominent women. They thus provide some evidence in favour of the hypothesis that newspapers and magazines foster stereotypes and gender bias.

The work most related to ours is due to Mateos de Cabo et al. [21], whose analysis of Spanish online newspapers found that females were more likely to be mentioned in female-authored articles, and to Shor et al. [28], whose analysis of about 2,000 news sources found that the fraction of females in articles increased from 19% in 1983 to 27% in 2008. The latter also found significant differences between editorial departments.

The digital humanities community has formulated a need to further the application of its methods to questions of gender. These voices include Miriam Posner [23], who criticized that gender-related work in the digital humanities does not receive sufÏcient attention, neither from the scholarly community nor from news outlets. In 2018 Susan Brown stated that a feminist perspective is largely lacking in the digital humanities, going as far as calling 'feminism' the 'f word', suggesting that feminist approaches are effectively silenced [6]. In 2019 Laura Mandell argued that studies on gender within the digital humanities would rather reproduce stereotypes than analyse them [20]. Coining the term 'data feminism' in 2020, D'Ignazio and Klein drew awareness to gender representation biases in the digital humanities and in other data-driven fields [10]. Although gender bias has been studied in various domains [9,25,15,30,19,11,24,12], we agree that a sufÏcient corpus of statistical results remains absent.

Data and methods

We start with the text and metadata of 87,032 articles, corresponding to the entire journalistic output of Der Standard between January 1, 2021 and December 31, 2022. The metadata of the articles includes authorship, publication date, title, and editorial department. We removed all articles from consideration that were written by a group of journalists or signed by a press agency. This restriction is in line with our interest in the behaviour of individual writers at a newspaper, as opposed to press agencies, group authorship, or anonymous authorship. We retain 36,204 articles.

Named entity recognition and gender-assignment. We use the Python package Gender-Guesser 0.4.0 to assign gender to the authors' given names. GenderGuesser uses a database of 40,000 gender-assigned first names to assign gender to a given name [2]. Since some given names, such as 'Andrea', 'Maria' or 'Robin', are not gender-exclusive, GenderGuesser returns 'mostly female', 'mostly male', and 'androgynous' in some cases. We assigned the first two of these categories to 'female' and 'male' respectively. We manually checked the genderassignments of all 1,571 authors and corrected three cases. We therefore treat the assignment of gender to the names of authors as certainly correct in the remainder of our analysis.

We used the Python package Flair 0.12.2 [1] to recognize personal names in the article texts using the named entity recognition model 'ner-german-large' [26]. In a first step we restrict attention to names consisting of a given name and a family name since the editorial policy of Der Standard prescribes the use of the full name at least once per article. The gender of the detected given names was then determined using GenderGuesser. Names such as 'Barack Obama', 'Viktor F. ', 'Angela Merkel', 'Nina H. ', and 'Luke Skywalker' were identified. In a second step we parse the articles for mentions of the identified persons using only a part of the full name. To clarify our counting method: An article mentioning two persons, the same male person 9 times and a female person once, is considered a text where 90% of mentioned persons are male.

We evaluated our extraction of full names in comparison to the manual counting of the number of female and male full names in 200 randomly selected articles (Table 2). We use binomial estimates to quantify the conditional accuracies of the automated extraction (Table 1). The fairness criterion of predictive parity requires the equality of the true positive rate for male and female cases [7]. We find that these two rates are numerically very similar, approximately 0.87 and 0.88, and that the hypothesis that they are equal can not be rejected (Binomial proportions test, 𝑝 = 0.68).

Generative model. Our main interest is the probability that an author, when mentioning a person, does mention a female person. Our task is complicated by the fact that the automated 2). detection of full names and the automated assignment of gender to the respective first names could be biased. The probability that we have access to is ℙ(detected as female|name detected).

We consider a simple generative model for our data (Figure 1), which illustrates that the events 'detected as female' and 'detected as male' do not allow the identification of the parameter of interest, that is 𝑝, unless certain assumptions are made. The two necessary assumptions are the absence of mix-ups, that is

ℙ(detected as female|male) = ℙ(detected as male|female) = 0,(1)

and unbiased non-detection, that is

ℙ(not detected|female) = ℙ(not detected|male).(2)

If Equations 1 and 2 hold, then ℙ(fem. name detected|name detected) = ℙ(fem. name mentioned|name mentioned). Statistical model. The details of our statistical approach are presented in Appendix A. Our basic modelling assumption is that the number of persons in an article is predetermined, but the respective journalist 'chooses' the gender of the mentioned persons independently from a Bernoulli distribution. The parameter of this Bernoulli distribution is specific to the subset of the data for which an estimate is desired. We believe that this model is sufÏcient to organise the data in a tractable and intuitive fashion. To be concrete: We estimate the probability that a journalist uses the name of a female person when using the name of a person. Note that our estimate does not equal the fraction of female persons in a certain subset of articles. Our estimate is descriptive of authors' behaviour, not of article output (see Appendix A). It is important to note that every author, regardless of the number of persons mentioned in their respective articles, has equal importance for our point estimates, but the differing degrees of uncertainty for different authors, caused by the different quantities of mentioned persons, are reflected in the interval estimates, that is in the confidence intervals. When we estimate probabilities specific to editorial departments, we employ a weighting scheme whereby the degree of membership of an author in an editorial department is taken into account. Consult Appendix A for details on our statistical approach.

Descriptive analysis

Of the 36,204 articles, 12,736 (35%) were written by a female author and 23,468 (65%) were written by a male author. Among the 1,571 unique authors 606 (39%) are female. The average author has contributed 23 articles to the dataset. The articles are of varying length, the median length being 3,859 characters (1st quartile = 2,560, 3rd quartile = 5,248). The mean of the fraction of females in an article equals 25%. The respective mean for the articles written by male writers equals 20%. On the other hand, the mean for the articles authored by female writers equals 33%. The empirical distributions of the fraction of female persons in an article are visualized in Figure 2. Note that we only consider articles that do mention at least one person. It is apparent that many articles that do mention persons do not mention females at all. This is true for female-as well as male-authored articles. It is evident that the distributions for female-and male-authored articles differ. The distribution for female-authored articles stochastically dominates the distribution for male-authored articles (McFadden's test [22], 𝑝 = 0.00). The distributions exhibit concentrations at the extremes. This illustrates that many articles solely mention persons of one gender.

Differentiating with respect to the editorial departments of Der Standard (Table 3), one finds that the department Family produces the smallest number of articles but has the largest fraction of female authorship. The largest number of articles is produced by the department Culture, which corresponds to roughly 13% of our data.

Statistical estimation

Our first interests are to estimate the probabilities that (i) an author who mentions a person does mention a female person, (ii) a female author who mentions a person does mention a female person, and (iii) a male author who mentions a person does mention a female person. The respective estimated probabilities are reported in Table 4 and visualised in Figure 3. While the probability that an author mentions a female person when mentioning a person is estimated to be roughly 27%, the respective estimate for female authors is roughly 46% and roughly 14% for male authors. The differences are highly statistically significant (𝑝 = 0.00). We conclude that there is strong evidence, at least in our data, that female journalists are more likely to mention female persons in their articles compared to male journalists.

To elucidate the differences between authors from different editorial departments, we estimate the probability that a journalist from a given department mentions a female person when mentioning a person. These estimates are reported in Table 5 and visualized in Figure 4. There are editorial departments whose output is likely to mention females, such as Female Standard and Family, and departments whose writers are unlikely to mention females, such as Automo- bile and Sport. These findings are in line with previous studies from different countries and languages [17,28,18].

To disentangle the effect of authors' gender and editorial departments, we stratify our analysis with respect to both. These estimates are visualized in Figure 5 and reported in Table 6, including hypothesis tests for the null-hypothesis that female and male authors behave identically. We obtain significantly different estimates for female and male authors for many edito- rial departments. The departments Opinion, Career, Female Standard, Science, Economy, Law, Culture, Lifestyle, Web, and Automobile exhibit highly significant differences with respect to author gender (𝑝 = 0.00). The departments Domestic, Government, Health, Realty, Travel, and Education do not exhibit statistically significant gender differences (𝑝 > 0.1).

Conclusion and caveats

We present some statistical evidence for the hypothesis that female and male journalists have differing propensities to mention female persons in their writing. This effect varies between editorial departments, at least in our data, and is stronger within certain editorial departments than across editorial departments. Further research is needed to elucidate whether the findings of the present study are driven by a mechanism through which female journalists write about different topics than male journalists. Even if this were true, it would remain ambiguous whether the ultimate cause is the issue-assignment policy within newsrooms, or a desire by journalists to write on topics featuring persons of a certain gender. The latter could also correspond to a conscious attempt by female journalists to highlight female persons in an attempt to counteract existing gender imbalances. We deem it an interesting avenue for further research to quantitatively elucidate the relationship between journalistic topics, gender representation, and authorship. If the observed differences were caused by different propensities in mentioning female persons even when reporting on the same topic, this would indicate gender-specificity in journalist's viewpoints. Finally we want to highlight that the present study is most certainly marred by the problem that many relevant and potentially confounding factors, such as topics, have not yet been taken into account. Therefore the present study is but an empirical quantification of a status quo.

where 𝑘 𝑖 denotes the number of persons detected as female in article 𝑖, 𝑚 𝑖 denotes the number of detected persons in article 𝑖, {𝑆 1 , ..., 𝑆 𝑙 } are disjoint subsets of the data {1, ..., 𝑛}, and the weight 𝑤 𝑖 equals the reciprocal of the number of natural persons detected in articles written by the respective journalist 𝑎 𝑖 , that is

𝑤 𝑖 = 1 ∑ 𝑗∶𝑎 𝑖 =𝑎 𝑗 𝑚 𝑗 .(4)

Note that we have discarded multiplicative constants from Equation 3. Weighted likelihood estimation is a well-established method in several circumstances [14]. The likelihood (Equation 3) is maximized at the parameter-values { p 1 , ..., p 𝑙 } given by p 𝑗 = ∑ 𝑖∈𝑆 𝑗 𝑤 𝑖 𝑘 𝑖 ∑ 𝑖∈𝑆 𝑗 𝑤 𝑖 𝑚 𝑖 .

(

)5

Under our choice of weighting (Equation 4), the maximum-likelihood estimates according to Equation 5 can be written as

p

This enables us to use a normal approximation to the distribution of p 𝑗 to construct confidence intervals.