We analyse gender representation in articles published by the Austrian daily newspaper 'Der Standard' in the years 2021 and 2022. We use named entity recognition and automated gender classification of first names to count the number of female and male persons in articles. The analysis reveals the dominance of male persons in article content. We find that female authors exhibit a significantly higher tendency to mention female persons in their articles.
are susceptible to bias. We therefore estimate the error profile of the gender assignment to first names and verify that there is no significant bias that might have distorted our analysis.
We statistically estimate the probability that a journalist mentions a female person when mentioning a person. We find that females are less likely to be mentioned by journalists. This ifnding holds for male as well as female authors, but we find it to be less pronounced in female journalists. These efects are present to diferent degrees in diferent editorial departments, with some departments not exhibiting the imbalance at all. The causal pathways leading to these efects could include statistical ’self-selection’, whereby female authors and male authors have a difering propensity to report on certain issues, or female authors could tend to highlight the roles of female persons more than their male colleagues, even when reporting on the same issue.
Literature review. The issue of gender representation and gender inequality is lively
discussed in the digital humanities, including computational linguisti1c2s][, digital film studies
[3, 32], game studies [31], and computational literary studie4s,[
The work most related to ours is due to Mateos de Cabo et a2l1.][, whose analysis of Spanish online newspapers found that females were more likely to be mentioned in female-authored articles, and to Shor et al. 28[], whose analysis of about 2,000 news sources found that the fraction of females in articles increased from 19% in 1983 to 27% in 2008. The latter also found significant diferences between editorial departments.
The digital humanities community has formulated a need to further the application of its
methods to questions of gender. These voices include Miriam Posne2r3][, who criticized that
gender-related work in the digital humanities does not receive sufÏcient attention, neither from
the scholarly community nor from news outlets. In 2018 Susan Brown stated that a feminist
perspective is largely lacking in the digital humanities, going as far as calling ’feminism’ the ’f
word’, suggesting that feminist approaches are efectively silenced6[]. In 2019 Laura Mandell
argued that studies on gender within the digital humanities would rather reproduce stereotypes
than analyse them [
We start with the text and metadata of 87,032 articles, corresponding to the entire journalistic output ofDer Standard between January 1, 2021 and December 31, 2022. The metadata of the articles includes authorship, publication date, title, and editorial department. We removed all articles from consideration that were written by a group of journalists or signed by a press agency. This restriction is in line with our interest in the behaviour of individual writers at a newspaper, as opposed to press agencies, group authorship, or anonymous authorship. We retain 36,204 articles.
Named entity recognition and gender-assignment. We use the Python package GenderGuesser 0.4.0 to assign gender to the authors’ given names. GenderGuesser uses a database of 40,000 gender-assigned first names to assign gender to a given name [2]. Since some given names, such as ‘Andrea’, ’Maria’ or ‘Robin’, are not gender-exclusive, GenderGuesser returns ‘mostly female’, ‘mostly male’, and ‘androgynous’ in some cases. We assigned the first two of these categories to ‘female’ and ‘male’ respectively. We manually checked the genderassignments of all 1,571 authors and corrected three cases. We therefore treat the assignment of gender to the names of authors as certainly correct in the remainder of our analysis.
We used the Python package Flair 0.12.21[] to recognize personal names in the article texts using the named entity recognition model ‘ner-german-larg2e6’][. In a first step we restrict attention to names consisting of a given name and a family name since the editorial policy of Der Standard prescribes the use of the full name at least once per article. The gender of the detected given names was then determined using GenderGuesser. Names such as ‘Barack Obama’, ‘Viktor F.’, ‘Angela Merkel’, ‘Nina H.’, and ’Luke Skywalker’ were identified. In a second step we parse the articles for mentions of the identified persons using only a part of the full name. To clarify our counting method: An article mentioning two persons, the same male person 9 times and a female person once, is considered a text where 90% of mentioned persons are male.
We evaluated our extraction of full names in comparison to the manual counting of the
number of female and male full names in 200 randomly selected articles (T2a)b.lWee use binomial
estimates to quantify the conditional accuracies of the automated extraction (Tab1l).eThe
fairness criterion of predictive parity requires the equality of the true positive rate for male and
female cases [
Generative model. Our main interest is the probability that an author, when mentioning a person, does mention a female person. Our task is complicated by the fact that the automated name is male detection of full names and the automated assignment of gender to the respective first names could be biased. The probability that we have access to is
ℙ(detected as female|name detected).
We consider a simple generative model for our data (Figu1)r,ewhich illustrates that the events ’detected as female’ and ’detected as male’ do not allow the identification of the parameter of interest, that is , unless certain assumptions are made. The two necessary assumptions are the absence of mix-ups, that is
ℙ(detected as female|male) = ℙ(detected as male|female) = 0, and unbiased non-detection, that is If Equations 1 and 2 hold, then
ℙ(not detected|female) = ℙ(not detected|male). ℙ(fem. name detected|name detected) = ℙ (fem. name mentioned|name mentioned). (1) (2) It is not necessary to assume that non-detection does not occur, but that non-detection is unbiased. The empirical probabilities corresponding to those in Equat2ioanre similar, approximately 0.10 and 0.13, and the hypothesis that they are equal can not be rejected (Binomial proportions test, = 0.31 ). As Table 1 reports, the empirical probabilities for mix-ups equal approximately0.00 and 0.02. We feel that these values are sufÏciently low to assume that Equation 1 holds. In Table2 the fraction of female persons among the mentioned persons equals 160/(160 + 715) ≈ 0.18while the fraction of names classified as female among the classified names equals143/(143 + 624) ≈ 0.1.9This numerically illustrates the absence of bias. Statistical model. The details of our statistical approach are presented in AppendAi.xOur basic modelling assumption is that the number of persons in an article is predetermined, but the respective journalist ’chooses’ the gender of the mentioned persons independently from a Bernoulli distribution. The parameter of this Bernoulli distribution is specific to the subset of the data for which an estimate is desired. We believe that this model is sufÏcient to organise the data in a tractable and intuitive fashion. To be concrete: We estimate the probability that a journalist uses the name of a female person when using the name of a person. Note that our estimate does not equal the fraction of female persons in a certain subset of articles. Our estimate is descriptive of authors’ behaviour, not of article output (see AppendAi)x. It is important to note that every author, regardless of the number of persons mentioned in their respective articles, has equal importance for our point estimates, but the difering degrees of uncertainty for diferent authors, caused by the diferent quantities of mentioned persons, are reflected in the interval estimates, that is in the confidence intervals. When we estimate probabilities specific to editorial departments, we employ a weighting scheme whereby the degree of membership of an author in an editorial department is taken into account. Consult AppenAdixfor details on our statistical approach.
Of the 36,204 articles, 12,736 (35%) were written by a female author and 23,468 (65%) were written by a male author. Among the 1,571 unique authors 606 (39%) are female. The average author has contributed 23 articles to the dataset. The articles are of varying length, the median length being 3,859 characters (1st quartile = 2,560, 3rd quartile = 5,248).
The mean of the fraction of females in an article equals 25%. The respective mean for the articles written by male writers equals 20%. On the other hand, the mean for the articles authored All articles
Male-authored articles
Female-authored articles by female writers equals 33%. The empirical distributions of the fraction of female persons in an article are visualized in Figur2e. Note that we only consider articles that do mention at least one person. It is apparent that many articles that do mention persons do not mention females at all. This is true for female- as well as male-authored articles. It is evident that the distributions for female- and male-authored articles difer. The distribution for female-authored articles stochastically dominates the distribution for male-authored articles (McFadden’s test [22], = 0.00 ). The distributions exhibit concentrations at the extremes. This illustrates that many articles solely mention persons of one gender.
Diferentiating with respect to the editorial departments oDfer Standard (Table3), one finds that the department Family produces the smallest number of articles but has the largest fraction of female authorship. The largest number of articles is produced by the department Culture, which corresponds to roughly 13% of our data.
Our first interests are to estimate the probabilities that (i) an author who mentions a person does mention a female person, (ii) a female author who mentions a person does mention a female person, and (iii) a male author who mentions a person does mention a female person. The respective estimated probabilities are reported in Tab4laend visualised in Figur3e. While the probability that an author mentions a female person when mentioning a person is estimated to be roughly27%, the respective estimate for female authors is rough4l6y% and roughly14% for male authors. The diferences are highly statistically significant (= 0.00 ). We conclude that there is strong evidence, at least in our data, that female journalists are more likely to mention female persons in their articles compared to male journalists.
To elucidate the diferences between authors from diferent editorial departments, we estimate the probability that a journalist from a given department mentions a female person when mentioning a person. These estimates are reported in Tab5leand visualized in Figur4e. There are editorial departments whose output is likely to mention females, such as FemSatalnedard and Family, and departments whose writers are unlikely to mention females, such as Automo0.81 0.21 0.91 0.92
Opinion International
Domestic
Career
Government Female Standard
Health
Science Economy
Sport
Family Panorama
Realty
Law Culture
Future Lifestyle
Travel
Web
Education Automobile 0
0.14 0.27 0.46
Probability that author mentions a female person when mentioning a person
1
bile and Sport. These findings are in line with previous studies from diferent countries and
languages [
To disentangle the efect of authors’ gender and editorial departments, we stratify our analysis with respect to both. These estimates are visualized in Figur5eand reported in Table6, including hypothesis tests for the null-hypothesis that female and male authors behave identically. We obtain significantly diferent estimates for female and male authors for many editoOpinion, fem. auth.
Opinion, male auth.
International, fem. auth.
International, male auth.
Domestic, fem. auth.
Domestic, male auth.
Career, fem. auth.
Career, male auth.
Government, fem. auth.
Government, male auth.
Female Standard, fem. auth.
Female Standard, male auth.
Health, fem. auth.
Health, male auth.
Science, fem. auth.
Science, male auth.
Economy, fem. auth.
Economy, male auth.
Sport, fem. auth.
Sport, male auth.
Family, fem. auth.
Family, male auth.
Panorama, fem. auth.
Panorama, male auth.
Realty, fem. auth.
Realty, male auth.
Law, fem. auth.
Law, male auth.
Culture, fem. auth.
Culture, male auth.
Future, fem. auth.
Future, male auth.
Lifestyle, fem. auth.
Lifestyle, male auth.
Travel, fem. auth.
Travel, male auth.
Web, fem. auth.
Web, male auth.
Education, fem. auth.
Education, male auth.
Automobile, fem. auth.
Automobile, male auth.
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Probability that author mentions a female person when mentioning a person gender mentions a female person when mentioning a person (with 95%-confidence intervals). rial departments. The departments Opinion, Career, FemaSlteandard, Science, Economy, Law, Education do not exhibit statistically significant gender diferences ( > 0.1 ).
Culture, Lifestyle, Web, and Automobile exhibit highly significant diferences with respect to author gender (= 0.00 ). The departments Domestic, Government, Health, Realty, Travel, and
We present some statistical evidence for the hypothesis that female and male journalists have difering propensities to mention female persons in their writing. This efect varies between editorial departments, at least in our data, and is stronger within certain editorial departments than across editorial departments. Further research is needed to elucidate whether the findings of the present study are driven by a mechanism through which female journalists write about diferent topics than male journalists. Even if this were true, it would remain ambiguous whether the ultimate cause is the issue-assignment policy within newsrooms, or a desire by journalists to write on topics featuring persons of a certain gender. The latter could also correspond to a conscious attempt by female journalists to highlight female persons in an attempt to counteract existing gender imbalances. We deem it an interesting avenue for further research to quantitatively elucidate the relationship between journalistic topics, gender representation, and authorship. If the observed diferences were caused by diferent propensities in mentioning female persons even when reporting on the same topic, this would indicate gender-specificity in journalist’s viewpoints. Finally we want to highlight that the present study is most certainly marred by the problem that many relevant and potentially confounding factors, such as topics, have not yet been taken into account. Therefore the present study is but an empirical quantification of a status quo.
The authors thank Martin Kotynek and Werner Weichselberger froDmer Standard for their support. ity via the Internet? An Analysis of Women’s Presence in Spanish Online Newspapers”.
In: Sex Roles 70.1 (2014), pp. 57–71. doi: 10.1007/s11199-013-0331-y.
D. McFadden. “Testing for Stochastic Dominance”. InS:tudies in the Economics of Uncertainty. Springer, 1989, pp. 113–134.
M. Posner. “What’s Next: The Radical, Unrealized Potential of Digital Humanities”. In: Debates in the Digital Humanities 2016. University of Minnesota Press, 2016, pp. 32–41. Game Culture: Exploring Gender specific Vocabulary in Video Game Magazines”. In: Proceedings of the Digital Humanities in the Nordic Countries 5th Conference (DHN 2020)
M. Schumacher and M. Flüh. “Made to Be a Woman: A case study on the categorization of gender using an individuation-based approach in the analysis of literary texts”. In: Digital Humanities Quarterly 17.3 (2023). url: https://www.digitalhumanities.org/dhq/v [26] S. Schweter and A. Akbik. “FLERT: Document-level features for named entity recogni[27] E. Shor, A. van de Rijt, and B. Fotouhi. “A Large-Scale Test of Gender Bias in the Media”.
In: Sociological Science 6 (2019), pp. 526–550. doi: 10.15195/v6.a20. [28] E. Shor, A. van de Rijt, A. Miltsov, V. Kulkarni, and S. Skiena. “A Paper Ceiling: Explaining the Persistent Underrepresentation of Women in Printed News”. AInm:erican Sociological Review 80.5 (2015), pp. 960–984. doi: 10.1177/0003122415596999. [29] Statista. Österreich Tageszeitungen nach Anzahl der Leser. 2022. url: https://de.statista.c ahl-der-leser. / des Computerspiels. 2021.
Big Bang Theory. 2023. [30] T. Underwood, D. Bamman, and S. Lee. “The Transformation of Gender in English[31] T. Unterhuber.Männlich codiert?: Annäherung an eine Medien- und Geschlechtergeschichte [32] E.-M. Venzmer.”Oh, the [digital] humanities¡‘ – Eine quantitative Gender-Analyse von The
Maximum likelihood estimate.
We consider the weighted likelihood
=1 ∈ ( 1, ..., ) ∝∏ (∏ ( (1 − ) − ) ) ,
(3) respective journalist , that is where denotes the number of persons detected as female in artic l, e denotes the number of detected persons in articl e, { 1, ..., } are disjoint subsets of the data{1, ..., } , and the weight equals the reciprocal of the number of natural persons detected in articles written by the 3) is maximized at the parameter-value{s 1̂, ..., ̂ } given by Note that we have discarded multiplicative constants from Equatio3n. Weighted likelihood estimation is a well-established method in several circumstanc1e4s][. The likelihood (Equation =
1 ∑∶ = ̂ = ∑
∈ ∑ ∈
. .
Under our choice of weighting (Equation4), the maximum-likelihood estimates according to Equation 5 can be written as ̂ = ∑
∈ ∑ ∈ , , = ∑ ∈ 1 , ∈ ∑ , , , , where denotes the set of unique authors, , denotes the number of persons detected in texts of author in subset , denotes the number of persons detected in texts of author , and , denotes the number of female persons detected in articles by auth orin subset . This is but the weighted mean of the naive per-author estimates for the subset, that is, / , , weighted by the ’degree of membership’ of author in subset , that is by , / . Note that this estimator is such that multiplying all data from a certain author by a constant does not change the estimate. In the special case of a single subset equal to the entirety of the data, the estimator takes the form =̂ 1 || ∈ ∑ ( ∑
∶ = ∑ ∶ = ) , which is but the arithmetic average of the per-author relative frequencies.
Confidence intervals. The variance of ̂ equals ( ̂ ) =
1 (∑∈
2 ) ∈ ∑ 2 ( )
(∑∈ ) 2 for the variance o f ̂ is where ∼ binomial( , )and hence ( ) = (1 − ) . Therefore the plug-in estimator This enables us to use a normal approximation to the distributio n ôtfo construct confidence intervals. (4) (5) (6) Hypothesis tests. To test null-hypotheses of the form = ′ , we construct a test using the test statistic where the variances are computed according to Equatio6n.
Model 0.011 0.013 0.010 0.048 0.032 0.035 0.023 0.021 0.016 0.014 0.081 0.023 0.030 0.040 0.012 0.050 0.021 0.058 0.024 0.056 0.037 [0.218,0.263] [0.178,0.229] [0.152,0.191] [0.211,0.399] [0.202,0.328] [0.691,0.827] [0.289,0.378] [0.267,0.348] [0.153,0.217] [0.108,0.163] [0.355,0.674] [0.211,0.302] [0.115,0.233] [0.124,0.283] [0.289,0.337] [0.220,0.417] [0.313,0.396] [0.159,0.388] [0.104,0.200] [0.293,0.511] [0.000,0.138] p-value, 0 ∶ = 0.052 0.023