Fairness interventions require access to sensitive attributes of candidates applying for a job, which might not be available due to limitations imposed by data protection laws. In this work we propose using a pre-processing technique to create counterfactual representations of the candidates that lead to a more diverse ranking with respect to intersectional groups. To be compliant with data protection laws we propose to train a model on the fairer representations and apply the model at inference time without having access to the sensitive attributes of the candidates. In experiments on the BIOS dataset, we ifnd this approach can improve the diversity of recommendations at top-ranked positions without harming performance.
Recruiters increasingly rely on automatic hiring systems to process the large amount of applications received for a job. We define an algorithmic hiring system to be a candidate recommendation system that recommends a ranked ing an objective automatic hiring system one would think that the hiring process is fair. However, such systems encode stereotypes and biases that already exist in the recruitment industry [5, 9, 16, 22], leading to actions that discriminate against minority groups [2, 11, 21].
Considering that the attention of recruiters decreases with the position of the candidate in the rank [13], the candidates at top positions are more likely to be considered for an interview. To combat existing disparities in the recruitment industry and avoid that they are perpetuated by the system, we aim to apply a fairness intervention to obtain a diverse ranking of candidates, in terms of sensitive attributes at top positions. According to the
access to special categories of sensitive attributes is limited [1, 23]. Exceptions of special sensitive attributes are gender and age. This limits the choice of fairness interventions that can be used in practice in recruitment.
Existing fairness interventions can be categorized as pre-processing, in-processing and post-processing methods [29]. Pre-processing methods aim to debias the data nEvelop-O method for the following reasons: (i) it showed promising results even without knowing the sensitive attributes at inference time, (ii) it provides a working framework for intersectional groups, and (iii) it complies with transparency requirements towards recruiters, candidates and audit companies. Unlike adversarial methods, it is easy (a) Model 1 (b) Model 2 to explain how the new representations are created and, thus, how the ranking of the candidates is generated. Figure 1: Causal models describing the data with sensitive By creating counterfactual representations of candidates, attributes gender (G) and nationality (N), non-sensitive atwe aim to have more diversity among the top candidates tributes (X), utility scores (Y), and occupation (O). of the ranked list, thus increasing the likelihood of the protected groups to be considered for an interview. Our work focuses on the intersectional groups created by gen- on the data, we have to determine a reference group toder and nationality. It is important to take into account wards which we want to transform the candidates in a intersectional groups, as candidates belonging to multi- counterfactual world. The idea is to replace the values of ple protected groups are more likely to be discriminated sensitive attributes with reference values and propagate [24]. We also check whether training a model ofline on the changes in the graph to compute values of the counthe counterfactual representations can lead to a diverse terfactual features and scores. The method estimates the ranking without having access to sensitive attributes at total causal efect of intersectional sensitive attributes on inference time. Our main finding is that we need to ex- the score. It estimates the direct efect and the indirect plicitly model the occupation as the bias direction varies efect mediated by the non-sensitive attributes, which across occupations. We also show that by training a are called mediators. The causal efects are estimated model on counterfactual representations the diversity using the mma R package [26], which performs mediaof the ranking is improved and the performance of the tion analysis with multiple mediators. To estimate the model is not afected. causal efects we propose three scenarios: do not model the occupation and apply Model 1 on the whole data (NoOccupation), apply Model 1 on the data correspond2. Counterfactual Representations ing to each occupation (SingleOccupation), and apply Model 2 on the whole data by also specifying a reference occupation (ModelOccupation).
After computing the causal efects of the sensitive attributes on the data, which represent the bias encoded in the data, one can compute the counterfactual representations. These are computed by changing the observed representations according to the causal estimates of the sensitive attributes. Counterfactual representations can be used to create a new ranking based on the counterfactual scores, or they can be used to train a model. Such a model can be used at inference time to predict the rank position of a candidate given the counterfactual representations of the candidate or the original representations. For the first option one needs access to sensitive attributes, which, according to the GDPR [1], is not possible in practice. We propose to use the second option: to train a model on counterfactual representations and apply it to the original representations without access to sensitive attributes at inference time.
to an occupation listing given a score that represents how well they fit it. Assuming there is a pre-existing bias in the features and scores of the candidates, we aim to create fairer representations by applying the method proposed in [25].
The method uses as input a causal model describing the data and the efects of the sensitive attributes on the data. A causal graph is a directed acyclic graph (DAG) where nodes represent variables, and directed edges between nodes represent causal relationships. A directed edge from node A to node B indicates that variable A causally influences variable B. Figure 1 shows two possible causal models that can be used to represent the data. Model 1 contains the following nodes: sensitive attributes, G (gender) and N (nationality), non-sensitive attributes of the candidates (X), and the utility score used to rank the candidates for a given occupation (Y), with edges from the features to the scores, and from the sensitive attributes to the features and the scores of the candidates. Model 2 contains the same set of nodes and edges, and additionally a node (O) representing the occupation with edges to the features and the scores. By adding the occupation node the model captures variations of the direction of bias across occupations.
To estimate the causal efects of sensitive attributes
The BIOS dataset [12] consists of real biographies collected from the web by filtering for lines that began with a name followed by the string “is a(n) (xxx) title,” where title is an occupation from the BLS SOC system1. We to the next page to view more candidates [13, 18]. The model a scenario where experienced candidates apply results reported are an average over the five runs for each for jobs in the same field. Each candidate is represented query. by non-sensitive features extracted from the text biog- RQ1: Do counterfactual representations lead to a raphy (term frequency of the occupation in the biogra- diverse rank in a recruitment scenario? We report phy, length and number of words of the biography) and the results of Model 1 in Figure 2 - NoOccupation. We see sensitive features: gender (provided by the dataset) and that the counterfactual representations do not increase nationality (inferred from each candidate’s name using the proportion for any of the groups, except for the Male the name2nat Python package [19]) under the assump- European group over the following occupations: filmtion nationality is inferred from the name, as a recruiter maker, journalist, software developer, surgeon, composer, might when reviewing a resume. painter and professor. This means that overall in the data
Data Pre-processing: Nationalities were grouped to- the Male European group has lower scores due to the gether by continent, but due to limited data in some of large number of candidates with lower scores in occupathe intersectional groups, the nationalities were grouped tions where the Female African-Asian-Latin group are to form an advantaged group (East-European and the over-represented.
West-European nationalities) and a disadvantaged group The results of estimating a causal model for each occu(African, Asian and Latin-American nationalities). The pation (Figure 2 - SingleOccupation) show an increase in American nationality was discarded due to ambiguity proportion across the sensitive groups for most occupabetween the inferred nationalities. The dataset doesn’t tions. The Female African-Asian-Latin group increased include Spanish nationality, avoiding ambiguity with the in proportion in the following occupations: physician, Latin-American group. Train-test splits are stratified chiropractor, comedian, and software engineer. Interestacross intersectional groups, with five consistent splits ingly, for comedian and software engineer, occupations per query using a 30% test set. Relevance Judgements: fully over-represented by men in the top 10, the proFor each occupation, candidates are ranked by the co- portion increased for both female groups, African-Asiansine similarity between the word2vec [17] embedding of Latin and European. The Male African-Asian-Latin group the occupation title and the text biography. Word2vec was increased in proportion for female dominated jobs, embeddings are known to perpetuate stereotypical as- e.g., paralegal, teacher and yoga teacher, but also in ocsociations [6, 14], simulating the pre-existing social bias cupations over-represented by Male Europeans, e.g., atin the data. The relevance judgements for training the torney and pastor. The Female European group was inmodel are assigned based on the cosine similarity with creased in male dominated jobs, e.g., poet, but also in values between 1 and 500 (most relevant candidate), with jobs dominated by Female African-Asian-Latin, e.g., yoga scores below 0.4 considered negative samples for training. teacher, model and dietitian.
Causal Estimation: The causal model was estimated on Results (Figure 2 - ModeledOccupation) show that the the train set. Following prior research [25], the disadvan- changes in proportion are similar to the ones obtained by taged group, Female African-Asian-Latin, is chosen as the estimating a causal model for each occupation, with some reference group. In experiments involving occupation minor fluctuations. For yoga teacher, the proportion of modeling, psychologists are the reference occupation due the ranking is more balanced using Model 2. to their balanced group distribution. Table 1 shows how far the proportion of the groups is from achieving statistical parity, which is achieved 4. Results and Discussion when the probabilities of a favorable outcome are equal between the groups [20], meaning that all groups have Fairness of a ranking is measured as the percentage of equal proportion in top 10. Positive values indicate undereach sensitive group among the top 10. Our aim is to cre- representation (0.25 means absence), approaching zero ate a diverse ranked list of the candidates with respected suggests a positive change, while negative values signify to the intersectional sensitive groups by increasing the overrepresentation (-0.75 means exclusive presence). Disproportion of the underrepresented groups, without pro- tance from zero implies a negative change in proportion. ducing a swap between the underrepresented group and The proportion increases often have a positive efect overthe over-represented group. If in one occupation females all, meaning that the proportion was increased for the are underrepresented, we do not want to over-represent underrepresented groups, however, in some situations them. Utility of the ranking model is measured using the increase in proportion of one group negatively afNormalized Discounted Cumulative Gain across the top fects another group (e.g., the occupation accountant is 10 (NDCG@10). We choose to evaluate at the top 10, as over-represented by men in the top 10, and regardless it is unlikely that the recruiter will scroll down or move of this the Male African-Asian-Latin group is increased in proportion afecting negatively the proportion of the Physician
Poet
Software Engineer females group which are underrepresented). jobs to increase the proportion of the females. RQ2: How to model the bias encoded in each occu- Applying the causal model over all occupations fails pation? Prior work [25] has applied counterfactual inter- to capture the variations of the bias direction across the sectionality to datasets that contain a single ranked list. occupations. This results in creating counterfactual repA recruitment scenario is more complex and challenging, resentations that change the observed representations as usually a recruiter needs a ranked list of candidates based on an overall estimated bias in the data, which for each job opening, and the direction and degree of bias does not reflect the real bias associated with each occupavary over occupations. For example, some occupations tion. Moreover, the bias estimates have very small values, are female dominated, such as nurse, while others are resulting in small changes of the scores and of the rank. male dominated, such as software developer [ 3]. Thus, One solution is to estimate a causal model for each for female dominated jobs one would want to increase occupation, or for groups of occupations with the same the proportion of the males, while for male dominated bias direction. We estimate a causal model for each occuAccountant pation without grouping, as there is no clear pattern for so that the bias estimate should be diferent for each ocgrouping the occupations due to the complexity added cupation. Results (Figure 2 - ModeledOccupation) are by the intersectional groups. Results (Figure 2 - SingleOc- similar to estimating a causal model for each occupation. cupation) show that estimating a causal model for each RQ3: Does training a ranking model on the counoccupation captures the variations of bias in each occu- terfactual representations create a diverse ranking? pation resulting in an increase in proportion across the Ideally, a model trained on fairer data would produce a sensitive groups for most occupations. Another solution fair ranking even without access to sensitive attributes. to capture the variations of the direction of bias across We use the Ranklib [10] implementation of RankNet [7], occupations is to introduce a node in the causal graph a pairwise learning to rank algorithm. Previous research representing the occupation. This indicates that the oc- [25] used ListNet [8], a listwise approach, but we obcupation has an influence on the bias present in the data, served RankNet has a better performance on the BIOS Statistical parity of the rankings, highlighting underrepresented and overrepresented group. Orig – original ranking of the candidates. NoOcc – Model 1 was applied over all occupations. SOcc – Model 1 was applied over each occupation. MOcc – Model 2 was applied over all occupations.
In this work we consider the applicability of existing fairness methods to recruitment. Legal requirements make many approaches dificult to use in practice, as access to special sensitive attributes is limited. We argue that pre-processing methods are well-suited to creating a diverse ranking of candidates, and we propose to use a counterfactual method to create fairer representations for candidates. The counterfactual method additionally makes score adjustments explicit, which can help satisfy transparency requirements. To apply this approach to the recruitment scenario, it is necessary to either estimate a causal model for each occupation or group of occupations, or to add a node representing the occupation to the causal graph.
Acknowledgments where we show that a model trained on the counterfactual representations can create a more diverse ranking, without having access to sensitive attributes at inference This research was (partially) funded by the FINDHR (Fairtime. These results confirm that modeling the occupation ness and Intersectional Non-Discrimination in Human is important in a recruitment scenario where diferent oc- Recommendation) project that received funding from the cupations may be associated with diferent biases. While European Union’s Horizon Europe research and innovathe BIOS dataset is a reasonable proxy, the data does not tion program under grant agreement No. 101070212, the come from a real recruitment scenario and may difer in Hybrid Intelligence Center, a 10-year program funded by several ways (e.g., distributions of people and bias may the Dutch Ministry of Education, Culture and Science difer from a real scenario, the dataset contains less de- through the Netherlands Organisation for Scientific Retailed information about education and work experience, search, https://hybrid-intelligence-centre.nl, and project and including artistic occupations might be unusual in a LESSEN with project number NWA.1389.20.183 of the hiring process). research program NWA ORC 2020/21, which is (partly)
Future work could investigate under what conditions financed by the Dutch Research Council (NWO). All conthe counterfactual representations lead to an increase in tent represents the opinion of the authors, which is not diversity that creates a balanced distribution of intersec- necessarily shared or endorsed by their respective emtional groups in top-ranked positions, given real features ployers and/or sponsors. for candidates such as education and work experience. arXiv:2006.08688, 2020. [26] Q. Yu and B. Li. mma: An R package for mediation analysis with multiple mediators. Journal of Open
Research Software , 5(1), 2017. [27] M. Zehlike and C. Castillo. Reducing disparate exposure in ranking: A learning to rank approach.
In Proceedings of the Web Conference 2020, pages 2849–2855, 2020. [28] M. Zehlike, F. Bonchi, C. Castillo, S. Hajian,
M. Megahed, and R. Baeza-Yates. FA*IR: A fair top-k ranking algorithm. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 1569–1578, 2017. [29] M. Zehlike, K. Yang, and J. Stoyanovich. Fairness in ranking, Part i: Score-based ranking. ACM Computing Surveys, 55(6):1–36, 2022. [30] R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and
C. Dwork. Learning fair representations. In International Conference on Machine Learning, pages 325–333. PMLR, 2013.