A vast number of systems in Europe and beyond currently use algorithmic decision making (ADM) to (partially) automate decisions that have previously been done by humans. When designed well, these systems promise both more accurate and more eficient decisions all the while saving large amounts of resources and freeing up human time. When ADM systems are not designed well, however, they can lead to unfair algorithms which discriminate against parts of the population under the guise of objectivity and legitimacy. Many examples of both fair and helpful as well as discriminatory algorithms exist in the wild to date. The group they fall into typically depends on the decisions made during their design. It is therefore clearly important to properly understand the decisions that go into the design of ADM systems and how these decisions afect the fairness of the resulting system. To study this, we introduce the method of multiverse analysis for algorithmic fairness.
(C. Kern)
1. Extended Abstract
Across the world, more and more decisions are being made with the support of algorithms, so
called algorithmic decision making (ADM). Examples of such systems can be found in finance,
the labour market, criminal justice system and beyond. While these systems are very promising
when designed well, raising hopes of more accurate, just and fair decisions, their impact can
be quite the opposite when designed wrongly. Ample examples exist of unfair ADM systems
discriminating against people in the wild, with the Dutch childcare benefits being an especially
prominent and recent example [
While these fairness problems are often due to biases in the underlying data, gathering
perfectly fair data is usually not an option, so the only way of making sure that the algorithm
doesn’t reinforce these biases is via the design of the ADM system. With the promise and peril
of ADM systems depending so much on their proper design, it is of clear importance to properly
understand the decisions that go into their design and how these decisions afect algorithmic
fairness. To enable this we introduce the method of multiverse analysis for algorithmic fairness.
Multiverse analyses were introduced in Psychology [
In the proposed adaptation of multiverse analysis for ADM one starts by making the many implicit decisions, also referred to as researcher degrees of freedom, during the design of an ADM system explicit. One of the diferences in the present analysis compared to a classic multiverse analysis is, that we will evaluate machine learning systems in the end, whereas classical multiverse analyses will typically culminate in a null-hypothesis-significance-test (NHST). While many of the decision points apply to any machine-learning system (e.g. choice of algorithm, how to preprocess certain variables, cross-validation splits), many of them are also domain specific (e.g. coding of certain variables, how to set classification thresholds, how fairness is operationalized). While we vary certain decisions related to the training of machine learning models, our focus will not be on hyperparameter-selection or optimization. In particular we focus on decisions made during the pre-processing of data and in the translation of predictions into possible decisions. Using all possible unique combinations of these decisions we create a grid of possible universes of decisions. For each of these universes, we compute the fairness of the ADM system and collect it as a data point. The resulting dataset of possible decisions and resulting fairness is treated as our source data for further analysis where we evaluate how individual decisions relate back to fairness.
Existing articles in the literature have focused on specific pre-processing or modeling decisions
in isolation, such as the influence of diferent imputation methods [
Here we present a generalizable approach of using multiverse analysis to estimate the efect
of decisions during the design of an ADM system on its algorithmic fairness. We demonstrate
the feasibility of this approach using a case study of predicting public health coverage in US
census data. We use the ACSPublicCoverage benchmark problem [
We will present preliminary results from the case study, demonstrating how plausible and seemingly small design decisions of the ADM system can have significant efects on its algorithmic fairness. We would welcome the discussion of other use cases and possible case studies, especially within the European context.