and for practitioners and developers of ADM systems to meet the societal requirement of having fair algorithms.

In particular, we build our critical analysis of fair-AI by distinguishing two broad aspects that are roots of several ambiguities:

1. Choice of sensitive attributes: It is not that discrimination per se is unjust, only discrimination with respect to some attributes that we have either to list and agree upon or to define with some reasonable criterion. (so-called sensitive or protected attributes) 2. What is the true meaning of unfair discrimination?: We need to clarify what we mean by making decisions involving such attributes, and in what cases such decision-making represent an unjust discrimination.

In Table 1, we summarise the key points raised throughout the paper.

In a nutshell, we claim that there is a fundamental ambiguity about what characteristics should be considered as protected for a given case and domain. High level non-discrimination principles can be found in several legislative frameworks, such as art 21,1 of the EU Charter of Fundamental Rights. But there is no clear specific legislation that organically discuss what individual characteristics should be considered protected. As an example, Table 2 tries to summarise the European non-discrimination laws, most of which consider specific domains of application and specific protected characteristics, thus making it unclear what to do with other potentially sensitive features and other domains. Something not dissimilar can be found in the US legislation; we refer to Barocas et al. [11, chap 6] and Barocas and Selbst [12] for more details. In general, we claim that there is no ethical or legal consensus on what are the dimensions (or the criteria to identify them) with respect to which we should assess and, eventually, mitigate for possible biases and discrimination. This can be summarised in the following:

Open Point 1 (Protected Groups). Given a specific phenomenon, what are the groups of people that we should consider as protected, and with respect to which we therefore have to take care of assessing and avoiding any unjust discrimination?

In turn, this ambiguity reveals a number of additional nuanced points. For example, it is unclear whether a fairness assessment should be conducted with regard to all protected groups even when the ADM system is not actually collecting such protected information. Indeed, while it is fairly common to

Moreover, there is ambiguity regarding the very definition of (protected) group. For instance, age can be aggregated in multiple ways, and assessing fairness with respect to different aggregations can potentially lead to very different results. On a more abstract level, there are concerns and discussions about the prospect of placing people in rigid and exclusive categories [13]. For instance, multiracial individuals come from various racial groupings. Indeed, at least on a biological/genetic level, race and ethnicity are now seen as extremely fluid and nuanced ideas rather than simple categorical attributes. Gender and sexual orientation are subject to very comparable criticism.

Finally, even if there are some proposals in the literature on how to fix this, there is still no consensus on how to deal with the exponentially growing number of subgroups to take into account when considering the intersection of several protected attributes at the same time [14,15]: Open Point 4 (Intersectional bias). Is it fair enough to evaluate unjust discrimination with respect to sensitive attributes separately? If not, which combinations of sensitive characteristics should we give priority to (given that we cannot realistically hope to assess all possible combinations)?

We believe that the crucial fuzziness around fair-AI lies in the fact that there is no consensus regarding what does it mean to unfairly discriminate a group of people with respect to others. Both legislative and ethical literature make the distinction between direct and indirect discrimination, 2 the former indicating an explicit use of protected characteristics to make a decision, while the latter being a discrimination through characteristics somehow associated to protected ones, but not protected per se. A possible example of indirect gender discrimination may be that of using income as a variable to make decisions on loan approvals, given that income is a variable typically correlated to gender. However useful, this distinction raises a set of open problems, the first reflected in the fact that it is not always possible to avoid both direct and indirect discrimination, and that some kind of balance should be somehow tolerated or even desirable (think, e.g., of affirmative action, which is indeed a strategy to remove indirect discrimination through explicit -i.e., direct -discrimination):

In particular, the concept of indirect discrimination is complex and hides many subtleties. In fact, most legislative frameworks actually admit that some characteristics can be legitimately used to make decisions, even when associated with protected attributes, since they represent a "business need" (such as the income in the previous example on loan approval). 3

In this respect, we would like to point out that a possible way out for Open Point 6 about legitimate business needs, would be that of identifying ex-ante a set of variables as the only legitimate features to be used in particularly delicate domains (such as job recruiting). Namely, this can be seen as the counterpoint of Fairness Through Unawareness, or Blindness [see, e.g. 23]. Blindness consists in building ADM systems that are not exposed directly to sensitive attributes. While this strategy prevents the possibility of direct discrimination, it leaves room for indirect discrimination through the use of variables associated to sensitive characteristics. Given this context, rather than offering a list of unusable attributes, we could offer a list of attributes that are the only allowed for a particular domain/case, assuming that we consider those attributes relevant for the task at hand and will therefore accept any discrepancies that may result from the association of such variables with sensitive attributes. 4In order to capture different concepts of (direct and indirect) discrimination, the fair-AI literature has developed a wide range of observational metrics [25,26]. However, on the one hand it has been proved that most of such metrics are mutually incompatible [26], and on the other hand metrics that are solely observational are blind to the real underlying mechanism and only provide a static picture of an often very intricate phenomenon. For a given use-case, choosing only one observational metric is, at best, exceedingly challenging and most likely simplistic. Some works have proposed guidelines -usually in the form of decision trees or diagrams-to help finding the most appropriate statistical metric given domain-specific constraints [see, e.g., 27,28,29,30]. However, as the authors of such works clearly acknowledge, the process of following the proposed decision diagrams is itself complicated, involving necessarily multi-disciplinary competencies, and in any case they warn not to take these diagrams too categorically or as a set of well-established prescriptions. To make things even more blurry, if it is true that having perfect parity with respect to different metric classes is mathematically impossible, allowing a limited level of disparity may be attainable with multiple metrics at the same time [31], suggesting that focusing too much on a single metric maybe counter-productive after all. Furthermore, employing such metrics creates more uncertainty about the numerical threshold that indicates the true existence of unfairness, or even about the specific form of metric that should be used (e.g., taking ratios vs. differences of significant quantities -see, e.g., Ruggieri et al. [2]).

Open Point 7 (Over-reliance on observational metrics). Purely observational fairness metrics should be taken with a grain of salt. At best, they can be used as a means for a deeper reasoning on the mechanisms underlying a phenomenon, rather than a final word on the presence or lack of unjust discrimination. A clear connection between quantitative metrics and unjust discrimination is still missing.

One of the problems of observational metrics is that they are blind to the causal structure of the underlying phenomenon, with the risk of attributing to bias and discrimination what is instead spurious correlation. Indeed, is true that using a causality-aware approach makes it possible to transparently disentangle between direct, indirect, and spurious effects, as brilliantly showcased by Plečko and Bareinboim [32].

However, identifying the causal structure of a given use case is far from straightforward. Moreover, there is an open philosophical debate regarding the very notion of human attributes as causes [33]. Therefore, even if we welcome the causal analysis of the underlying phenomenon in order to better assess for the presence of bias and discrimination, we would like to raise the attention on the following point:

Open Point 8 (Assumptions of causal structure). Causal tools require strong, often unverifiable, assumptions. Downstream consequences of wrong assumptions can lead to wrong or even harmful actions. Therefore, particular care must be taken when relying on such tools.

In this work, we support the view that the understanding of the landscape of unjust discrimination in ADM, despite the impressive work done in the last decade or so by the fair-AI community, is not yet mature enough to be "put into practice". In particular, there are gaps at the intersection between the mathematical and statistical tools developed by statisticians and AI researchers, the legal nondiscrimination provisions, and the ethical and social notions of fairness. We believe these gaps are deep, and not so easy to bridge, in part precisely because they lie at the boundaries between quite different worlds.

We believe that the requirement to develop fair algorithms is still too vague and that, in order to be put in place, we have to clarify and be aware of a set of open points, most of which are societal in nature rather than technical. Given the strong multidisciplinary content, this challenge will be best addressed and discussed when researchers and practitioners from the various fields involved (e.g., statisticians, AI experts, ethicists, and legal experts) collaborate toward the goal, potentially creating a shared vocabulary and set of working notions. In fact, this is the spirit that also guided this work.

As a final remark, notice that there are of course other problematic aspects of unjust discrimination in ADM systems that are somehow out of the scope of this work and would have required a much broader analysis. We can cite, e.g., how to face the challenge of bias in models with unstructured data such as images and text, especially in generative AI models such as the modern Large Language Models; or more technological problems, such as those raised by the modularity of AI systems, that are usually composed of several steps and components, making it complicated to clarify how fairness issues may propagate through the process.