Many diferent types of bias are discussed in the algorithmic fairness community. A clear understanding of those biases and their relation to fairness metrics and mitigation techniques is still missing. We introduce Bias on Demand: a modelling framework to generate synthetic datasets that contain various types of bias. Furthermore, we clarify the efect of those biases on the accuracy and fairness of ML systems and provide insights into the trade-ofs that emerge when trying to mitigate them. We believe that our open-source package will enable researchers and practitioners to better understand and mitigate diferent types of biases throughout the ML pipeline. The package can be installed via experiments are available at https://github.com/rcrupiISP/BiasOnDemand. We encourage readers to consult the full paper [1].
https://www.ifi.uzh.ch/en/scg/people/Baumann.html (J. Baumann) Contributions We build on computer science and philosophical literature from the field of algorithmic fairness to explore fundamental types of bias. We provide a modelling framework to generate synthetic datasets that can include those biases. We use our proposed framework to investigate the interconnection between biases and their efect on performance and fairness evaluations. Furthermore, we provide some initial insights into mitigating specific types of bias through post-processing techniques [12–15].
We provide a mathematical representation of the following types of bias: i) Historical bias –
sometimes referred to as social bias, life bias, or structural bias [
Showcase As a simple example, we generate datasets for diferent magnitudes of historical bias and measurement bias on the features , denoted by ℎ and . Figure 1 shows the efects of those biases w.r.t. diferent performance and fairness metrics and for applying various bias mitigation techniques: DP, FTU, and TPR parity. In line with [17–20], we find that FTU should be applied with particular care, despite its simplicity. FTU has no efect whatsoever since the information on group membership is redundantly encoded in (see Figure 1a). However, there are even cases in which the application of FTU leads to biased results and performance deterioration even when the unconstrained model does not (see Figure 1b). Similarly, Figure 2 shows the results for diferent magnitudes of historical bias and measurement bias on the labels , denoted by ℎ and . The results of historically biased are comparable to the ones of historically biased features, except for FTU, which results in DP (see Figure 2a). As Figure 2b shows, the case of measurement bias on is particularly subtle: having access only to a (biased) proxy of , it is only possible to control the bias when imposing fairness criteria that do not use the target variable, namely DP and FTU in our experiments. All examples show that there are trade-ofs between fairness and accuracy as well as between diferent fairness criteria. 2 Outlook This work aims to raise awareness of bias in artificial intelligence (AI) systems and its potential impacts on individuals and society, promoting the development of bias-free AI systems. This is in line with the European Union’ Proposal for a regulation laying down harmonised rules on AI (AI Act) [21]. By exploiting our toolkit, we hope to encourage the research community to conduct further studies using synthetic datasets where real-world datasets are missing. 1Notice that we use algorithmic bias as an umbrella term for aggregation bias, learning bias, and evaluation bias as they are all associated with the ML model development [8]. 2The entire and reproducible set of experiments and the code to develop new ones are available in open-source. 3 6 bias ( hR) Selection rate A0 Selection rate A1 9 0 3 6 bias ( hR)
Acknowledgments The authors wish to thank Andrea Cosentini, Ilaria Penco and all the members of the Artificial Intelligence & Data Science Department of Intesa Sanpaolo for their commitment to supporting responsible AI initiatives. We also thank Michele Loi and the members of the Social Computing Group at the University of Zurich (Anikó Hannák, Nicolò Pagan, Corinna Hertweck, Stefania Ionescu, Aleksandra Urman, Azza Bouleimen, Leonore Röseler, and Desheng Hu) for their helpful feedback on an earlier version of this manuscript. We would also like to thank the anonymous reviewers for their valuable comments and helpful suggestions. Joachim Baumann was supported by the National Research Programme “Digital Transformation” (NRP 77) of the Swiss National Science Foundation (SNSF) – grant number 187473 – and by Innosuisse – grant number 44692.1 IP-SBM. 3 6 bias ( hY) Selection rate A0 Selection rate A1 9 0 3 6 bias ( hY) Discovery 10 (2020) e1356. [8] H. Suresh, J. Guttag, A framework for understanding sources of harm throughout the machine learning life cycle, in: Equity and access in algorithms, mechanisms, and optimization, 2021, pp. 1–9. [9] N. Pagan, J. Baumann, E. Elokda, G. De Pasquale, S. Bolognani, A. Hannák, A Classification of Feedback Loops and Their Relation to Biases in Automated Decision-Making Systems (2023). arXiv:2305.06055. [10] A. Castelnovo, R. Crupi, G. Del Gamba, G. Greco, A. Naseer, D. Regoli, B. S. M. Gonzalez, Befair: Addressing fairness in the banking sector, in: 2020 IEEE International Conference on Big Data (Big Data), IEEE, 2020, pp. 3652–3661. [11] A. Castelnovo, R. Crupi, G. Greco, D. Regoli, I. G. Penco, A. C. Cosentini, A clarification of the nuances in the fairness metrics landscape, Scientific Reports 12 (2022) 1–21. [12] M. Hardt, E. Price, N. Srebro, Equality of opportunity in supervised learning, in: Advances in neural information processing systems, 2016, pp. 3315–3323. [13] S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, A. Huq, Algorithmic Decision Making and the Cost of Fairness, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, Association for Computing Machinery, New York, NY, USA, 2017, pp. 797–806. doi:10.1145/3097983.3098095. [14] J. Baumann, A. Hannák, C. Heitz, Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Suficiency, in: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, Association for Computing Machinery, New York, NY, USA, 2022, pp. 2315–2326. doi:https://doi.org/ 10.1145/3531146.3534645. [15] S. Bird, M. Dudík, R. Edgar, B. Horn, R. Lutz, V. Milan, M. Sameki, H. Wallach, K. Walker, Fairlearn: A toolkit for assessing and improving fairness in AI, Technical Report MSR-TR2020-32, Microsoft, 2020. URL: https://www.microsoft.com/en-us/research/publication/ fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/. [16] C. Hertweck, C. Heitz, M. Loi, On the moral justification of statistical parity, in: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 747–757. doi:10.1145/3442188.3445936. [17] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, R. Zemel, Fairness through awareness, in: Proceedings of the 3rd innovations in theoretical computer science conference, 2012, pp. 214–226. [18] F. Kamiran, T. Calders, Data preprocessing techniques for classification without discrimination, Knowledge and Information Systems 33 (2012) 1–33. [19] I. Y. Chen, F. D. Johansson, D. Sontag, Why is My Classifier Discriminatory?, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, Curran Associates Inc., Red Hook, NY, USA, 2018, pp. 3543–3554. [20] S. Corbett-Davies, S. Goel, The Measure and Mismeasure of Fairness: A Critical Review of
Fair Machine Learning, 2018. arXiv:1808.00023. [21] The European Commission, Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts, 2021. https://digital-strategy.ec.europa. eu/en/library/proposal-regulation-laying-down-harmonised-rules-artificial-intelligence.