This paper discusses the necessity of integrating fairness into the development of trustworthy AI systems, focusing on methods and tools designed within the fAIr by design project - a collaborative approach to guide development teams towards the creation of non-discriminatory AI systems. Practical applications, challenges, and recommendations based on real-world use cases are shared from a data science and machine learning team perspective. The paper advocates for continuous learning, diverse team assembly, and ongoing monitoring to ensure AI systems remain fair and inclusive, encompassing the whole life cycle of AI systems.
Artificial Intelligence (AI) has become increasingly integrated into various aspects of society,
from communications to recruitment[
The challenge begins with defining fairness — a concept that proves elusive across disciplines.
Drawing upon Mehrabi et al.[
Recognizing the dynamic nature of fairness, influenced by societal norms and technological advancements, adds complexity to the efort of implementing fairness in AI systems but also enables AI systems to be finely tuned to balance fairness with performance. This balanced approach requires ongoing dialogue and adjustments between ethical principles and system eficiency, ensuring continuous alignment through monitoring and iterative enhancements.
The research consortium fAIr by design[
This chapter delves into a strategy employed by the fAIr by design team to navigate the
challenges of developing algorithms with embedded fairness constraints. Drawing on the adapted
Assurance Case method [
Originating from safety engineering, the Assurance Case method ofers a structured pathway for
translating high-level goals into specific, actionable, and verifiable technical specifications [
Components of the method include claims, sub-claims, evidence and reasoning, which enable the project team to identify and systematically integrate a comprehensive understanding of the AI system and fairness needs through the definition of sub-claims, each supported by (technical) evidences. The overall process included developing evidences and tests, a thorough exploration of model selections, fairness metrics, and the application of mitigation strategies.
The AC emphasizes the crucial role of integrating "challengers" (technical or non-technical) into the AI development process to identify and point out aspects that are not clear and can lead to unfairness. Having a comprehensive understanding of each component of the AI system from the outset is invaluable, and the involvement of "challengers" enhances this understanding by ensuring that fairness considerations are addressed. The "challenger" can assist in clarifying data needs, testing protocols, evaluation metrics, and mitigation strategies, laying the groundwork for in-depth fairness testing. This also equips the team to manage potential future challenges.
This chapter reports on the application of the methodologies and tools found during fAIr by design. The implementation of the AC was done together with social scientists and legal experts, but learnings and recommendations center on technical application and data science perspective.
At the onset of developing a fair AI system, it’s important from a technical and social-science standpoint to engage in a series of critical inquiries and clarifications with the use case partner, which include: • The definition of fairness from the partner’s perspective. • The characteristics that constitute a fair AI system for them. • The (envisioned) structure of the AI system and it’s components. • The risks identified and the sub-claims that can be substantiated with technical evidence. • The feasible tests, along with a potential responsible person. • The prerequisites for conducting these tests, including necessary data, knowledge, and resources. • The applicable metrics and thresholds for fairness.
• The mitigation strategies to be employed should these thresholds not be met.
The use of methods and tools developed within the Assurance Case or those available as
open access resources[
All steps of fairness testing require data quality and overall machine learning and data science expertise (e.g. ablation studies, hyper-parameter optimization or inverse relation modeling). Addressing biases in training and evaluation data is vital to prevent AI systems from replicating or exacerbating existing inequities. For some companies it can be a challenge to build the necessary knowledge around data quality, conducting systematic evaluations, and testing model components, which should be addressed from the onset.
Building a proficient team that specializes in appropriate data science and machine learning
methods is essential to circumvent fairness testing pitfalls efectively. It requires a concerted
efort to integrate knowledge on fairness into AI, delineating clear responsibilities among senior
management and development teams. Fairness, much like other critical quality criteria, must be
integrated into a wide array of business processes, gaining prominence especially in high-risk
AI systems[
The establishment of regular auditing and accountability mechanisms is pivotal in upholding non-discriminatory practices in AI. Continuous monitoring and evaluation of AI systems enable organizations to proactively identify and address any potential biases or discriminatory outcomes, thereby demonstrating their commitment to fairness and ethical AI development.
Another critical aspect of promoting non-discriminatory AI development is fostering collaboration and transparency. Organizations should actively seek partnerships with diverse stakeholders, including ethicists, community representatives, and regulatory bodies, to gain insights into the potential biases and discriminatory risks in AI systems. By promoting transparency in AI development processes, organizations can build trust and accountability with the public, while ensuring that fairness remains an integral part of the AI life cycle. This is useful as a quality assurance tool, ensuring that the product being developed actually satisfies market needs.
Collaborative eforts with partners have provided valuable insights into the practical challenges and strategies for developing non-discriminatory AI systems. These collaborative endeavors shed light on the significance of data science and machine learning maturity, as well as the willingness to invest time, efort, and resources into crafting fair, non-discriminatory, and trustworthy AI systems. However, navigating fairness throughout the AI life cycle presents challenges, including the dynamic nature of fairness definitions and the intricacies of measuring fairness.
The fAIr by design use cases, focusing on small companies (SMEs), start-ups and cultural organizations/NGOs - each with distinct applications, domains, and developmental stages - underscore the diverse landscape within which fairness considerations are embedded, emphasizing the importance of contextual understanding and adaptability. However, we have not been able to work with larger, more established organizations, with a more advanced data management structure, but may face other challenges.
The journey towards non-discriminatory AI development requires the concerted eforts of organizations, policymakers, and society as a whole. fAIr by design, including the Assurance Case, provides a structured approach to the development of fair AI systems. Recognizing the contextual nature of fairness, the adoption of ethical AI principles should be accompanied by continuous training for development teams, enabling them to incorporate ethical guidelines into their day-to-day decision-making processes. In this project we have seen how beneficial it is for social sciences, data sciences and use case partners working together. Moving forward, the exploration of cross-industry collaborations and the proposition of structured frameworks for engaging organizations in fairness discussions are promising directions for the advancement of non-discriminatory AI. The potential to draw on diverse perspectives and expertise, could ultimately lead to industry-wide standardized approaches, thus bringing us closer to a future where ethical principles in AI are upheld across the board.