Machine Learning (ML) systems are capable of reproducing and often amplifying undesired biases. This puts emphasis on the importance of operating under practices that enable the study and understanding of the intrinsic characteristics of ML pipelines, prompting the emergence of documentation frameworks with the idea that "any remedy for bias starts with awareness of its existence". However, a resource that can formally describe these pipelines in terms of biases detected is still amiss. To fill this gap, we present the Doc-BiasO ontology, a resource that aims to create an integrated vocabulary of biases defined in the fair-ML literature and their measures, as well as to incorporate relevant terminology and the relationships between them. Overseeing ontology engineering best practices, we re-use existing vocabulary on machine learning and AI, to foster knowledge sharing and interoperability between the actors concerned with its research, development, regulation, among others. Overall, our main objective is to contribute towards clarifying existing terminology on bias research as it rapidly expands to all areas of AI and to improve the interpretation of bias in data and downstream impact.
The breakthroughs and benefits attributed to big data and, consequently, to machine learning (ML) -or AI -systems [1,2], have also resulted in making prevalent how these systems are capable of producing unexpected, biased, and in some cases, undesirable output [3,4,5]. Seminal work on bias (i.e., prejudice for, or against one person, or group, especially in a way considered to be unfair) in the context of ML systems demonstrates how facial recognition tools and popular search engines can exacerbate demographic disparities, worsening the marginalization of minorities at the individual and group level [6,7]. Further, biases in news recommenders and social media feeds actively play a role in conditioning and manipulating people's behavior and amplifying individual and public opinion polarization [8,9]. In this context, the last few years have seen the consolidation of the Trustworthy AI framework, led in large part by regulatory bodies [10], with the objective of guiding commercial AI development to proactively account for ethical, legal, and technical dimensions [11]. Furthermore, this framework is also accompanied by the call to establish standards across the field in order to ensure AI systems are safe, secure and fair upon deployment [11]. In terms of AI bias, many efforts have been concentrated in devising methods that can improve its identification, understanding, measurement, and mitigation [12]. For example, the special publication prepared by the National Institute of Standards and Technology (NIST) proposes a thorough, however not exhaustive, categorization of different types of bias in AI beyond common computational definitions (see Figure 1 for core hierarchy) [13]. In this same direction, some scholars advocate for practices that account for the characteristics of ML pipelines (i.e., datasets, ML algorithms, and user interaction loop) [14] to enable actors concerned with its research, development, regulation, and use, to inspect all the actions performed across the engineering process, with the objective to increase trust placed not only on the development processes, but on the systems themselves [15,16,17,18]. In addition to human-readable (i.e., textual descriptions in a format that humans can read and understand) documentation frameworks for machine learning pipelines [19,15,20], semantic data models (e.g., ontologies, knowledge graphs) can also play a crucial role in enhancing the accuracy and interpretability of ML systems [21], as well as to perform "bias assessment, representation, and mitigation" tasks [22], in a way that is also machine-readable (i.e., makes available a fine-grained description of data in a format manageable by computers). This characteristic improves the findability, accessibility, interoperability and reusability (FAIR) of data-centric resources in the Web [23,24]. Ontologies to model existing ML fairness metrics [25,26], as well as the semantic specifications to catalog risks in terms of compliance and conformance of AI systems under the EU's AI Act1 [27,28] have been proposed, however, a resource that can formally describe ML pipelines, and provides a vocabulary to characterize them in terms of measured biases is still amiss.
Proposed Solution We propose an ontology-driven approach to describe and document biases detected across machine learning pipelines. Here, we refer to documentation as the process of generating metadata represented in formats understandable by humans and also by machines [29], where formal data models like ontologies and controlled vocabularies provide standardized concepts for expressing this metadata. Our ontology, Doc-BiasO, is a resource developed with the objective to introduce an integrated vocabulary system of AI-related biases as defined in the literature and their measures; represent their relationships with other relevant terminology, i.e., datasets, ML systems, fairness, harm, risk; and semantically annotate ML pipelines based on bias measures values. The version presented here has 389 classes, 72 object properties, and 28 data properties.
Contributions: Concisely, our contributions are the following:
1. Doc-BiasO, an integrated vocabulary system of ML-related biases; 2. an ontology-based approach to document bias in ML pipelines; 3. a technical evaluation of Doc-BiasO.
The remainder of this paper is structured as follows: Section 2 introduces relevant Semantic Web concepts and presents a review of related literature. Section 3, details the design of Doc-BiasO. The results of the evaluation are reported in Section 4. Finally, Section 5 outlines our conclusions and future lines of work.
Human Bias
Upper category Subcategory Specialized subcategory
Figure 1: Types of Bias. Core categories of bias in relation to AI systems as per the NIST [13]. In the center and in turquoise, we depict the biggest circle for the Bias concept, as the most abstract and highest category. Via thicker arrows, we depict three smaller blue circles representing three main sub-categories of Bias. Using finer arrows that draw out from the three sub-categories, we depict smaller soft violet circles, to represent further sub-categories in the bias hierarchy.
Ontologies and Machine Learning Gruber [30] defines an ontology as a formal, explicit specification of a shared conceptualization that is characterized by high semantic expressiveness required for increased complexity. Ontologies include abstract concepts or classes, represented as nodes, and predicates representing the relations of these classes, edges in an ontology; the meaning of the predicates is represented using rules. Ontologies are specified using knowledge representation models, making the expressiveness of the ontology dependent on the expressive power of the representation model. The Resource Description Framework (RDF)2 enables the description of entities in terms of classes and properties; while subsumption relations between classes and properties can be modelled with the RDF Schema (RDFS). 3 More expressive formalisms like the Ontology Web Language (OWL) 4 make available a larger number of operators which enable the representation not only of classes, properties, and subsumption relations, but also class and property constraints, general equivalence relations, and restrictions of cardinality. Several examples of the usefulness of context-aware ontologies for bias awareness and mitigation in ML systems are explored in the work presented in [22].
In the context of bias modelling, the Bias Ontology Design Pattern (BODP) [31] is one of the first works to propose a formalization for the bias concept. Its objective is to capture a high-level representation of bias as an abstract term and not necessarily in the context of ML systems. We re-use part of BODP as a building block to repurpose it for scope and intended use of Doc-BiasO, which is to document bias in AI pipelines. Similar to our work, the fairness metrics ontology (FMO) [25,26] models fairness metrics (fmo:fairness_metric) from the literature and relates them to their use-case. The conceptualization of bias and fairness in relation to ML systems are often intertwined; however, distinctions between both concepts need to be made explicit, as they are not always used in conjunction, nor to study the same phenomena [32,33]. Fairness, in relation to ML (fair-ML), takes the form of algorithmic interventions that incorporate mathematical formalizations of moral or legal notions for the fair treatment of different populations into ML pipelines. These interventions aim to prompt ML models to satisfy statistical non-discrimination criterion for a given subpopulation [4]. In our case, we focus on modelling biases in data identified in the literature and the existing measures defined to detect them. Specifically, we propose a descriptive bias vocabulary that can be used and incorporated into varying frameworks as needed and that can be extended to further semiautomatize documentation tasks. Concepts and relations pertaining to bias are not made explicit in the current version of FMO, however, we consider both ontologies to be complementary, thus we re-use FMO to foster the development of a comprehensive vocabulary that provides coverage of terminology that pertains to the responsible development of ML systems. We follow a similar approach with the AI Risk Ontology (AIRO) [27], and by-effect, the Vocabulary of AI Risks (VAIR) [28]. In this case, risk in relation to ML systems, under the broader label of AI, is defined as systems that are likely to cause serious harms to health, safety, or fundamental rights of individuals as per European Union (EU) Law. These works are ontologydriven approaches to account for the compliance and conformance of AI systems under the EU's AI Act's specifications. 5 Specifically, AIRO is a modular ontology created to identity whether an AI system is classified as high-risk, whilst VAIR provides semantic specifications for cataloging AI risks, re-using core concepts in AIRO (e.g., airo#Risk, airo#Consequence). Lastly, [34] proposes a descriptive framework (ACROCPoLis) to describe ML systems and their societal impact by making explicit the interrelations and diverging perspectives of relevant stakeholders (individuals, groups of people, institutions). While this is beyond the scope of our work, should the conceptual model be formalized and made publically available, a study for re-use and extension of Doc-BiasO ontology would be undertaken in a future iteration. The Semantic Web community has also proposed other technical solutions to improve the interpretability and transparency of machine learning pipelines. The provenance ontology, PROV-O [35], enables the representation of provenance information generated by different entities, and can be easily applied to multiple contexts (i.e., training datasets). Standard schemas for data mining and machine learning algorithms, such as the Machine Learning Schema
Bias A concentration on, or interest in one particular area or subject. Whilst a more value-laden definition, conceptualizes bias as prejudice for, or against one person, or group, especially in a way considered to be unfair [45].
The use, purpose or application of a machine learning system. Examples include, recommenders, speech recognition, etc.
Task or ML Problem is the formal description of a process that needs to be completed (e.g., based on inputs and outputs) [36].
Dataset A collection of data, published or curated by a single source, and available for access or download in one or more representations [36].
Adverse lived experiences resulting from an ML system's deployment and operation in the world [46,47].
Bias Measure A quantitative metric or indicator that assesses the presence and extent of bias in a particular context, via predefined thresholds [48].
(MLS) ontology [36], and the Description of a Model (DOAM) 6 ontology, provide fine-grained vocabularies to represent ML models characteristics. Moreover, the issue of reproducibility in ML has also been addressed [37]. Correspondingly, the Data Catalog Vocabulary (DCAT) [38] enables the fine-grained description of datasets and data services in a catalog using a controlled and rich vocabulary. Adhering to ontology engineering best practices [30], all these ontologies and vocabularies have been re-used in the composition of Doc-BiasO.
The opaqueness of the inner processes of ML systems can hinder the understanding of how they work. [19], Gebru et al. [15], [20], Hupont et al. [39], thus advocate for the production of value oriented, human-readable documentation for datasets (Data Statements for Natural Language Processing, Datasheets for Datasets), ML models (Model Cards for Model Reporting and Use Case Cards). Doc-BiasO aims to follow their stride by combining the different components of the AI pipeline (input, model, output data) to produce comprehensive descriptions in human-and machine-readable format of data-driven pipelines. Other documentation approaches, such as Sun et al. [40] introduce a tool to assess fitness for use of datasets. This automated data exploration tool delimits its focus to three dimensions: representativeness, bias, and correctness. In a similar line, [41] introduces a bias visualization tool for computer vision datasets. This exploration tool narrows down their assessment to three sets of metrics: object-based, gender-based and geography-based dimensions. Further, interactive tools-developed by industries-(e.g., [42,43,44]) enable dataset exploration, visualization, and comparison. The extensible and modular design of Doc-BiasO, allows users to describe and document their data-driven pipelines, and seamlessly incorporates additional descriptive dimensions and components as needed. Further, the underlying knowledge-driven framework prompts the integration and fine-grained description of multiple data sources, and leverages reasoning capabilities for enhanced data analytics.
In this section, we describe the design stages of Doc-BiasO. We also describe its implementation and include an example of an instance.
To determine the scope of our ontology, we perform a domain and content analysis following a hybrid strategy. On the one hand, through our own position within a research project on bias in relation to the development and regulation of AI 7 , we have held fruitful discussions with experts researching different dimensions of bias from a multidisciplinary and critical point of view, i.e., [49,12]. Further, these discussions have helped identify what concepts make up our universe of discourse, for instance, bias, ML model, dataset, task, application, fairness, harms, risks; as well as how these concepts interact or relate to each other. In Table 1, we have summarized the core concepts defined in our ontology. Each of these concepts represents the top-most abstract concept in a hierarchy of terms, with less abstract or more concrete concepts being defined as the ontology grows to give a broader coverage. For example, Bias is the most abstract representation, while Representation Bias is a more concrete type of bias.
The exchanges with researchers have also helped deepen our understanding and characterization of bias in data from a critical stance (e.g., there is never just one bias, bias detection is contextual, bias detection can depend on data modality, biases cannot be eradicated [12]) and to identify challenges not only in modelling bias, but also in relation to the underlying documentation process, primarily on how it should not be fully automatized. In developing a tool like our ontology, it is important to aim for a careful balance between an effective, useful and comprehensive vocabulary that supports streamlining documentation tasks, while at the same time, avoid dissuading practitioners from critical thinking when engaging in both documentation and bias analysis. The aim of both of these practices is to mitigate negative consequences arising from the deployment of ML systems. However, it is always possible that unintentionally through enforcing standardization or automation on practitioners, new gateways are created that worsen the problem. Some influencing factors are the lack of experience, domain knowledge, or the right incentives [46,50,51]. Ultimately, this rapport informs our design choices across all iterations of ontology engineering, makes us aware of the limitations of our technical tool, and creates opportunities for refinement in later versions.
On the other hand, the scope of our ontology is also informed by the growing body of literature on our topic of interest. In this case, we particularly rely on official reports, as is the NIST Special Publication 1270 [13], and by periodically identifying relevant work in order to gather background information for a rich vocabulary of biases-(e.g., [52,33,53,14,54]), while also consider emerging work on this topic published at venues such as: ACM FAccT8 , AAAI/ACM AIES9 , ACM EAAMO. 10 Concisely, we pay attention to discerning bias, and its detection measures from fairness notions and their measures, combining keywords such as "machine learning", "artificial intelligence", "datasets", "bias", "metrics", and "bias mitigation".
To design and model our ontology, we adhere to ontology engineering best practices [30,55]. As such, after the scope is determined and competency questions are defined, re-usable ontologies are identified following a layered approach (i.e., a foundational layer for general metadata and provenance, a domain-dependent layer to cover standards for the relevant area of use, a domain-dependent layer of ontologies specific to our problem of interest) [55].
We first specify the competency questions that emerged during the analysis phase and that represent the intended use of Doc-BiasO: a tool that can be integrated into AI documentation frameworks and that can offer the vocabulary required to characterize these pipelines; ideally, a resource that informs AI practitioners or researchers on the ways in which bias interacts with other components in the AI pipeline, and as a controlled repository as they study the development of a new measure and wish to survey those that already exist.
We then lay the foundation of our ontology by re-using ontologies such as: the SKOS data model [56], the PROV data model (PROV-O) [35], and the Friend of a Friend (FOAF) vocabulary [57]. The next layer incorporates standard schemas for data mining and machine learning algorithms, such as the Machine Learning Schema (MLS) ontology [36]. This schema provides fine-grained descriptions to represent the characteristics and intricacies of ML models. Similarly, the Data Catalog Vocabulary (DCAT) [38] enables the fine-grained description of datasets and data services in a catalog using a controlled and rich vocabulary. By extension, the Data Quality Vocabulary (DQV) [58] provides a framework and vocabulary to assess the quality of a dataset, offering an extensive catalog of quality metrics. For our third layer, we look at previous work on bias, specifically the BODP [31] and the Artificial Intelligence Ontology (AIO). 11 The class AIO:Bias is our starting point, which we organize in hierarchies via rdfs:SubClassOf, as per the AIO modelling, and in order to represent different kinds of bias identified in the literature i.e., representation bias, popularity bias, demography bias. We build on the pattern and ontology, however, it does not suffice to our modelling needs. For this reason, all missing concepts are incorporated manually, as we set out to capture and explicitly document otherwise unstated assumptions about bias in relation to ML systems [59]. Critical data studies [47,59] maintain that for bias detection tasks to be meaningful, practitioners must reflect on possible harms that can emerge upon the deployment of an ML system in dynamic societal and cultural contexts. Here, we emphasize thus on both, the importance of assisting practitioners via the development of tools that streamline tasks that may be perceived as a burden [50], while avoid dissuading them from reflecting about harms that could emerge from deploying these systems. For that reason, in our modelling we align scoped biases with harms, with the objective to make explicit the articulation of otherwise alleged, unstated negative consequences attributed to ML systems. However, our expectation and recommendation, is that users will enrich the proposed vocabulary with the results derived of their own explorations, in a similar line as with AI incident databases. Furthermore, bias is not singular, and highly context dependent, meaning that most biases are studied and defined in association to a particular ML application. To represent both of these concepts, we model bias:Harm and bias:Application. The central concept in our ontology is bias:BiasMeasure. This class represents a measure defined in some foaf:Document, evaluated in a dcat:Dataset (that has some characteristics), and for a particular mls:MLTask. bias:BiasEvaluation is the class that represents the n-ary relationship between entities schematized in the extended entity relationship model completed at the start of the design phase. Figure 2 illustrates a conceptual overview of the core classes and relationships of Doc-BiasO.
The Trustworthy AI framework requires a comprehensive formal vocabulary that unifies approaches and contemplates terminology and concepts of ML pipelines, and in broader terms AI, holistically. This type of resource can contribute to the generation of metadata that primes reproducibility and traceability of research results [23,24], a known issue in ML research and development [60,37]. Moreover, it can help achieve a certain degree of standarization for the area. Motivated by this, we perform an analysis of the FMO [25] and VAIR [28] ontologies, as to determine their characteristics and how they fit into our model. We also do this with the aim to achieve a good balance between ontology re-use and down the line overhead derived from doing so [55]. Key takeaways are:
1. FMO complements Doc-BiasO by giving coverage to existing fairness metrics used to evaluate ML systems. Specifically, metrics pertaining to machine learning problems of classification and regression; 2. VAIR captures a wider scope of AI system deployment to instill accountability on an AI provider (i.e., a party that places the system on the market) and thus capture specifications of risky applications of AI from a regulatory point of view; 3. both ontologies represent bias, however, with differing modelling objectives. FMO organizes fmo#Bias in a hierarchy with seven subclasses, two of these are used in relation to
Gini coefficient of the in-degree distribution To avoid constraining our modelling, we opt to not import either ontology in its entirety. When needed, we implement OWL axioms to assert class equivalence, i.e., owl:equivalentClass.
Otherwise, we reference external concepts using annotation properties.
Doc-BiasO Axiomatization The conceptualization of the Doc-BiasO ontology is specified using OWL logical axioms, given that OWL is formally defined in Description Logic. By using OWL to formalize our ontology, we enable consistency checks and logical inferences on a resulting RDF knowledge graph. Further details can be found in the ontology documentation. 12Instantiating Doc-BiasO To showcase an instantiation of Doc-BiasO, we look at an example based on bias detection in relation to recommender systems, commonly implemented in online social networks. The class Bias is instantiated as Popularity Bias. This bias is Associated With, an instance of the class Application, Recommender System and has a Bias Measure, "Gini coefficient of the in-degree distribution". In this example, Popularity Bias is Aligned With the instance of the class Harm, which is Erasure. We illustrate this in Figure 3.
The domain analysis and scope definition of Doc-BiasO, as already described in Section 3.1, derived a set of competency questions that was also used to convey the requirements that Listing 1: SPARQL Query for Competence Question Q4.1 would guide the engineering of our ontology. As part of the process, we tested and refined the Doc-BiasO ontology by implementing the formalization of the competency questions originally expressed in natural language as SPARQL queries. 13 The queries were tested to make sure the results were the expected ones. The set of queries can be access through our GitHub. 12 To illustrate their adequacy, we continue with the example introduced earlier, and start by posing Q1 "Given a particular bias, what is its definition?" ; our example uses Popularity Bias. Below the query result: "When collaborative filtering recommenders emphasize popular items (those with more ratings) over other "long-tail", less popular ones that may only be popular among small groups of users. "@en This expected result is expressed as a rdfs:Literal in English. We follow this question by posing Q4.1 "How many measures have been documented for it?". The results produced by executing the corresponding query, specified in Listing 1, are that for Popularity Bias, we have 3 measures. We choose the measure, Gini coefficient of the in-degree distribution, to learn more about it. We proceed to execute the query that corresponds to Q6. what is its formalization?. The corresponding SPARQL query is specified in Listing 2, with its execution projecting the definition for the chosen measure and the formalization for it in natural language.
As part of the evaluation process, we also report on the quality of Doc-BiasO. Table 2 summarizes the results obtained according to three indicators defined in [61].
This version of Doc-BiasO has also been validated with online tools to verify its consistency and syntactical validity, as well as to check for modelling anomalies or errors. First, we checked that our ontology is syntactically correct using the W3C RDF validation service. 14The results indicated a successful validation of our RDF document. Second, we checked for 15 We choose this engine as it is a complete reasoner. The results determined that Doc-BiasO is logically coherent and consistent. Finally, we scanned our ontology with the "OOPS! Ontology Pitfall Scanner" [62] to automatically dismiss the existence of modelling pitfalls; the evaluation results were also positive, as there were no bad practices detected by the tool.
In this work, we presented Doc-BiasO, an ontology for bias measures found in the literature that can support the elaboration of documentation of bias in machine learning pipelines. Our objective is to contribute towards improving the interpretation of these pipelines in terms of biases captured, and the derived harms attributed to ML systems. Further, we make a call for a unified controlled vocabulary for the Trustworthy AI framework, and assess existing relevant work. We technically evaluated Doc-BiasO and showcase an example of its instantiation. Notwithstanding, our work is not without limitations. First, research on bias in ML, and by extension AI, is a fast-moving field, thus providing adequate and updated coverage with our tool is a challenge. Second, bias evaluation are highly complex and context dependent tasks. This means that our modelling cannot account for all potential existing biases, and that in general, bias analysis cannot be fully automated, requiring a human-in-the-loop. Third, our resources are yet to be evaluated by AI practitioners outside a research environment. Nevertheless, the addressed limitations are an opportunity for future work. In particular, we intend to add and expand on aspects left unmodeled in this version, and we will liaise with AI practitioners to evaluate the suitability of our tool in real world scenarios. We will also continue the development of a controlled vocabulary for Trustworthy AI, as this resource can foster effective communication between the different actors involved across the AI pipeline.
We thank Guillermo Climent-Gargallo, Sammy Sawischa and Yukti Sharma for their support during this research. Mayra Russo is supported by EU-Horizon 2020 research and innovation programme under the MCSA-grant agreement No. 860630, project: NoBIAS. Maria-Esther Vidal is partially supported by Leibniz Association, program "Leibniz Best Minds: Programme for Women Professors", project TrustKG-Transforming Data in Trustable Insights; Grant P99/2020.