Introduction

This work is part of ongoing discussions within the Knowledge Graph Alliance's working group Explainable-AI-ready data and metadata principles (XAIR principles). 1 This working group will develop recommendations (principles) that, if followed, contribute to making data and models explainable-AIready (XAIR). This will be supported by metadata standards, specifically, for epistemic metadata, i.e., metadata related to the knowledge status of data and models. To our understanding, data and models are XAIR to the degree that they are semantically enriched in such a way as to facilitate making best use of explainable learning techniques, broadly understood. That is, first, this does include XAI as it is commonly understood in a narrow sense, such as techniques based on Shapley values or local interpretable model-agnostic explanations [1]. But it also extends to deductive reasoning, since a logical or mathematical proof can be considered an explanation of that which is being proven; there, our group is particularly interested in providing a XAIR documentation to answer set programming as well as reasoning applied to (fragments of) OWL description logic. Third, our notion of explainable learning also includes simulation based on models that have some structural agreement with the phenomenon that is being modelled; there, most saliently, physics-based modelling such as molecular simulation. Eventually, it is the aim of the working group to advance good practices in data and metadata documentation that help feeding models and data obtained in all sorts of ways into frameworks where they become reusable to all these modalities of explainable learning. This means that not only the output from an explainable learning technique has to become XAIR, but also the input -such as the datasets used for training, validation, and testing in machine learning, or experimental data that are used to parameterize and validate a molecular model. It is as of now not yet characterized what exact prerequisites need to be met to achieve «XAIR status»; instead, this is supposed to be a result from the working group's community consultations. We expect it to be centered on model and data documentation through appropriate metadata, just as it is the case for the FAIR principles [2]. Accordingly, the recommendations for explainable-AI-readiness will build on FAIR, and then go beyond it.

It is a notorious issue in XAI that core concepts such as explainability, interpretability, etc., are most often used without any clearly communicated conceptualization of what they would entail [3]; but in a technical context, clear definitions of technical terms are a necessity, all the more for a task like ours: Supporting data documentation for XAI-readiness. To our work, central questions at this stage include:

1. What properties do objects 𝑜 and 𝑝 need to have as a prerequisite for 𝑜 relying on 𝑝? 2. How about 𝑜 trusting 𝑝? 3. What characterizes the difference between trust and reliance? 4. What characterizes the difference between explanations and grounding? 5. What semantic artefacts need to be put in place so that the above can be documented?

The XAIR principles WG has been formed in March 2024, and its work plan for the time being extends over 40 months, until June 2027. At the present stage, we are engaged with meta-reviewing what perspectives from the literature need to be taken into account, regarding questions such as those mentioned above, but also to identify the core concepts to XAI-readiness and explore what definitions they have been given. The outcome of this exploration will feed into discussions on actionable definitions of the key concepts that the WG itself will accept; these will then be used for a revision of the previously developed ontologization of cognitive processes and epistemic metadata [4]. First, however, that preexisting ontology, PIMS-II, needs to be redesigned into a simpler framework to become more amenable to the planned work. It will for that purpose be split into multiple lite mid-and domain-level OWL 2 EL ontologies, improving tractability of reasoning tasks as well as intelligibility to developers. Eventually, as an output from the group's work, reference ontologies will be provided to users who want to use them to document models and data in a XAIR way. Nonetheless, «XAIR status» is to be ontology-agnostic, and users will in no way be required to use these ontologies or give preference to them over others that support achieving compliance with the recommendations for explainable-AI-readiness equally well.

Literature on the core concepts for explainable-AI-readiness

A list of XAIR core concepts has been issued as part of a recent announcement [5]. It comprises: «Explainability and explanation; reproducibility, reliability, and reliance; opacity and transparency, interpretability and interpretation; DIKW: Data, information, knowledge, and wisdom; responsibility, trust, trustworthiness, and [. . . ] motivations for trusting; model design, parameterization, and optimization; holistic validation and unit testing (of models and simulation codes); theoretical virtues (of models); epistemic agents, vices, and virtues; the four elements of FAIR [. . . ]; simulation; applying and evaluating models; context awareness, subject matter, and logical subtraction» [5]. At present, we are conducting a preliminary literature analysis on a subset of the above. We call this a meta-review, since the resources that we are evaluating are themselves already reviews (or other aggregated material). The aim is not to obtain a complete understanding of how all these matters have been discussed in the literature from all possible angles, but to sample a variety of perspectives as an initial guidance to discussions about recommendations for good practice in explainable-AI-ready data and model documentation.

Wisdom hierarchy as reviewed by Rowley

A hierarchy according to which data (D) can be analysed/improved qualitatively first to information (I), then to knowledge (K), and finally to wisdom (W) has a long tradition in data management and is often visualized as a DIKW pyramid. This «hierarchy» is not a taxonomy, i.e., wisdom is not understood to be a subclass of data, or vice versa; instead, D → I → K → W is a kind of value chain or dialectical sequence. For the present exploration, the review by Rowley [6] is our starting point. The review is from 2007, but measured by the time scale of progress within epistemology, that means it is recent.

Agency as reviewed by Conte

Within the present framework, we want to keep the concept of agency at a very general level, so that (almost) anything can be analysed as an agent, similar to the use that is made of the concept of a system in thermodynamics. Here, if a system has something that can be understood as sensors and actuators or is by means of some other mechanism capable of perceiving and doing, it can be analysed as an agent. This makes it all the more important to include a taxonomy of agents as part of an associated mid-level ontology, since more specific subclasses will be needed to talk about kinds of agents. Our point of departure in this field is given by the review and the definitions proposed by Conte [7].

Reproducibility as reviewed by Plesser

Reproducibility is a key component of industry standards, as seen in Quality Management Systems like ISO 9001, which emphasize reproducibility through consistent quality management practices, requiring detailed documentation and standardized procedures to ensure reliable process repetition [8]. Journals are now encouraging or requiring the submission of data and code alongside published articles to enable the replication of results, while platforms like the Open Science Framework promote open access to research materials, enhancing reproducibility by providing researchers with the tools to share their data, code, and protocols openly [9]. However, the drive for novel results obtained from complicated methods often leads to practices that hinder reproducibility [10]. A balance is needed to ensure that research outputs are both innovative and reproducible [11]. Plesser's concise review [12] is the starting point for our exploration of frameworks suitable for documenting reproducibility claims and the validation of such claims; see also Section 3.3, as well as previous work where this was discussed more in depth [13].

Trust as reviewed by Ohlhorst and the ONTrust reference ontology

Ohlhorst [14], beside making his own philosophical argument on the nature of trust, provides an review on concepts that are central to his theory, in particular, hinge, entitlement, and virtue. Ohlhorst's work and the state of the art summarized therein need to be taken into account for our working group's discussions on the relation between explanations and trust in AI systems (Section 3). Attempts at ontologization can build on the ONTrust reference ontology by Baratella et al. [15].

Reviews from the OntoCommons and CHEIKHMAT projects

The Review of Domain Interoperability [16] (RoDI) is a deliverable from the OntoCommons project (H2020 GA no. 958371) that preceded the formation of the Knowledge Graph Alliance. The purpose of the RoDI document is to provide a foundation to discussions and technical innovation addressing domain-level interoperability now that OntoCommons is completed and in its place, the KGA has been created as a self-sustaining organization. As such, RoDI both reviews and provides recommendations for designing semantic artefacts. These fit into the OntoCommons ecosystem [17] (OCES) that permits a plurality of foundational ontologies (DOLCE [18], EMMO [19], BFO [20]), and strategies for semantic heterogeneity and alignment, such as bridge concepts. This approach forms part of the background to the ongoing ontology redesign (cf. Section 4) within the KGA's working group on XAIR principles.

In addition, the interoperability requirements matrix from RoDI [16, Section 5.1] develops an understanding of key concepts that can be related to the wisdom hierarchy, at the level of successfully communicating knowledge that, in the DIKW pyramid, is referred to as wisdom [6]. There, RoDI distinguishes interoperation of data, humans, software, and organizations, and three interoperability levels: Syntactic, terminological/semantic, and pragmatic [16]. In addition to this output from OntoCommons, the CHEIKHMAT project has recently produced a review that very closely connects to our ongoing survey of key concepts, with a focus specifically on ontology-based and semantics-based XAI [21].

The list of sources and topics included in this first effort is far from sufficient. Given the task of the working group, it urgently has to be expanded by catalogues or reviews of explainability metrics [1], since this is a key part of what will need to be documented in practice. As an output by month 10 of the group's work plan (December 2024), a Synopsis of XAIR Core Concepts will be delivered, which will take the form of proceedings from a workshop organized by the projects AI4Work and BatCAT [5].

Exploration of actionable key concepts

Reliance and reliability

It does not make sense to speak of an agent 𝐴 as relying upon something unless it is done in view of some objective 𝜏 . In other words, there must be an adverse consequence for the agent if the relationship fails; namely, that an expected outcome is not achieved. We do not for now require goal-directedness, i.e., 𝐴 is not required to have an awareness or internal representation of 𝜏 as the agent's intention; but it is necessary for 𝐴 to be goal-oriented, i.e., have an objective (conscious or not). The entity acting as the subject of a reliance relationship (that which does the relying) must therefore be a goal-oriented agent.

The reliance relation can be formalized as:

• The ternary predicate «𝐴 relies on 𝐵 about 𝑞», where 𝐵 is an interlocutor, i.e., another agent (this can also e.g. be a queriable software running on a machine), and 𝑞 is the subject matter that the relationship is about. This realizes itself by 𝐴 relying upon the following rule: «The antecedent "agent 𝐵 asserts 𝑐, where 𝑐 is about subject matter 𝑞" necessarily implies the consequent 𝑐.» • A more general ternary predicate «𝐴 relies upon rule 𝑟 for 𝜏 », where 𝑟 : □ ℓ (𝑎 → 𝑐) can be any rule according to which antecedent 𝑎 necessarily (to the degree ℓ) implies consequent 𝑐. This is to be interpreted loosely, e.g., 𝑎 and 𝑐 will usually share free variables, with universal quantification applied to these variables. For the reliance relationship to work in practice, it is necessary for 𝐴 to also rely upon 𝑎: At some point in time, 𝐴 may know (or not know, but act as if) 𝑎 holds, so the rule can be applied. Otherwise, reliance or non-reliance upon 𝑟 remain indistinguishable.

The index ℓ of the necessity operator represents the level of necessity at which the rule 𝑟 is understood to apply; e.g., in some cases, exceptions may be tolerated, and in some cases, there may be a margin within which quantitative deviations can be accepted. Descriptors for this can become complicated, see the discussion of what-you-see-is-what-you-get guarantees in Grote et al. [22]. For the predicate «relies on», the level ℓ(𝑞) is understood to be contained in the subject matter 𝑞. Here, the notion of aboutness or subject matter can be taken to be that from Yablo [23]. • The even more general ternary predicate «𝐴 relies upon 𝜙 for 𝜏 », where 𝜙 can be any proposition, not restricted to a rule. The reliance relationship only makes sense if 𝜙 somehow causally contributes to 𝜏 : It must deliver some contribution to the agent 𝐴 reaching the goal 𝜏 .

Rule 𝑟 : □ ℓ (𝑎 → 𝑐) is reliable whenever it is true, i.e., whenever 𝑎 implies consequent 𝑐 to the level ℓ.

In action, the relationship of reliance is not constituted by the agent 𝐴 believing or knowing that the rule holds; i.e., that under all the described conditions, 𝑐 is actually true. That may or may not be the case. Instead, the agent 𝐴 practices reliance upon a rule 𝑟 by behaving as if 𝐴 knew that 𝑟 was reliable, or in terms that will be clarified further below (Section 3.2), in 𝐴 emulating an agent 𝐴 ′ such that 𝑟 is grounded to 𝐴. The emulated agent 𝐴 ′ is identical to 𝐴 in all respects other than about acceptance of 𝑟 as knowledge. The relation of relying on someone can accordingly be expanded as 𝐴 relies on 𝐵 about 𝑞 ≡ 𝐴 relies upon rule 𝑟(𝐵, 𝑞) : □ ℓ(𝑞) ∀𝑐 ((𝑐 is about 𝑞 ∧ 𝐵 asserts 𝑐) → 𝑐)

≡ about 𝑞(𝑟), 𝐴 emulates 𝐴 ′ , who is like 𝐴, except that 𝐴 ′ knows 𝑟(𝐵, 𝑞).

Therein, 𝑞(𝑟) refers to the scope of reliance upon 𝑟. The above does not exclude the case where 𝐴 really knows 𝑟 to be true. In that case, it simply reduces to 𝐴 = 𝐴 ′ ; i.e., grounded reliance does not require 𝐴 to emulate another hypothetical agent who accepts 𝑟 as knowledge, since 𝐴 already is that agent. Based on the above, we can characterize agents' reliability: Interlocutor 𝐵 is reliable about 𝑞 whenever the rule 𝑟(𝐵, 𝑞) from the right-hand side of Proposition ( 1) is reliable, i.e., whenever it is true.

Grounding and explanations

A proposition/rule and the associated reliance relationship is grounded to some agent 𝐶 if that agent knows it to be true. The most relevant case of this is applying this definition with 𝐴 = 𝐶, i.e., asking whether the agent who does the relying has sufficient knowledge to ground that reliance relationship (if that is not the case, we can infer that the agent trusts in the relationship, instead of knowing). However, we can also insert ourselves as 𝐶; in this case, the question becomes whether we know the rule that is relied upon to be true, i.e., whether 𝐴's reliance relationship is grounded to us (only then, we would say that the rule is reliable). Following this approach, reliance is grounded in knowledge. Accordingly, first, some information would need to be established as having the status of knowledge -grounding of reliance presupposes grounding of knowledge. The question of what exactly can ground knowledge, and how that is to be documented, is at the core of the problem of standardizing epistemic metadata [4].

Reliance does not require any grounding, i.e., we can rely on someone or upon something without knowing that this will be successful and lead to the intended outcome. In that case, we say that we trust.

Any relationship of trust presupposes that there is a relationship of reliance that is ungrounded to the trustor 𝐴, i.e., the relying and trusting agent: Where knowledge would be needed to ground the reliance relationship, that knowledge is absent. This can be either because the rule that the agent relies upon is in fact unreliable, or because while it is in fact reliable, that fact is not known to 𝐴. With Proposition (2) we can summarize that according to our tentative formalization proposed here, if 𝐴 trusts in (and, therefore, does not know) some 𝜙, that is realized by 𝐴 emulating an agent 𝐴 ′ who is like 𝐴, except that 𝐴 ′ knows 𝜙 (and, therefore, does not trust in 𝜙). This emulation is limited to a scope 𝑞(𝜙), namely, the scenarios that the proposition is intended to be applied to, i.e., those that it is about [23].

Similar to reliance, trust does not require any grounding: An agent can trust blindly in someone or something without any good reason at all. However, trust can also be grounded [24], i.e., an entitlement (epistemic warrant) can be provided in many different ways [14], all of which can be called explanations.

Societal norms and community practices can impose limits on what behaviour around trust is acceptable:

The agent can be held responsible by the community to trust only in appropriate agents or propositions. In science, naturally, blind trust is not tolerated, while as Koskinen [25] points out, some trust is necessary. The space between the necessary and the forbidden trust in data-driven modelling would need to be characterized by a social epistemology that, following Koskinen, still needs to be developed [25].

For guidelines that advance XAIR models and data, it is therefore a requirement to develop metadata standards (cf. Section 4) for documenting three kinds of grounding: First, epistemic grounding, or grounding of knowledge, in line with practices that are accepted as scientific within a scientific community; second, grounding of reliance by knowledge; and third, grounding of trust by explanations.

Reproducibility and transparency

Reproducibility ensures the capacity to consistently replicate results under identical conditions, providing reliability and verification, cf. Section 2.3. Key elements include documentation, standardization, and data availability; where these are absent, we speak of dark data [26,27,28].

Good practices in research data management avoid dark data and facilitate reproducibility, e.g., by complying with suitable documentation and transparency requirements. Transparency involves providing clear, accessible, and understandable information about processes, enabling scrutiny and understanding. The key components include clarity, accessibility, accountability, and traceability.

We remark that the notion of an «implication necessary to level ℓ» proposed in Section 3.1 for rules upon which agents rely is similar to, and can probably be unified with, the «conditional necessity of 𝜙, given 𝜅» proposed in previous work [13] to formalize the semantics of a reproducibility claim.

Mid-and domain-level ontology redesign

Once the discussions on the core concepts are further progressed, the synopsis has been copiled, and own definitions of core concepts are sufficiently mature, our working group will provide recommended metadata standards for annotating models and data in a way that improves their XAI-readiness. There is a major element to this from previous work that will require a redesign: The PIMS-II ontology [4] was developed for cognitive processes (with a focus on research workflows as cognitive processes) and epistemic metadata (knowledge claims, provenance, and validity claims). This is, however, too broad for a single ontology to be easily accessible to users; it is even less suitable for a focused discussion of key concepts, since it will be hard to adjust such a monolithic conceptual scheme to novel perspectives that might be agreed upon as an outcome from group discussions. In addition, PIMS-II has developed substantial philosophical overhead (with a FOL axiomatization of its core [29] that distracts from its purpose and is little accessible to its potential user base), created from an attempt [30] to connect to the metaphysics underlying the EMMO foundational ontology [19]. However, with the development of OCES [17,31], it is no longer needed or even advisable for mid-and domain-level ontologies to attach themselves to the EMMO so closely, just to interoperate with other EMMO-related ontologies. OCES and bridge concepts [31] can just as well be used for this purpose [16,17]. Another element that might be reused, contingent upon redesign, are (some of) the VIMMP ontologies [32]; the most relevant ontologies from VIMMP would be OSMO [33,34], the ontology version of CWA 17284 MODA [35], and MMTO [36], the ontology version of the EMMC CSA Translation Case Template [37]. The VIMMP ontologies, specially OSMO, suffer from the same problem as well: Substantial work went into them, and now, a single ontology addresses too many issues at once, reducing its usefulness for targeted development tasks; beside, like PIMS-II, the VIMMP ontologies too became complicated through attempting to conform with preliminary development versions of the EMMO and its metaontological overhead (and, in the case of OSMO, additionally with MODA). Both PIMS-II and the VIMMP ontologies also make heavy use of OWL 2 Full semantics and do not prioritize the tractability of reasoning tasks.

Generally speaking, mid-and domain-level ontology redesign should focus on refining and reorganizing ontological structures to significantly improve clarity, usability, and alignment with domain-specific requirements, ensuring the ontology accurately represents the required knowledge and relationships within a domain. This process facilitates better interoperability, data integration, and reasoning capabilities. Essential considerations in this redesign include achieving conceptual clarity by defining all concepts and relationships without ambiguity, removing redundancies, and resolving inconsistencies to enhance the ontology's functionality and usability [38]. Organizing concepts into a clear hierarchical structure is critical; this involves not only identifying and establishing natural or logical parent-child relationships but also ensuring that subcategories are appropriately nested to reflect the true complexity of the domain [39]. The design should be modular, allowing for easy updates and extensions, which supports ongoing improvements and adaptability. The ontologies should also be aligned with a foundational ontology and widely used, or otherwise important, metadata standards from the domain [38]; in our case, the latter include the reference ontology of trust [15] (cf. Section 2.4), the German NFDI4Ing project's Metadata4Ing [40] (m4i), and further domain ontologies as a function of the addressed use case. Within BatCAT, e.g., EMMO-related domain ontologies such as the BattINFO ontology [41] and the battery value chain ontology (BVCO). In some domains, BFO-related ontologies would predominate [38]. Here, DOLCE [18] is used as the foundational ontology with which a strong alignment is made, using DOLCE Lite [42]. DOLCE has been stable for over twenty years and can therefore be relied on as a building block that will not change in the foreseeable future. In relation to other work, we can rely on weak alignment based on the SKOS relation skos:closeMatch and additionally benefit from OCES for connecting to any ontology development that has been done under the BFO and the EMMO [16,17].

The redesigned mid-and domain-level ontologies, called MSO-EM: Ontologies for modelling, simulation, optimization (MSO) and epistemic metadata (EM), are/will be licensed under CC BY-ND 4.0, and their development is openly accessible through a public github repository. 2 The design principles include:

1. The expressivity is restricted to OWL 2 EL, corresponding to ℰℒ++ description logic. 2. Each ontology from MSO-EM has at most three taxonomy levels and three concepts at its highest level. (This does not mean that there will only be three hierarchy levels within MSO-EM as a whole, but that the tree depth for the concepts defined in a single module TTL file is limited to three.) Only include concepts that we expect to be directly required in technical practice. Remove unnecessary concepts in the middle region of the tree, if it is the leaves of the tree that developers would rather refer to instead. 3. Define few relations, prefer abstract (high-level) relations where possible, and be specific in terms of the concepts rather than the relations. There is no inherent logical reason to prefer this, but it is in line with human communication -we tend to use few, comparably generic verbs most of the time. On the other hand, the nouns we use in speech or writing easily become very specific. 4. Define relations one-way only; where applicable, use the direction that can be used to define more meaningful restrictions on concepts. Normally, this means that the greater (or whole) entity should be the subject and the smaller (or part) entity should be the object, i.e., relations of the type «has part» are preferred over those of the type «is part». This is because OWL EL does not allow owl:inverseOf, and the greater-to-smaller direction is more frequently used in restrictions. 5. Use strong alignment (rdfs:subClassOf and rdfs:subPropertyOf) only in reference to DOLCE Lite, not in reference to other ontologies outside the MSO-EM system itself. All concepts from MSO-EM are subsumed under a concept from DOLCE Lite. Where it can be done, use skos:exactMatch for strong alignment with DOLCE Lite, m4i, and PIMS-II, but not with any other ontologies. 6. Use weak alignment (skos:closeMatch) where possible for D-SI, EMMO, GPO [43], schema.org, and any of the legacy ontologies other than PIMS-II; skos:broader and skos:narrower are not used.

At submission time, one of these ontology modules is ready as a demonstrator:

The MSO-EM agency ontology (agency.ttl in the repository 2 ), which implements Conte's taxonomy [7] and relations and concepts connected to it; compared to the Agent branch of PIMS-II, 3 the taxonomy was reduced to three hierarchy levels, eliminating e.g. the concept IntelligentAgent for which, there is little use in concrete practice, since almost always, more specific subclasses would be used instead.

Due to the restrictions imposed on each single ontology, the number of ontologies in this system will be comparably large, but each of those is expected to be easy to understand. Simplicity in terms of interconnections and a comparably flat taxonomy will make it possible to adjust definitions to the outcome of community and group discussion processes; at least, this will be less problematic than in case of the large monolithic ontologies from previous work such as OSMO and PIMS-II.

Conclusion

While this position paper reflects the overall perspective from discussions within the KGA working group on XAIR principles, much of the uptake and further development will depend on the actual use made of these developments for semantic interoperability, data documentation, and the integration of models and data into AI-driven systems. The research projects involved in the present effort, so far, include AI4Work, BatCAT [44], DigiPass CSA, and MaRDI [45]. 4 Within BatCAT, based on the project's requirements analysis [44], plans include to give ontology design principles such as those from Section 4 the status of a meta-metamodel (M3-model), at the top of a hierarchy in compliance with the Object Management Group's ISO standard Meta Object Facility [46] (MOF). Accordingly, the ontologies themselves become the metamodel (M2-model), below which knowledge graph shapes, specified through OO-LD, 5 are the model (M1-model). Using MOF helps establish a coherent system of semantic and technical interoperability, capable of supporting advanced data operations through integration with other OMG standards such as BPMN, which is also used in the project.