The notions of trust and trustworthiness in the field of AI are currently the focus of a collective, interdisciplinary efort for clarification. In this work, we contribute to this ongoing debate by identifying two senses in which an agent might place trust in an AI system. The first sense, referring to trustworthiness as formalised in previous work, considers the results of tests conducted on the system alongside the agent's expectations. The second sense, extends the former by factoring in the agent's “pragmatic” background when considering these tests. We argue that these two forms of trust can be understood in relation to well-known approaches in statistical inference: the ifrst aligns with a frequentist interpretation, while the second reflects a Bayesian view of trust.
The concepts of trust and trustworthiness in AI are currently the subject of an interdisciplinary efort
of conceptual, formal and procedural clarification. This is evident from the increasing attention these
notions are receiving across various fields of research. AI engineers, for instance, are working to
incorporate properties into AI systems to enhance their trustworthiness (see, e.g., [
The challenge of understanding trust and trustworthiness in AI is not purely theoretical. The widely
discussed European proposal for the Artificial Intelligence Act [
In this paper, we contribute to this ongoing debate by identifying two senses in which an agent might place trust in an AI system. The first sense considers the results of tests conducted on the system alongside the agent’s expectations. The second extends the former by factoring in the agent’s “pragmatic” background when considering these tests. We argue that these two forms of trust can
CEUR
ceur-ws.org be understood in relation to well-known approaches in statistical inference: the first aligns with a frequentist interpretation, while the second reflects a Bayesian view of trust.
When it comes to trusting an AI system, particularly a Machine Learning (ML) system, there are at least
two distinct ways in which an agent can do so. To illustrate this, we borrow and slightly modify an
example from [
To address this, let us first examine what allows 1 to trust . We will assume some conditions
we consider necessary for to be considered trustworthy. In the following we assume a notion of
trustworthiness for non-deterministic computational system as the one proposed in [
This notion of trust is widely referenced in the literature on evaluating AI trustworthiness (see, for
example, [
Let us return to our example of the classifier and examine the reasons why the second agent, 2, might not trust the classifier, unlike the first agent, 1. Assume that both agents possess the same knowledge about . However, 2 may have additional beliefs regarding the potential inaccuracies of the output, even if they acknowledge that such inaccuracies are expected. We argue that these additional beliefs, which lead to the divergence in trust between 1 and 2, stem from the specific pragmatic background of 2. By pragmatic background, we refer to the set of beliefs an agent holds prior to interacting with the AI system. These beliefs may be shaped by various factors, including education, experience, cultural context, and moral or ethical principles. Based on this understanding, we can characterize this extended form of trust in an AI system, which we will refer to as 2: An agent 2 an AI system if 1 and the output of is compatible with the pragmatic background of the agent.
As evident from the previous characterization, 2 extends 1 by incorporating an agent’s belief set and comparing it with the output provided by the AI system. With the definitions of 1 and 2 established, let us revisit our motivating example of the classifier and compare the two forms of trust held by 1 and 2.
Viewing the example through the lens of these definitions, we can assert that 1 1 to classify both pictures as wolves. Furthermore, 1 also 2 for the same classification, because the received output does not contradict their pragmatic background. In contrast, while 2 1 to classify both pictures as wolves, their situation diverges. 2 does not 2 for this classification. This divergence may stem, for instance, from 2’s background as a dog trainer who has previously trained a Siberian Husky. In this context, a single error from does not undermine its overall trustworthiness, and 2 is aware of this. Thus, 2 maintains 1 in . However, 2 recognizes the Siberian Husky and believes, based on their education and experience, that a Siberian Husky is not a wolf. Consequently, they would not base further reasoning or actions on this erroneous output. In this sense, 2 does not 2 , as the output contradicts their pragmatic background.
As evident from our definitions of 1 and 2, these notions are not mutually exclusive; rather, 1 is included in 2. The primary distinction is that while 1 is based solely on the calculation of the correspondence between data obtained from a suficient number of tests and an agent’s expectations about an AI system, 2 incorporates the agent’s overall background into the reasoning process. From this perspective, we argue that the two kinds of trust discussed in this article naturally correspond to two forms of statistical inference: frequentist and Bayesian approaches.
Frequentists define the probability of an event as the limit of its relative frequency over a large
number of trials, whereas Bayesians extend probabilities to account for varying degrees of certainty
about statements (see, e.g., [
As previously noted, 1 relies on the knowledge of a system’s trustworthiness, as discussed in
[
Let us reconsider our example in frequentist terms. In order to establish 1 an agent takes into account the probability of getting a result which identifies wolf as wolf (lets dub it Result), during a suficient number of tests ( Data): () = # #
Then, the agent verifies whether () matches an acceptable threshold against the expected probability for # . In our example, both agents evaluate that the observed frequency of Result sits within an acceptable threshold compared to its theoretical counterpart, thereby inferring trustworthiness. Notice, that even though for 2 the output was not accounted in the number of Result, the diference is so insignificant that () still matched an acceptable threshold.
Since 1 is included within 2, trustworthiness – and, by extension, frequentist statistical reasoning – plays a significant role in establishing 2. However, a distinguishing feature of 2 is its incorporation of the agent’s pragmatic background in the evaluation. This pragmatic background reflects the current state of the agent’s knowledge, not only regarding the results of testing an AI system (i.e., the data) but also encompassing prior information and hypotheses about the system and acceptable outcomes. From this perspective, the pragmatic background can be viewed as a prior probability, which represents the probability assigned to an output before receiving the relevant information. In this broad sense of pragmatic background, 2 seems to align with a Bayesian interpretation of the probability of the output, as it incorporates the dimension of the agent’s prior credence or degree of the agent’s beliefs, which can be updated subsequently.
Returning to our example of the classifier , we can say that 1 and 2 establish their 1 in based on the overall trustworthy behavior of the classifier, which is measured in frequentist terms. In the case of 2, however, the agents 1 and 2 appear to have diferent priors –specifically, difering knowledge and assumptions about dogs and wolves – which influences their attitudes toward the output. From this perspective, we notice that the conditional belief (posterior belief in Bayesian terms) of 1 and 2 difers, once it is calculated via Bayes’ theorem:
( ∣ ) × () ( ∣ ) = , () that is the conditional belief in event ( ( ∣ ) ) is calculated by multiplying prior belief of the agent by the likelihood ( ∣ ) that will occur if is true. Clearly, ( ∣ ) would be significantly diferent for 1 and 2, given that () is diferent for them, as well as the likelihood for 1 is much higher than the one for 2.
The association of the two forms of trust we introduced with established methods of statistical inference supports the distinction between 1 and 2. These forms of trust allow for a focus on diferent objectives when evaluating an AI system. Specifically, 1 can be seen as measuring the reliability of a system with respect to a benchmark, while 2 involves an agent’s attitudes and hypotheses, which may not be directly tied to the AI system itself. A notable aspect of our analysis is the relationship between trust and trustworthiness, where trust inherently presupposes trustworthiness. In our framework, trustworthiness is always relative to the agent. From this perspective, it seems natural to assert that if an agent trusts an AI system, they must perceive it as trustworthy. However, the reverse is not necessarily true: an agent may consider a system trustworthy without actually placing their trust in it. Formally, both 1 and 2 can be applied depending on the desired level of abstraction in the model. The development of a formal framework to represent these two types of trust remains a topic for future research.
The authors would like to thank two anonymous reviewers for their comments. All authors acknowledge the support of the Project PRIN2020 BRIO - Bias, Risk and Opacity in AI (2020SSKZ7R) awarded by the Italian Ministry of University and Research (MUR). Giuseppe Primiero is further funded through the project PRIN2022 SMARTEST - Simulation of Probabilistic Systems for the Age of the Digital Twin (2022E8Y4X) awarded by the Italian Ministry of University and Research (MUR). The research of Ekaterina Kubyshkina is funded under the “Foundations of Fair and Trustworthy AI” Project of the University of Milan. Giuseppe Primiero and Ekaterina Kubyshkina are further funded by the Department of Philosophy “Piero Martinetti” of the University of Milan under the Project “Departments of Excellence 2023-2027” awarded by the Ministry of University and Research (MUR). Mattia Petrolo acknowledges the financial support of the FCT – Fundação para a Ciência e a Tecnologia (2022.08338.CEECIND; R&D Unit Grants UIDB/00678/2020 and UIDP/00678/2020) and the French National Research Agency (ANR) through the Project ANR-20-CE27-0004.