The current literature clearly highlighted the need to define a fast and eficient tool for trust assessment, even in lack of direct information, as much as possessing mechanisms allowing a matching between a selected task and a reliable agent able to carry it out. Direct experience plays a big part, yet it requires a long time to ofer a stable and accurate performance and this characteristics may represents a strong drawback especially within huge agents' communities. We support the idea that category-based evaluations and inferential processes represent a useful resource for trust assessment. Within this work, we exploit simulations to investigate how eficient this inferential strategy is, with respect to direct experience, focusing on when and to what extent the first prevails on the latter. Our results suggest that in some situations categories represent a valuable asset, providing even better results.
the goal by executing a task , which afects the world. The most interesting case occurs if the agent want/must delegate the execution of .
At an abstract level, each agent possesses some skills and resources, defined here as features which determine its ability in carrying out the tasks it has to face. Nevertheless, not all the features associated with an agent are crucial for the execution of and the majority of these will not even be necessary. If I were to ask someone to cook for me, it would be interesting to know how fast she/he is, how good is her/his culinary imagination or if she/he knows how to cook specific dishes; however, knowing that she/he loves reading astrophysics books would not be of any help.
It is therefore a fundamental precondition that an agent identifies which features are necessary to carry out . Then, needs a mental representation of any other , which comprises, at least, the subset of the features which are relevant to execute . It is also important to underline that just the possession of these features is not enough, it is also very relevant ’s willingness (following its motivations) to actually realize . Of course, two diferent tasks, say 1 and 2, require diferent features to be eficiently addressed.
Thanks to its mental model, is able to estimate the likelihood (, ) that will positively
bring to completion that specific agreed task, for each agent ∈ . The function (, )
measures the degree of trust [
It is crucial to point out that the assessment of trust is not only task-dependent but also context-dependent, because external causes may amplify or downsize the trust between the trustor and trustee. For instance, assume that wants to get to the airport one hour before the departure of her/his flight and suppose that is confident that is able to safely drive and she/he is knowledgeable of obstacles to trafic flow (e.g., limited access roads), and thus, puts a high degree of trust in . However, unforeseen circumstances (e.g., stucks in a trafic jam) may prevent from being at the airport at the scheduled time: such an event negatively influence the trust from to , even if, of course, the liability of is limited.
The procedure to select the agent to which the task has to be delegated is thus entirely driven from the calculation of the function (, ): the trustor should select the agent ⋆ for which (, ) achieves its largest value, i.e., ⋆ = arg max (, ). Such a protocol is, unfortunately, infeasible in real-life applications: in fact, is capable of estimating the trust of those agents – in short – with which it interacted in the past and of which it knows features. In real-life applications, we expect that the size of is much larger than that of and, thus, the search of a successful partner is likely to end up in a failure.
An elegant solution to the problem of selecting partners in large agent communities is
described in [
Broadly speaking, a category is a subset of agents in such that each category member possesses homogeneous features. Their unique nature makes categories very interesting and particularly useful. Since the members of a category possess similar features, even their performance concerning the same task will be similar. For sure, we have to consider a certain degree of uncertainty, due to the specific peculiarities of the individuals.
The specific categories to take into consideration change with the context and with the task of interest. For instance, suppose that correspond to a community of people working in food service with diferent roles; chefs, waiters, and sommeliers are possible examples of categories in this context.
Because of the existence of categories, the set of agents that the trustor can evaluate significantly expands in size and it consists of the following type of agents: 1. The aforementioned set , which consists of the agents with which has had a direct experience. 2. The set of agents, such that each agent ∈ belongs to at least one of the categories = {1, 2, . . . , }; here we suppose that has had a direct experience with at least one agent in each of the categories in . 3. The set of agents with which had no direct experience but which have been recommended to by other agents in (for instance, on the basis of their reputation). 4. The set of agents , such that each agent in belongs to a category which contain at least one agent in .
Advantages arising from the introduction of categories have been extensively studied in past
literature [
As the authors of [
When all three of these conditions are met, then the category ’s evaluation can be used for the agent , concerning the task since, by definition of category, all agents in will share the same features of and, thus, if the other agents in are able to successfully execute the task (or not), we can reasonably assume that even can do it (or not).
Of course, only some of the categories 1, . . . , possess the qualities to successfully execute the task while others do not. As a consequence, the first step to perform is to match the task with a set of categories capable of executing .
At a basic level, such a matching could be implemented through a a function (, ) which takes a category and a task and returns True if agents in are capable of executing , False vice versa. The computation of the function requires an analytical and explicit specification of: (a) the chain of actions to perform to execute and (b) for each action mentioned in (a), the features an agent should possess to perform such an action.
The protocol above easily generalizes to the case in which the trustor has a limited experience (or, in the worst case it has no previous experience): in this case, in fact, the trustor could leverage the sets of agents and .
The growing need to deal with bigger and bigger agents’ networks makes it dificult to find
reliable partners to delegate tasks. It becomes clear that, in such situations, direct experience
[
Several authors proposed trust propagation as a solution to this topic. Trust propagation
[
Many articles have discussed the use of categories/stereotypes in trust evaluations [
Pursuant to these considerations, our contribution within this work concerns the investigation of how eficient this inferential strategy is, with respect to direct experience, focusing on when and to what extent the first prevails on the latter.
In this section we illustrate our procedure to estimate the performance (in short called the true quality) ( ) of agents in the category to successfully execute a particular task .
Because of the assumptions of our model (illustrated in Section 1), agents belonging to the same category share the same features and, thus, their performances in executing are roughly similar; this implies that if an agent ∈ is able (resp., not able) to execute , then we expect that any other agent ∈ is also able (resp., not able) to execute .
In the following, we suppose that agents in are able to execute , i.e. in compliance with notation introduced in Section 1, we assume that (, ) = True. In contrast, if (, ) = False, it does not make sense to estimate ( ).
The next step of our protocol consists of selecting one of the agents, i.e., the trustee, in
to which delegate ; to this purpose, we could select, uniformly at random, one of the agents
in , as illustrated in [
In the light of the reasoning above, the true quality ( ) quantifies the expected performance of an arbitrary trustee in in the execution of .
We assume that ( ) ranges in (−∞ , +∞): positive (resp., negative) values of ( ) are an indicator of good (resp., bad) performances.
The first step to compute ( ) consists of modelling the performance ( ) of an arbitrary agent ∈ in executing . To capture uncertainty in the performance of , we represent ( ) as a Gaussian random variable with mean and variance 2 .
The assumption that all of the agents in the same category should reach the same performance implies that = ( ) for each agent ∈ .
The variance 2 controls the amount of variability in the performances of the agent : large (resp., small) values in 2 generate significant (resp., irrelevant) deviations from ( ). In this paper we considered two options for 2 , namely: 1. Fixed Variance Model: we suppose that 2 = 2 for each ∈ . 2. Random Variance Model: we suppose that 2 is a uniform random variable in the interval [, ].
Based on these premises, the procedure to estimate ( ) is iterative and, at the -th iteration it works as follows: a) We select, uniformly at random, an agent, say from b) We sample the performance ^ () ∼ ( ) of
Steps a) and b) are repeated times, being the number of agents we need to sample before making a decision. In addition, in Step a), agents are sampled with replacement, i.e., an agent could be selected more than once. The algorithm outputs the average value of sampled performances, i.e.:
E() = E(E( | )) Var( ) = E(Var( | )) + Var(E( | )) Our algorithm actually converges to the true value ( ) as stated in the following theorem: Let be the number of agents queried by our algorithm and let ^( ) be the estimation of the true quality ( ) the algorithm returns after rounds. We have that in both the fixed variance and random variance models ^( ) converges to ( ) at a rate of convergence of √1 .
Let us first analyze the individual agent performances ( ) and we are interested in
computing the mean and variance of ( ). If we opt for the Fixed Variance Model, then ( ) is a
Gaussian random variable with mean ( ) and the variance is equal to a constant value 2.
In contrast, if we are in the Random Variance Model, then the estimation of the mean and the
variance of ( ) can be obtained by law of total mean and the law of total variance [
We apply Equations 2 and 3 to = ( ) and = ; if we condition on = , then ( ) is a Gaussian random variable with mean equal to ( ) and variance equal to and therefore:
E( ( )) = E(E( ( ) | = )) = E( ( )) = ( ) In addition, and which jointly imply
E(Var( ( ) | = )) = E( ) = +
2 Var(E( ( ) | = )) = Var(E( ( ))) = Var( ( )) = 0
+ Var( ( )) = 2
As a consequence, independently of the agent , we have that the agent performances ( ) have the same distribution which we denote as ( ). Therefore, in both the Fixed Variance Model and Random Variance Model, the algorithm selects a random sample of agents 1, 2, . . . , of size in which, for each such that 1 ≤ ≤ , is the average performance of the agent selected at the -th iteration and it is distributed as ( ). The algorithm calculates: = 1 + 2 + . . . + (4)
Because of the Central Limit Theorem [
We designed our experiments to answer two main research questions, namely: RQ1 What are the benefits arising from the introduction of categories in the selection of a trustee against, for instance, a pure random search or a direct-experience based strategy? RQ2 How quickly our algorithm to estimate ( ) converges?
In what follows, we first describe a reference scenario in which our task consists of recruiting a chef from a database of applicants (see Section 4.1). Then, in Sections 4.2 and 4.3, we provide an answer to RQ1 and RQ2.
We assume that features associated with our task are as follows: (i) Culinary Education, measured as the (overall) number of hours spent in training courses with qualified chef trainers, (ii) Expertise, i.e., the number of years of professional experience, (iii) Language Skills, defined as the number of foreign languages in which the applicant is proficient, (iv) Culture, measured on Task 1 2 3 1 2 3 4 5 a scale from 0 (worse) to 10 (best) and which is understood as the ability of preparing diferent kind dishes (e.g. fish, meat, vegetarian and so on) in diferent styles (e.g. Indian, Thai or Italian) and (v) Creativity, measured on a scale from 0 (worse) to 10 (best). The list of features is, of course, non-exhaustive. We suppose that each feature is associated with a plausible range: for instance, in Table 1, we consider three potential tasks and the corresponding requirements.
In the following, due to space limitations, we concentrate only on the task 1 and we suppose that five categories exist, namely: Professional Chefs - 1, who are trained to master culinary art. Members in 1 are able to provide creative innovation in menu, preparation and presentation, Vegan Chefs - 2, specialized in the preparation of plant-based dishes, Pastry Chefs - 3, who are capable of creating chocolates and pastries, Roast Chefs - 4, who have expertise in preparing roasted/braised meats and Fish Chefs - 5, who are mainly specialized in the preparation of dish ifshes. Each category consists of 100 agents and, thus, the overall number of agents involved in our experiments is 500. Features associated with categories 1-5 are reported in Table 2.
In our scenario, only agents in 1 are able to fulfill 1; agents in other categories are, for diferent reasons, unable to execute 1: for instance, the expertise of agents in categories 2, 3 and 5 is not suficient while agents forming categories 3-5 correspond to applicants with a high level of specialization in the preparation of some specific kind of dishes (e.g., fish-based dishes) but they are not suficiently skilled in the preparation of other type of foods and, thus, agents in these categories showcase an insuficient level of culture.
To simplify discussion we suppose that, through a proper normalization, the performance ( 1) (see Section 3) of an individual agent as well as the true quality ( 1) of a category (for l = 1 . . . 5) range from 0 to 1. Here, the best performance of an agent can provide (resp., the highest true quality of a category) is 1.
In our first experiment, we compare three strategies to search for a trustee, namely: a)
RandomBased Search: here, the trustor selects, uniformly at random, a trustee to execute 1. The trustor
measures the performance ( 1) provided by the trustee in the execution of 1 . b)
CategoryBased Search: here, the trustor considers only agents in the most appropriate category (which
in our reference scenario coincides with 1); as suggested in [
In our experimental setting, we considered two values of , namely = 10 and = 30 and we discuss only results in the Fixed Variance Model with = 0.05 and = 0.15. We applied the Random-Based, the Category-Based and the Direct-Experience Search strategies 20 times; a sketch of the probability density function (pdf) of ( 1) for each strategy is shown in Figure 1 and 2.
As expected, Category-Based Search performs consistently better than the Random-Search one. In addition, the standard deviation of ( 1) in Category-Based Search is much smaller than that observed in the Random-Based strategy and such a behaviour depends on the diferent degree of matching of categories 1-5 with the task 1: in other words, if the trustee is in 1, the performances it provides are constantly very good; in contrast, in Random-Based Search strategy, the measured performance may significantly fluctuate on the basis of the category to which the trustee belongs to and this explains oscillations in ( 1).
The analysis of the Direct-Experience Search strategy ofers many interesting insights which are valid for both = 0.05 and = 0.15. Firstly, notice that if = 10, then the DirectExperience Search strategy achieves significantly better performances than the Random-Based strategy, which indicates that an even short learning phase yields tangible benefits. If increases, the trustor is able to see a larger number of agents before making its decision and, in particular, if is suficiently large, then the trustor might encounter the best performing agent ⋆ in the whole community. In this case, the Direct-Experience strategy would outperform the CategoryBased Search strategy: in fact, in the Category-Based strategy, the trustor chooses, uniformly at random, one of the agents in 1 which provides a performance worse than (or equal to) ⋆. In short, for large values of , the Direct-Experience strategy achieves performances which are comparable and, in some cases, even better than those we would obtain in the Category-Based strategy, as shown in Figure 1 and 2. However, the budget has the meaning of a cost, i.e., it is associated with the time the trustor has to wait before it chooses the trustee and, thus, in many practical scenarios, the trustor has to make its decisions as quick as possible.
It is also instructive to consider a further strategy, called Mixed-Based Search, which combines the Random-Based Search strategy with the Category-Based Search strategy.
In Mixed-Based Search, we assume the existence of a warm-up phase in which the trustor selects the trustee by means of the Random-Based Search strategy; unlike the Direct-Experience Search strategy, the trustor collects not only ( 1) but it also records the category of the trustee. In this way, the trustor is able to identify (after, hopefully, a small number of steps) the category with the highest true quality, i.e., 1. From that point onward, the trustor switches to a Category-Based strategy and it selects only agents from 1. From a practical standpoint, we suppose that a performance ( 1) ≥ 0.6 is classified as an indicator of good performance (in short, positive signal); as soon as the trustor has collected 2 positive signals, it makes a decision on the best performing category and it switches to the Category-Based Search strategy.
We are interested at estimating, through simulations, the length ℓ of the warm-up phase, i.e., the number of agents that the trustor has to contact before switching to the Category-Based search strategy. In Figure 3 and 4 we plot the pdf of ℓ.
Here, the variance has a minor impact and we notice that, the pdf achieves its largest value at ℓ ≃ 10, i.e., 10 iterations are generally suficient to identify the best performing category.
We conclude our study by investigating how the Fixed Variance Model and the Random Variance model influence the rate at which our algorithm estimates the true quality ( ) of a category.
To make exposition of experimental outcomes simple, we suppose that = 1 (which models a scenario in which agents in 1 showcase an exceptionally high ability in executing 1).
We considered the Fixed Variance Model with ∈ {0.05, 0.1, 0.15} and the Random Variance Model in which is uniformly distributed in [0.01, 0.3].
We investigated how ^( ) varied as function of the number of queried agents; obtained results are reported in Figure 5.
The main conclusions we can draw from our experiment are as follows:
1. Individual agent variability (modelled through the parameter ) greatly afects the rate at which ^( ) converges to ( ). Specifically, Figure 5 suggests that less than 5 iterations are enough to guarantee that |^( ) − ( )| < 10− 2 if = 0.05. In addition, as gets larger and larger, we highlight more and more fluctuations in ^( ): as an example, if = 0.15 (green line), we highlight the largest fluctuation in ^( ) and, at a visual inspection, at least = 30 queries are needed to achieve a significant reduction in |^( ) − ( )|. 2. An interesting case occurs in the Random Variance Model: in some iterations of the algorithm, agents with a small variability are selected (i.e., we would sample agents with ≃ 0.01) while in other cases agent with a larger variability are selected (here ≃ 0.3). Overall, agents with small variability fully balance agents with high variability and, thus, the algorithm converges to ( ) (red line) generally faster than the case = 0.1 (orange line) and = 0.15 (green line).
Although highly populated networks are a particularly useful environment for agents’
collaboration, the very nature of these networks may represent a drawback for trust formation, given
the lack of data for evaluating the huge number of possible partners. Many contributions in the
literature [