Establishing trust in multiagent environments: realizing the comprehensive trust management dream Thomas Tran Robin Cohen Electrical Engineering and Computer Science Computer Science University of Ottawa, Canada University of Waterloo, Canada ttran@eecs.uottawa.ca rcohen@uwaterloo.ca Eric Langlois Computer Science University of Waterloo, Canada edlango@uwaterloo.ca Abstract In this paper, we present a framework for enabling agents in multiagent systems to engender trust from the other agents in the environment. Opposite to the aims of most existing trust modelling research, our algorithms serve to guide the actions of trustees (instead of trustors). Our approach requires trustees to predict the types of the trustors, modifying their behaviour when predictions prove to be inaccurate. The primary aim of trustees is to acquire high levels of trust, of value for future interactions with trustors. Illustrated in the context of electronic marketplaces, we offer a validation of our approach in comparison with competing algorithms that drive the behaviour of trustees which are more focused on immediate rewards. Through simulations, we are able to demonstrate important gains achieved by our approach. As such, we offer an important first step in addressing a novel challenge for trust modelling, articulating how trust may be successfully engendered. 1 Introduction Sen’s vision paper at AAMAS 2013 (Sen [2013]) challenged researchers to stretch their investigation of trust modelling in multiagent systems, towards establishing a comprehensive solution for trust management. This vision promoted exploration of four central considerations: evaluating, establishing, engaging in and using trust, in multiagent contexts. Our research focuses on the second step above; establishing trust refers to the desire to design algorithms which enable an agent to engender the trust of its peers within the environment. This is a dramatic departure from most trust modelling research, which is focused on the trustor instead of the trustee (typically offering approaches for trust to be effectively modelled and recorded, for an agent’s decision making). Projected into the multiagent context of electronic marketplaces, we present a proposed set of actions for trustees which include modelling trustors and selecting appropriate price and quality of goods to be offered, in an effort to maximize the trust that is achieved. Our approach distinguishes expected types of trustors and aims to optimize satisfaction from each trustor type. Learning is used to adjust actions taken, in light of feedback received subsequent to attempted transactions. To measure the effectiveness of our framework we present a series c by the paper’s authors. Copying permitted only for private and academic purposes. Copyright � In: R. Cohen, R. Falcone and T. J. Norman (eds.): Proceedings of the 17th International Workshop on Trust in Agent Societies, Paris, France, 05-MAY-2014, published at http://ceur-ws.org 1 of simulations, running our algorithm and comparing to competitors which are less attuned to engendering trust and which lack some of the detailed modelling that is integrated into our proposed solution. We are able to demonstrate noticeable advantages of our approach. Though our current design is fairly preliminary, we outline possible extensions for the future and emphasize the value offered by the outcomes achieved to date. We also include a discussion of related work, driving home the exciting promise of finally endeavouring to address a problem that complements the vast array of existing trust modelling research. 2 Our Model Below we provide an overview of our proposed model for engendering trust. Our trustees begin with a character- ization of the possible trustors in the environment; this classification is used to drive the actions of the trustees to secure high trust from those trustors. For the remainder of the paper, we situate ourselves in the setting of e-marketplaces, in order to present our proposal in full. As a result, trustors (buyers) are characterized by the trustees (sellers) in terms of their sensitivity to price and to quality. From now on, we use the terms “trustor” and “buyer” interchangeably, and the terms “trustee” and “seller” interchangeably. Consider an e-marketplace populated with buying agents and selling agents. Let X and Y be a pair of buyer and seller that may have a business transaction with each other. Seller Y seeks to attain high trustworthiness in buyer X in the hope that it will be relied on by X for X’s future purchases, and therefore may demand higher prices for its products or services. We present below a proposed strategy that Y can follow to establish itself to be a trustworthy agent to X, and similarly to other buyers in the marketplace. But we begin first with a trust model that X may use to evaluate the trustworthiness of Y . 2.1 Buyer’s Trust Model We make use of the trust model proposed in Tran [2010]. Let TX (Y ) ∈ [−1, 1] be the trust value or trust score indicating the extent to which buyer X trusts (or distrusts) seller Y according to X’s experience in interacting with Y . Initially, X sets TX (Y ) = 0 (default value) for every Y in the environment. If X is satisfied after the transaction with Y , it will increase its trust value in Y by � TX (Y ) + α(1 − TX (Y )) if TX (Y ) ≥ 0, TX (Y ) ← (1) TX (Y ) + α(1 + TX (Y )) if TX (Y ) < 0, where 0 < α < 1 is a positive increment factor. Otherwise, if X is unsatisfied after the transaction with Y , it will decrease its trust value in Y by � TX (Y ) + β(1 − TX (Y )) if TX (Y ) ≥ 0, TX (Y ) ← (2) TX (Y ) + β(1 + TX (Y )) if TX (Y ) < 0, where −1 < β < 0 is a negative decrement factor1 . 2.2 Seller’s Strategy for Engendering Trust Buyer’s Utility Function In general, a buyer X is satisfied with its purchase of a product if the value of the product received, in terms of its price p and quality q, exceeds or equals to the value that X expected in that product. This can be implemented, as shown in the Validation section below, by allowing buyer X to construct a utility function UX (p, q) on the product’s price and quality. The construction of the utility function UX (p, q) is buyer-specific, i.e., depending on how each individual buyer considers the relative importance of the price and quality of the product. In addition, each buyer X may set a minimum expected utility value uX , and X is satisfied with an offer of (p, q) for a product if UX (p, q) ≥ uX . 1 For an agent to properly set α and β, further insight may be gained from Tran [2010], which provides proofs for avoiding infinite harm in e-marketplaces. 2 Seller’s Utility Function On the sellers’ side, a seller Y desires some profit when selling a product or service to a buyer X. To receive different levels of profit, seller Y may sell a product at various prices and qualities to different buyers, or to the same buyer at different times. The quality of a product, as we consider here, consists of not only the inherent quality built in the product, but also features and services that add to the value of the product (e.g., convenient order handling, on-time delivery, assistance with selecting the product, after-sale support, etc.). Thus, for each product, seller Y has a set of prices and a set of qualities from which Y can choose a pair of specific price and quality (p, q) to offer to a buyer. Following the utility approach, each seller Y can also construct a utility function UY (p, q) and choose an offer of (p, q) such that UY (p, q) is not less than a threshold uY determined by Y , to ensure some level of profit in the transaction. Note that the setting of uY is seller-specific: some sellers may set uY low initially, and therefore are willing to incur some loss of profit for some number of transactions, in order to build high trust in buyers and hence gain more profit in the future. Seller’s Strategy Seller Y ’s strategy for engendering trust is based on the assumption that buyer X will assign a high trust value for Y if Y ’s offer results in a high utility value for X. The greater the utility value X receives, the higher trust score X allocates for Y . Thus, for each interaction, Y ’s overall strategy is to choose an offer of (p, q) aimed at maximizing X’s utility function UX (p, q) and also meeting Y ’s threshold requirement, UY (p, q) ≥ uY . In more detail, Y categorizes buyers into three types: (a)price-sensitive buyers who are more interested in a low price than a high quality, (b) quality-sensitive buyers who are interested in a high quality more than in a low price, and (c) balanced buyers who consider price and quality equally important. So, Y will behave differently when dealing with different classes of buyers, as follows: • When selling a product to a buyer X, if Y predicts that X is a price-sensitive buyer, Y will offer the lowest price pmin in its price set and the highest possible quality qfair chosen such that UY (pmin , qfair ) = uY . • If Y predicts X to be a quality-sensitive buyer, Y will offer the highest quality it can from its quality set, qmax , and the lowest possible price pfair determined to ensure Y ’s threshold requirement UY (pfair , qmax ) = uY . • If Y believes that X is a balanced buyer, Y will offer a balanced pair of price and quality, (pbal. , qbal. ), where pbal. and qbal. are the lowest possible price and the highest possible quality that Y can choose from its price set and quality set respectively that meet the balance condition pbal. = ρqbal. with ρ being the balance factor2 , and that also satisfy UY (pbal. , qbal. ) = uY . Learning the Correct Buyer In our experimentation, for simplicity we make use of a communication protocol by which X will communicate its satisfaction with Y after the transaction. Thus, Y can predict the classification of X by initially guessing a category for X and making the corresponding offer. If X is not satisfied, it is assumed that the category is incorrect and another is tried until X is satisfied. If X periodically changes its category during its lifespan in the environment, then a more sophisticated learning strategy is required. For example, Y can repeatedly try all categories for a fixed number of transactions, and then settle down on the category with the highest satisfaction rate. Further, Y ’s learning process would be more challenging in the absence of buyers’ feedback (that is, without the communication protocol). In this case, Y would continue with its classification of X if its initial guess results in X choosing Y again in the next transaction – a signal of X’s satisfaction, or Y would switch to another classification when not being chosen for some period of time after the first transaction – a signal of X’s dissatisfaction (similar to a reinforcement learning approach). 3 Validation In this section, we describe the simulation that we constructed in order to validate our proposed approach for trustees to engender trust. 2 If Y uses the same number scale to represent its sets of prices and qualities, factor ρ can be set to 1 and (p bal. , qbal. ) can be in the proximity of (pmedian , qmedian ). 3 3.1 Simulation Framework Our model is tested in a simulated electronic marketplace environment (algorithm 1). The environment contains a fixed population of trustors (buyers) and trustees (sellers). Each trustor interacts numTransaction times with trustees over the course of the simulation. In each interaction, the trustee, Y , provides some service to the trustor, X (e.g. selling a product to X), at a price and quality specified by Y . X may or may not be satisfied with the transaction and communicates this satisfaction to Y . X builds a trust model of each trustee over the course of the simulation and in general may use this model to guide selection of trustees in future transactions. In our experiment, Y is always chosen randomly. It is our goal to observe the trustworthiness of the trustees and this selection policy ensures all trustees are equally represented in transactions. Algorithm 1 Simulation pseudocode for iterationIndex = 1 to numTransactions do for all X ∈ trustors do Y ← X.selectTrustee(trustees) offer ← Y.makeOffer(X) satisfied ← X.recieveOffer(Y, offer) Y.receiveFeedback(X, offer, satisfied) end for end for 3.2 Simulation Data The simulation tracks the satisfaction rate3 and the trust score for pairs of trustees and trustors. There are several types of trustees and trustors in the simulation so the data is averaged over all pairs of trustors and trustees having the same types. To reduce noise, the results are averaged over every period of n iterations4 . Not every trustor/trustee pair will interact in each period so the results are only aggregated for the pairs that did interact. The average satisfaction rate is reset for each averaging period while trust is retained over the course of the entire simulation. Given the trust model described in subsection 2.1, the satisfaction rate represents the rate of change of the trust value so a satisfaction rate of 100% results in optimal trust, while 0% satisfaction results in pessimal trust. 3.3 Experiment: Exploration The experiment explores the behaviour of a simple model of our trustor in comparison to other simple agents. It is used to get an idea of the quality of the presented trustor model and to identify areas of the simulation and agent models requiring improvement. Much of the agent behaviour is determined exactly by model parameters. Parameters were selected tentatively to provide a good distribution. The experiment was run with a population of 2000 trustees, 2000 trustors, over 10000 transactions per trustor. The trustee and trustor types were evenly distributed over all the types described below. The results are shown in Figure 1 and Figure 2. 3.3.1 Agents The trustees in our simulation provide some service at a specified price, p, and quality, q, in each interaction with a trustor. The profit of a trustee Y is given by a utility function UY (p, q). Similarly, a trustor X measures the success of a transaction with a utility function UX (p, q) on the price and quality offered by a trustee. Trustors The trustors used are all “Simple Cutoff Trustors”, meaning a trustor X has a minimum expected utility uX and X is satisfied with an offer of (p, q) so long as UX (p, q) ≥ uX . There are three sub-classes of trustors used in the simulation: 3 Satisfaction rate is the number of times times trustee Y satisfies trustor X divided by the number of transactions between X and Y 4 An iteration is an iteration of the outer loop of Algorithm 1 in which every trustor interacts with a trustee 4 Price Sensitive Trustor This trustor is more interested in a low price than a high quality. This is represented by the utility function ap.s. q U p.s. (p, q) = (3) (bp.s. + p)2 where ap.s. and bp.s. are parameters. In this experiment, ap.s. = 10 and bp.s. = 1. Balanced Trustor The balanced trustor is equally interested in low price and high quality. The corresponding utility function is abal. q U bal. (p, q) = bal. (4) (b + p) In this experiment, abal. = 1 and bbal. = 1. Quality Sensitive Trustor The quality sensitive trustor is interested in a high quality more so than a low price. aq.s. q 2 U q.s. (p, q) = (5) (bq.s. + p) In this experiment, aq.s. = 0.1 and bq.s. = 1. In all cases, the minimum expected utility is uX = 0.5. These trustors proved to be rather simple in the context of our simulated marketplace. Their behaviour is unchanging over repeated interactions with trustees so each trustor X may be modelled as a function sX : Z2 → {unsatisfied, satisfied}. � unsatisfied if UX (p, q) < uX sX (p, q) = satisfied if UX (p, q) ≥ uX Since trust is based on satisfaction, a trustee Y may maximize their trustworthiness relative to trustor X by conducting a search of price-quality space for some (p, q) pair such that sX (p, q) = satisfied. Furthermore, any trustee that eventually settles on a fixed price and quality to offer a given trustor (call this behaviour “convergent”) will eventually have either optimal or pessimal trust score relative to that trustor. Trustees Several simple categories of trustors were explored. Some trustee categories have subcategories that are able to offer better or worse deals to the trustors. Not all trustees base decisions on a utility function, but those that do use the function: UY (p, q) = ap − bq 2 In this experiment, a = 6 and b = 1. Fixed Offer (Profit Maximization) This trustee always offers a fixed price and quality to all trustors. These are specified as parameters to the agent. The profit maximization trustees are defined differently but reduce in practise to fixed offer behaviour. A profit maximization trustee has ranges [pmin , pmax ] prices and [qmin , qmax ] of qualities it may offer. It maximizes profit by always offering (pmax , qmin ). Parameters In the experiment, the profit maximization trustees play the role of fixed offer trustees. There are three subcategories with different price/quality ranges. Low Quality/Price Price: [5, 10], Quality: [0.1, 10] Medium Quality/Price Price: [5, 20], Quality: [1, 20] High Quality/Price Price: [20, 30], Quality: [20, 30] 5 Behaviour For a given class of trustors, a fixed offer trustee either always satisfies or never satisfies the trustors. This can be clearly seen on the satisfaction rate graphs (Figure 1). As a result, their trust score (Figure 2) either increases or decreases monotonically as the number of transactions increases. They attain either optimal or pessimal trust score for the number of transactions completed at any given moment. Randomized This class of trustee has a target utility uY and randomly picks a price p and a quality q satisfying UY (p, q) = uY . This is accomplished by selecting a random price in the interval [pmin , pmax ] then finding a quality that gives the desired utility. Parameters The price range is the same as the medium quality/price randomized trustee: [5, 20]. There are three subcategories with different target profits. High Profit, Low Quality/Price uY = 20 Medium Profit, Medium Quality/Price uY = 10 Low Profit, High Quality/Price uY = 0 Description These trustees are a more general class than constant. They satisfy the trustors some fixed percent of the time. This results in a trust score that is essentially monotone but with added noise. The trust scores need not be optimal or pessimal. Classifying Trustee This trustee embodies our approach. It attempts to classify the trustors as either price- sensitive, quality-sensitive, or balanced sensitivity and use that knowledge to maximize its trust score. Note that using classification as a strategy is helpful for maximizing trust but it is also helpful for maximizing profit (in the more realistic scenario where profit is related to how often the buyer is satisfied). In this experiment, only a simple classification is implemented. The trustee predicts a category cX ∈ {p.s., bal., q.s.} for trustor X. For each classification c, there is a fixed price and quality (pc , qc ) that the trustee offers. These are derived in a way similar to the randomized trustee: the price pc is specified as a parameter and the quality is selected such that UY (pc , qc ) = uY . This means that the offers a classifying trustee makes are a subset of the offers a randomized trustee with the same target utility may make. The actual classification of a trustor X is made by guessing a category for X and making a corresponding offer. If X is not satisfied it is assumed that the category is incorrect and another is tried until X is satisfied. Parameters uY = 10, the same as for the medium profit randomized trustee. Behaviour The behaviour of the classifying trustees on a class of buyers can be divided into two cases: Case 1: None of classifications result in offers that satisfy the trustor. In this case, satisfaction rate is a fixed 0% and trust score is pessimal. Case 2: A classification results in an offer that satisfies the buyer. In this case, the trustees have some baseline satisfaction rate due to the initial guess. The satisfaction rate grows to 100% as the trustees learn how to correctly classify the trustor. Trust value may initially decrease but eventually grows to approach 1. 3.3.2 Results Explanation of Plots The results of the experiment are shown in Figure 1 and Figure 2. The left column contains plots pertaining to satisfaction rate and the right column is trust score. Each row is a class of trustor and the lines show the average behaviour of each class of trustees with those trustors. The x-axis is the number of transactions so far for each buyer (trustor). Recall that the simulation (algorithm 1) has numTransaction iterations and on each iteration every trustor interacts with a single trustee. Plotted is either the average satisfaction rate (number of satisfactory transactions / number of transactions) or the trust score of pairs of trustors and trustees. Each data 6 Figure 1: Satisfaction Rate Plots Figure 2: Trust Plots 0.8 1 0.6 0.4 0.8 0.2 Satisfaction Rate 0.6 Trust Value 0 -0.2 0.4 -0.4 0.2 -0.6 -0.8 0 -1 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Number of Transactions Number of Transactions (a) Price Sensitive Buyer (a) Price Sensitive Buyer 1 1 0.8 0.6 0.8 0.4 Satisfaction Rate 0.2 0.6 Trust Value 0 0.4 -0.2 -0.4 0.2 -0.6 -0.8 0 -1 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Number of Transactions Number of Transactions (b) Balanced Buyer (b) Balanced Buyer 1 1 0.8 0.6 0.8 0.4 Satisfaction Rate 0.2 0.6 Trust Value 0 0.4 -0.2 -0.4 0.2 -0.6 -0.8 0 -1 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Number of Transactions Number of Transactions (c) Quality Sensitive Buyer (c) Quality Sensitive Buyer Randomized Trustee High P&Q Profit Maximizing Trustee High P&Q Profit Maximization Trustee Profit Maximizing Trustee Low P&Q Randomized Trustee Low P&Q Randomized Trustee Classifying Trustee 7 point is an average over 100 transactions and over all trustees/trustor pairs of the appropriate types that have interacted in the last 100 iterations5 . The randomized trustees are the red, black, and blue lines representing the high, medium and low price/quality subclasses. respectively, As expected, each has a fixed satisfaction rate and their trust score is correspondingly monotone (with added noise). Similarly, the profit maximizing trustees (which are effectively fixed offer) have 0% or 100% satisfaction rate for each group of trustors and pessimal or optimal trust scores. The profit maxi- mizing trustees are the cyan, green, and yellow lines which again represent the respective high, medium and low price/quality subclasses. The classifying trustee (magenta) is the only without a fixed satisfaction rate. For all trustors, the classifying trustee begins with some initial satisfaction rate that grows to 100% as the population of classifying trustees learns the trustor types. The trust scores (computed according to the equations in subsection 2.1) are closely related to satisfaction rate and on average increase or decrease exponentially at a rate given by the satisfaction. 4 Discussion and Conclusion In order to situate our research with respect to related work, it is important to begin with the paper that motivated us in the first place. Sen (Sen [2013]) contends that most research to date from the multiagent systems trust modelling community has focused on algorithms for a trustor to model the trustworthiness of trustees, in order to make effective decisions about which trustees to select. Some researchers have included elements of their proposals which integrate trustee modelling of trustors as well. For example, Tran (Tran [2003]) includes seller modelling of buyers as part of his reinforcement-learning based approach for modelling trust in e-marketplace environments. Over time, a seller learns about the buyers so that the bids that are offered are then selected in order to increase the chances that the seller will be successfully chosen. While this is a concrete first step, the learning that is done is fairly broad-brushed and is not drilled down to the level of a characterization of the buyer, leading to differing strategies for different types of buyers. Within the field of multiagent system trust modelling, there are threads of research which should also be useful starting points for this novel direction of developing models for trustees to engender trust. Agents that are focused on achieving high trust levels as part of their plans to be selected as partners of other agents in the community are in some sense directed more towards future than immediate gain. This concept is explored as part of an incentive mechanism for honesty in e-marketplace environments presented by (Zhang and Cohen [2007b,a]). The work of Smith and des Jardins (Smith and desJardins [2009]) offers further insights into how buying agents can behave in order gain payoffs from cooperative behaviour. Having agents begin to reason about attaining these payoffs is a possible starting point for a more detailed exploration of engendering trust. That work also explores the value of gaining trust currently at the expense of possible future trust; exploring the balance of these two elements should continue to be a thread for future work. Other relevant research is that of Noorian et al. (Noorian et al. [2012]) who propose that trusters model the willingness of trustees. As trust models begin to expand in sophistication, trustees will be challenged to also work harder on gaining trust in order to be successful in their interactions with other agents. Work on detecting and identifying collusion has also begun to emerge, which now presents a significant challenge to those agents focused on trying to emerge with benefits, while avoiding true honest behaviour (Kerr and Cohen [2012]); this work also suggests an impending upswing in interest on models to engender trust in multiagent systems. Perhaps one of the most cogent observations for the future comes from Josang (Jøsang [2011]) : he clarifies that we are increasingly immersed in massive online social networks where a person’s value is ultimately measured by their trustworthiness. It is environments like these that suggest an ongoing interest from users in ensuring that their trustworthiness remains well respected within their communities. There are various independent avenues for future work. The first would be to extend our current set of simulations in order to explore additional kinds of validations for the approach. Included here would be comparing our proposed approach with more generous competitors. For example, we are interested in exploring a) profit- maximizing sellers who are willing to still incur some loss of profit initially by offering greater quality goods; these sellers would still have a much lower tolerance for loss of profit from transactions than those who follow our trust-maximizing behaviour b) profit-maximizing sellers who may end up securing more trust by somewhat more clever initial behaviour: acting honestly at first on some small-priced transactions, only to later disappoint on larger items (the so-called value imbalance problem outlined in (Kerr and Cohen [2009]). 5 This final restriction is necessary because there are significantly more trustor/trustee pairs than the number of transactions in 100 iterations. As a result, most pairs have “stale” trust scores. We are interested in changes in trust score as number of transactions increase so it is helpful to exclude these stale data points 8 More elaborate variations work that interest us include the following. We could be mapping a function of both trust and profit for the agents following our trust-maximizing behaviour. We expect that our agents may initially lose profit in order to attain longer term gains. We would be cautious in interpreting profit gains alone by agents following competing algorithms as offering an improvement. We would also like to move on to allow the trustors to be modelling the trustworthiness of the trustees more fully, to be selecting preferred partners. At the moment our simulation aims to ensure that all kinds of agents are tested, by making some random choices of partners. The next steps would be to integrate a modelling of trusted and untrusted partners, with penalties for disappointment, as in the work of (Tran and Cohen [2004]). Once this kind of trust modelling was in place, we could also explore variations where some random selection occurred at first, to populate the simulation with appropriate values and then the reasoning about trustworthiness would be integrated. With these more intelligent trustor models in place, we could then experiment both in mapping trust and satisfaction and in gauging how profit has panned out, as well. In conclusion, we have offered a concrete first step towards addressing the challenge of articulating algorithms for trustees to engender trust from trustors, using strategies that incorporate reasoning about trust that is incurred, not just immediate profit. We have provided simulation results to confirm the value of the approach and to begin to quantify its advantages over competing algorithms. 5 Acknowledgements Financial support was received by NSERC. Thanks to Paul Kates for his aid in preparing this paper. References Audun Jøsang. Trust management in online communities. New Forms of Collaborative Innovation and Production on the Internet, page 75, 2011. Reid Kerr and Robin Cohen. Smart cheaters do prosper: Defeating trust and reputation systems. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 993–1000. International Foundation for Autonomous Agents and Multiagent Systems, 2009. Reid Kerr and Robin Cohen. Detecting and identifying coalitions. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 3, pages 1363–1364. International Foundation for Autonomous Agents and Multiagent Systems, 2012. Zeinab Noorian, Jie Zhang, Michael Fleming, and Stephen Marsh. Determining the optimal reporting strategy in com- petitive e-marketplaces. In Trust, Security and Privacy in Computing and Communications (TrustCom), 2012 IEEE 11th International Conference on, pages 350–357. IEEE, 2012. Sandip Sen. A comprehensive approach to trust management. In Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, pages 797–800. International Foundation for Autonomous Agents and Multiagent Systems, 2013. Michael J Smith and Marie desJardins. Learning to trust in the competence and commitment of agents. Autonomous Agents and Multi-Agent Systems, 18(1):36–82, 2009. Thomas Tran. Reputation-Oriented Reinforcement Learning Strategies for Economically-Motivated Agents in Electronic Market Environments. PhD thesis, University of Waterloo, 2003. Thomas Tran. Protecting buying agents in e-marketplaces by direct experience trust modelling. Knowledge and Infor- mation Systems (KAIS), 22(1):65–100, 1 2010. Thomas Tran and Robin Cohen. Improving user satisfaction in agent-based electronic marketplaces by reputation mod- elling and adjustable product quality. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 828–835. IEEE Computer Society, 2004. Jie Zhang and Robin Cohen. Design of a mechanism for promoting honesty in e-marketplaces. In PROCEEDINGS OF THE NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE, volume 22, page 1495. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2007a. Jie Zhang and Robin Cohen. Engendering trust in buying and selling agents by discouraging the reporting of unfair ratings. In AAAI Spring Symposium: Interaction Challenges for Intelligent Assistants, page 139, 2007b. 9