Persuasive negotiation involves negotiating using rhetorical arguments (such as threats, rewards, and appeals), which act as persuasive elements that aim to force or convince an opponent to accept a given proposal. In the case of rewards, these have a positive nature as they use the argument that something positive will happen to the opponent if he accepts to do the requirement sent by the proponent. A proponent agent can generate more than one reward depending on the information he has modeled of his opponent. The problem appears when the agent has to choose a reward, to send to his opponent, among a set of rewards. One measure that could help him in his choice is the strength each reward has. Thus, the goal of this work is to analyze the rewards components and to propose a model for calculating their strength. We propose two ways for calculating the strength of rewards depending on the kind of negotiation the agent is participating. The first proposal is to be used when the agent negotiates only with one opponent, and the second when the agent negotiates with more than one opponent.
Persuasive negotiation involves negotiating using
rhetorical arguments, which act as persuasive elements
that aim to force or convince an opponent to accept a
given proposal [
Although some authors argue that threats are the
strongest rhetorical arguments ([
Let’s see the following persuasive negotiation scenario where boss is an agent proponent, employee an agent opponent and the goal of boss is that employee works every weekend1. Taking into account the knowledge base of agent boss, the following rewards can be generated: • boss: if you work every weekend, you will receive an interim payment. • boss: if you work every weekend, you will have more holidays.
The question is which of these rewards will boss
choose to persuade employee to work every weekend?
One way of knowing this is by calculating the strength
of the generated rewards. According to Ramchurn et
al.[
Rewards are constructed using both proponent and
opponent’s goals (for example, earning more money).
Some researches on this topic take into account the
importance of the opponent’s goal for the agent
opponent and the certainty level of the beliefs used
for the argument generation [
2. Agent boss has already done rewards before and he has rarely fulfilled it, and obviously employee knows about it. In this case, the strength of a reward is also influenced by the execution credibility that the proponent has from the point of view of his opponent. Thus, even when the goal of an opponent is very important and/or achievable, a low level of credibility could diminish the value of the strength of a reward.
In the first case, notice that besides importance, there exists another criterion to evaluate the quality of the goal of an opponent, because it does not matter how important a goal is if it is not possible to be achieved. And in the second case, the credibility execution level of the proponent (from the point of view of the opponent, i.e. the execution level the proponent believes the opponent has about him) should also be considered.
With respect to the first criterion, to determine
how achievable a goal is, we will use the belief-based
goal processing model proposed by Castelfranchi and
Paglieri [
Part of our proposal for calculating the strength of a reward considers the state of the goals of the opponent. Depending on this state, a goal can be considered more or less rewardable, a goal is considered more rewardable when it is closer of the executable state and less rewardable when its state is active. Thus, the aim of this article is to propose a model for calculating the strength of rewards by taking into account new criteria, which will lead to a more accurate calculation.
The paper is organized as follows: Section 2 presents the goal processing model on which our strength calculation model is primarily based. In Section 3 a negotiating agent architecture that considers the necessary mental states, structures and functions that support our proposal is defined. A formal definition of reward and the mechanism for its generation are presented in Section 4. Section 5 shows an analysis of the elements of a reward and our proposed strength calculation model. In Section 6, the main related works are compared with our proposal. Finally, Section 7 is devoted to some conclusions and future works. 2
In this section, the four stages of the goal processing model of Castelfranchi and Paglieri are presented2. The aim of this section is not to present in detail the beliefs used in each stage. We focus on the goals states and make clear when a goal is considered active, pursuable, chosen, and executable, because these states will be used in the strength calculation model that is proposed in this work. Following a brief description of each stage:
1. Activation stage: In this stage, goals are activated by means of motivating beliefs. For example, if the agent has the belief that today is Thursday, it activates the goal of going to the French class, or the motivating belief that today is sunny activates the goal of playing football. When a motivating belief is satisfied, the supported goal becomes active. An active goal can also be seen as a desire.
2. Evaluation stage: In this stage, goals are evaluated using assessment beliefs. When there are no assessment beliefs for a certain goal, it becomes pursuable. Three types of assessment beliefs were defined: (i) those that control that there is no impossibility for a goal be pursued; (ii) those that control goals that are realized in the world autonomously and without the direct intervention of the agent; and (iii) those that control goals that have already realized, and that will remain as such.
3. Deliberation stage: The aim of this stage is to act as a filter on the basis of incompatibilities and preferences among pursuable goals. Goals that pass this stage are called chosen goals. These beliefs are concerned with the di↵erent forms of incompatibility among goals that lead an agent to choose among them. For dealing with incompatibilities, an agent uses preference beliefs.
4. Checking stage: The aim of this stage is to
evaluate whether the agent knows how to achieve a
goal and if it is capable of performing the required
actions to achieve a chosen goal, in other words if the
agent has a plan and he is capable of executing it.
Goals that pass this stage are called executable goals
and have the same characteristics of intentions.
2A more detailed version of this model is presented in [
In this section, we define the basic and compound structures, and functions of the agent, which are necessary in order to be able to generate rewards and calculate their strengths3. This architecture is based on the Castelfranchi and Paglieri’s model.
Before defining the structures of an agent, let L be a first order logical language which will be used to represent the goals and beliefs of the agent. ^ ,_ and ¬ denote the logical connectives conjunction, disjunction and negation, and ` stands for the classical inference. Definition 3.1. (Basic structures) An agent has five basic structures: - K is the knowledge base of the agent; - Op stores the opponents of the agent; - G = Ga [ Gp [ Gc [ Ge is the set of goals of the agent, such that Ga the set of active goals, Gp the set of pursuable goals, Gc the set of chosen goals and Ge the set of executable goals. It holds that Gx \ Gy = ; , for x, y 2 { a, p, c, e} and x 6= y;
- G O = G Os [ G Oa [ G Op [ G Oc is the set of goals of the opponent, such that G Os is the set of sleeping goals of the opponent4, G Oa is the set of active goals of the opponent, G Op the set of pursuable ones, and G Oc the set of chosen ones. It holds that Gx \ Gy = ; for x, y 2 { s, a, p, c} and x 6= y. Finally, let State(goi) = z be a function that returns the state of a given goal; for z 2 { 0, 1, 2, 3} where 0 means that the goal is sleeping, 1 active, 2 pursuable and 3 that it is chosen; - Rws stores the rewards constructed by the agent. The logical definition of a reward is given in Section 4. Definition 3.2. (Compound structures) These store characteristics of the basic structures.
- Opdet = {(opi, )} such that opi 2 Op and is the
execution credibility level of rewards the proponent
has from the point of view of opponent opi. Hereafter,
we denote that 2 [
- G Odet = {(goi, , op j )} such that goi 2 G O is a
goal of opponent opj 2 Op whose importance is given
3In this work, we assume that the agent has in advance
the necessary information (importance and state of the
goals, and the value of the credibility level of execution of
rewards) for generating and calculating the strength of rewards.
Some interesting works about opponent modelling related to
argumentation are [
4Sleeping is one of the states a goal may take that is proposed
in [
- EvalRecomp(goj ) is a function that takes as input a rewardable goal of an opponent and returns a reward action. This function lets the proponent choose an adequate reward action, that is within the possibilities of the proponent, thereby, he can fulfill the o↵ered reward. 4
A reward is constructed based on two goals: 1. An outsourced goal of the proponent: This kind of goal needs the opponent involvement for being achieved. For example, the goal of agent boss is that employee works every weekend, for this goal be achieved it is necessary that employee executes the required action. Considering the goal processing stages defined in Section 2 the state of this goal is executable.
Definition 4.1. (Outsourced goal) An outsourced goal gi is an expression of the form gi(opk, gi0), such that, opk 2 Op and gi0 is an action that opk has to execute. Let f irst(gi) = opk and second(gi) = gi0 be the functions that return each component of gi.
2. The goal of an opponent: goi 2 G O is a goal that proponent agent knows its opponent wants to achieve. For example, boss knows that employee wants to visit his parents. Besides knowing the goal of his opponent, the proponent has to know the state of that goal and its importance.
The construction of a reward begins when (i) an outsourced goal gi passes all the goal processing stages and becomes executable and, (ii) after a failed first attempt of proponent agent to make his opponent to do the requested action gi0. The process of construction of a reward is the following: 1. Function Op goals returns the set of rewardable goals the proponent knows about the opponent opj . Let Sgo = Op goals(opj ) be the returned set. 2. If Sgo 6= ; (a) For each goj 2 Sgo: i. Obtain rop = EvalRecomp(goj ) to know a reward option for goal goj . ii. Generate the two rules necessary for constructing a reward. The first is an expression of the form rrw1 = gi0 ! rop and the second is rrw2 = rop ! goj.
iii. Construct a reward and save it in Rws.
Rewards in Rws are called candidates. After the strength calculation, the strongest one is sent to his opponent to try to persuade him.
Following, we present the formal definition of a
reward, this is based on the definition given in
[
Let’s call Rrw and gi0 the support of the reward and goj its conclusion.
Example 4.1. Let us define the mental state of boss: Ge = g1, where: g1 = make(employee,‘work(weekend)’) is an outsourced goal, and therefore g10 = work(weekend) GOa = {go1}, where go1 = visit(parents), GOc = {go2}, where go2 = f ix(car), GOdet = {(go1, 0.8, employee), (go2, 0.6, employee)}, Op = {employee}, Opdet = {(employee, 1)}, Rws = {}
Let us suppose that agent employee rejected to do action g10 = work(weekend). Therefore, boss begins the process of construction of candidate rewards: 1. Sgo = Op goals(employee), so Sgo = {go1, go2} 2. Sgo 6= ; , then (a) EvalRecomp(go1) = have(holidays)
Generate Rrw1 = {g10 ! have(holidays), have(holidays) ! go1}
Construct rw1 = hRrw1, g10, go1i (b) EvalRecomp(go2) = give(inter paym)
Generate Rrw2 = {g20 ! give(inter paym), give(inter paym) ! go2}
Construct rw2 = hRrw2, g20, go2i
Finally, Rws = {rw1, rw2} 5
The strength of a reward is mainly based on the “value” that the rewardable goal has for the opponent. On the other hand, the credibility the proponent has in the face of his opponent(s) regarding his ability to fulfill his rewards is an aspect that also influences the strength calculation.
The strength calculation of a reward depends on:
1. The goal of the opponent goi (or
rewardable goal): two aspects are considered:
(a) Its importance: Like in some related works ([
(b) Its state: Let’s recall that we use 0 to denote that a goal is sleeping, 1 to denote that it is active, 2 to denote that it is pursuable, and 3 to denote that it is chosen.
2. Execution credibility level: It is also important that the proponent agent be able to execute its rewards, from the point of view of his opponent. This value (represented in the proponent) reflects what the proponent believes the opponent thinks about the execution level of credibility of the proponent.
Considering these aspects, the formalization of our proposal is defined as follows.
Definition 5.1. (Basic strength of a reward) The basic strength of a reward depends on the importance and the state of the rewardable goal. Let rw = hRrw, gi0, goji be a reward, its the basic strength is obtained applying:
State(goj) + Importance(goj) 2 STbasic(rw) = num states 1 (1)
Where num states = 4 is the total number of states that an opponent goal can have and function State return the state of the opponent’s goal (0=sleeping, 1=active, 2=pursuable, and 3=chosen).
A direct consequence of the above definition is that
the value of the basic strength of a reward is a real
value between 0 and 1. Formally:
Property 5.1. Let rw = hRrw, gi0, goji be a reward.
STbasic(rw) 2 [
Proof Since the value of the basic strength is based on the values of the importance and the normalization of the states value of goj, and this values are of between 0 and 1, then the value of the basic strength is also limited by 0 and 1.
When the proponent agent constructs a set of rewards only for one opponent, the value of the basic strength is enough to choose the reward that will be sent. Nevertheless, a more exact value can be obtained if the execution credibility level is also considered. This aspect is even more important when the proponent agent generates rewards for more than one opponent as it will let him know which opponent may be convinced more quickly when faced with one of his rewards.
Definition 5.2. (Combined strength of a reward) The combined strength of a reward depends on the basic strength of the reward and the execution credibility level of the proponent. Let rw = hrrwk, gi0, goj i be a reward and opn 2 Op the opponent whose rewardable goal is goj . The combined strength of rw is obtained applying: STcomb(rw) = STbasic(rw) ⇥ Level Execrw(opn) (2) Property 5.2. The maximum value of the combined strength of a reward is at most the value of its basic strength: STcomb(rw) 2 [0, STbasic(rw)].
Example 5.1. Let us continue with example 4.1: State(go1) = 1, Importance(go1) = 0.8 State(go2) = 3, Importance(go2) = 0.6
Level Execrw(employee) = 1 is the execution credibility level of agent boss from the point of view of agent employee.
Applying equation 1, the basic strengths of rewards in Rws are STbasic(rw1) = 0.57 and STbasic(rw2) = 0.8. Since agent boss generated rewards only for one opponent, he can choose the strongest one without calculating the combined strengths, even more when in this case the values of the combined strengths are the same of the basic strengths. Therefore, boss would sends rw2 because it is the strongest reward. Example 5.2. Let us suppose that boss has another opponent: employee2. Let us also suppose that boss has generated two rewards for employee2 with the following basic strengths: STbasic(rw3) = 0.85, STbasic(rw4) = 0.55, with Level Execrw(employee2) = 0.7.
Taking into consideration only the basic strengths, the strongest reward is rw3 = 0.85. This means that boss could send this reward to employee2 as it seems that it would be more e↵ective than sending a reward to employee. However, the execution credibility level of boss (from the point of view of employee2) is lower than the execution credibility level from the point of view of employee. Thus, the combined strengths (equation 2) have the same values of the basic strengths for employee, but di↵erent values for employee2:
STcomb(rw1) = 0.57 ⇥ 1 = 0.57 STcomb(rw2) = 0.8 ⇥ 1 = 0.8 STcomb(rw3) = 0.85 ⇥ 0.7 = 0.6 STcomb(rw4) = 0.55 ⇥ 0.7 = 0.41
Therefore, the best option for boss is to sent reward rw2 to his opponent employee. 6
Servapali et al. [
In [
This work makes a deeper analysis of the components of a reward and considers new criteria for the calculation of their strength. Using these criteria, two forms for calculating the strength of a reward were proposed: the basic strength and the combined strength calculation. These two di↵erent ways of calculus is one of the advantages of our proposal as, depending on the need of the situation, the proponent can use either the basic or the combined equation.
We also presented a process for rewards construction and an agent architecture based on the goal processing model of Castelfranchi and Paglieri. We believe that our proposed process gives more flexibility for the generation of rewards as the rewards rules are generated dynamically from the set goals of the opponent and by applying the EvalRecomp function, which evaluates the rewardable goal to return an adequate reward.
As future works, we want to analyse the calculation of strength of rewards from the point of view of the opponent. Besides the importance and the state of the rewardable goal, the utility of the reward for the opponent may be considered.
We also want to work on experience-based calculation; this would be done after the proponent receives the answer of the opponent, which can be positive or negative. When it is negative, a recalculation of the initial strength should be done.