When exchanging arguments with other people, it is interesting to know who of the others has the most similar opinion to oneself. In this paper, we suggest using weighted argumentation graphs that can model the relative importance of arguments and certainty of statements. We present a pseudometric to calculate the distance between two weighted argumentation graphs, which is useful for applications like recommender systems, consensus building, and finding representatives. We propose a list of desiderata which should be fulfilled by a metric for those applications and prove that our pseudometric fulfills these desiderata.
In real-world discussions, people exchange arguments on a dedicated issue, such as
improving the course of study [
Each individual participant of an argumentation has a personal view on the arguments and their relative importance: Users can decide for themselves which arguments they consider more convincing, thus which arguments they agree to and how much they agree with a statement. They may consider some positions more important than others.
Based on those individual views, there are useful applications for measuring the similarity or distance between two users: Clustering can be used to find representatives for a group of people with similar argumentation behavior or for finding a consensus. Another application is opinion polling, where one wants to find out why two persons or organizations come to different conclusions. What is more, collaborative filtering, which needs some definition of distance between users, can be used for pre-filtering arguments in applications like Kialo2. Brenneis et al. / How Much Do I Argue Like You? Towards a Metric on Weighted Argumentation Graphs 3
In this paper, we propose solutions to the two main challenges to achieving the goal of comparing the argumentations of two users: We define weighted argumentation graphs which are a suitable representation of argumentation covering the mentioned aspects, including importance of arguments and agreement with statements. Secondly, we suggest a pseudometric for calculating the distance between two weighted argumentation graphs, which considers the specific structure of argumentation graphs (e.g., opinions deeper in a graph are less important). We contribute a list of useful desiderata for a metric which compares argumentations, and prove that our pseudometric fulfills those properties.
In the following chapter, we present our definition of weighted argumentation graphs. The third chapter introduces our pseudometric and desiderta for a useful metric. Finally, we discuss some limitations of our pseudometric and take a look at related work.
To be able to determine the similarity of real-world argumentations, there has to be a
suitable representation of them. This representation should be able to capture all aspects
mentioned in the introduction, and it should be as simple as possible. Therefore, the
following definition is based on the IBIS model [
For the application purposes described in the introduction, the model should be able
to represent the known opinions and arguments of a person as close as possible. Thus, we
use statements, not arguments, as atomic elements in our definition, which then can be
composed to arguments which can support or attack another statement. Note, though, that
this definition can be translated to classical abstract argumentation frameworks based on
Dung’s definition [
Let S be the (finite) set of all statements, with the special statement I 2 S. The set A S n fIg S is a set of arguments. For an argument a = (s1; s2) 2 A, s1 is called premise, s2 conclusion of a. Let s 2 S, then a!s := f(t; u) 2 A j u = sg is the set of arguments with conclusion s.
Note that I is excluded to be a premise since it is the issue in the IBIS model. We
refer to I as “personal well-being”, which allows us to interpret an edge like (b; I) in
Figure 1 as “My personal well-being improves, because more wind power plants will be
built.” The premises of arguments with conclusion I are called positions. Positions are
actionable items like “A wall between Austria and Germany should be built,” and play
an important role in real-world argumentation, e.g. decision-making problems [
Note that this model does not include different relations for attack and support, as known from bipolar argumentation frameworks. Whether an argument is supportive are not, is up to a person’s interpretation of the natural language representation of the arguments. The purpose of the model is solely to capture the hierarchy of statement, which we later need for our metric; bipolarity would add unnecessary complexity in this paper.
Every person can have a personal view with personal attitudes on a common argumentation graph G, as depicted in G0 in Figure 1. To get an intuition for our next definia1 a11
G a a2
I b b1
G0 on G tion, let us have a look at what we can deduce about Alice’s attitudes from her graph G0: Alice strongly accepts position b (rating :4) and is slightly against a (rating :2). She accepts the statement a2 more than statement a1 (rating :5 > rating :3). The counterargument (a2; a) “No wall should be built, because a wall is expensive.” is far more important for her than the argument (a1; a) (relative importance :8 > :2).
Furthermore, it makes sense to sort the positions: She considers building more wind power plants (b) more important than building a wall (a, relative importance 32 > 13 ). So when comparing her attitudes with someone else’s attitudes, she would consider a contrary opinion on b more severe than a different opinion on a. For ordinary statements, which are not positions, having an importance does not make sense: One cannot say that “The wall stops illegal immigration of cows” is twice as important as “Cows can come via Switzerland” (important regarding what?); one can only say that the arguments regarding building a wall which are built by those statements are of differing importance.
We will use real numbers to represent those weights and ratings.
Definition 2 (Weighted Argumentation Graph). Let G = (S; A) be an argumentation graph. A weighted argumentation graph G0 on G is a quadruple (S; A; r; w) with functions r and w. r : S ! [ 0:5; 0:5] assigns an agreement score (rating) to every statement, where negative values mean disagreement, 0 no opinion/don’t care, and positive values agreement. w : A ! [0; 1] assigns an importance weight to each argument. The value indicates the importance of that argument relative to other arguments with the same conclusion. The value 0 means that the argument is not used (i.e. has no relevance), and 1 means that the argument is the only relevant argument for the conclusion. The following conditions must hold:
Formula 1 means that the sum of weights of arguments with the same conclusion is 1 if there is an argument with positive weight (cf. (a1; a) and (a2; a) in Figure 1); the sum is 0 iff no argument for a common conclusion has a weight (cf. w(b1; b) = 0 in Figure 1). This assures that w represents relative, not absolute importance. To simplify notation, we write w( ; ) instead of w(( ; )). If the underlying argumentation graph G happens to be a directed tree, we call G0 a weighted argumentation tree.
w and r can be represented as matrix or vector, respectively, where undefined values are set to the default value 0. For the example in Figure 1, one gets:
Brenneis et al. / How Much Do I Argue Like You? Towards a Metric on Weighted Argumentation Graphs 5 0 0 0 0 0 0 0 01 B0:33 0 0 0 0 0 0C
BB0:67 0 0 0 0 0 0CC w = BB 0 0:2 0 0 0 0 0CC ;
BB 0 0:8 0 0 0 0 0CC B@ 0 0 0 0 0 0 0CA 0 0 0 0 0 0 0 0 0 1 B 0:2C
BB 0:4 CC r = BB 0:3 CC
BB 0:5 CC B@ 0 CA 0 (3)
The entry in row i column j of w is the weight of the argument with premise i and conclusion j. The first column and first row must refer to I as premise or conclusion, respectively. Because of Formula 1, the column sum is always 0 or 1.
If we draw or talk about a weighted argumentation graph, “non-existing” edges a are edges with w(a) = 0 (argument with no importance), and “non-existing” nodes s are nodes with r(s) = 0 (neutral statement). An example is G0 shown in Figure 1.
The importance of a position p is represented as weight of the “argument” (p; I)
leading to the “personal well-being” I. An application could obtain weights and ratings
from a user, for example, by asking them to mark statements which are considered more
important, or sorting arguments by relevance, which we do in our deliberate system [
We now propose a pseudometric for calculating a distance between two weighted argumentation graphs, and prove several properties we expect of a function which compares two argumentations. The goal of the metric is to indicated how close the opinions and used arguments of two persons are, considering graph structure and individual assessments of importance; we do not want to compare argumentations on abstract levels like consistency, number of arguments used, or if other person’s arguments are countered. 3.1. The Pseudometric We define a distance measure of two weighted argumentation graphs G1 = (S; A; r1; w1), G2 = (S; A; r2; w2) on G = (S; A) as: dG(G1; G2) = (1
¥ a) å aikwi1[:;1] r1 i=1 wi2[:;1] r2k1 (4) where a 2 (0; 1) determines the influence of opinions deeper in the graph: A lower a emphasizes opinions on statements r(s), a higher value the similarity of the argumentation underneath a statement s. w[:;1] denotes the first column of the weight matrix w, ai is the i-th power of the scalar a, wi the i-th power of the square matrix w, and the Hadamard (entrywise) product. The i-th summand calculates the contribution of the paths with length i ending at I. We drop the index G if the underlying argumentation graph is clear from the context.
The intuition behind this distance measure becomes clearer when rephrasing it for the special case of argumentation trees (which have no cycles or re-used statements, thus unique paths from each statement to I). In case of argumentation trees T1 and T2 on T , the distance dG is equivalent to: dT (T1; T2) = (1 a) å adepth(s) r1(s) Õ w1(a) r2(s) Õ w2(a) s2S a2rI!s a2rI!s (5) rs1!s2 is the sequence of all arguments (edges) on the path from s1 to s2 2 S (where s2 is deeper in the tree). depth(s) = jrI!sj is the length of the path from I to s, i.e. the number of arguments; depth(I) = 0.
The terms in the absolute value measure the similarity of the opinions of a statement s as difference of their ratings, scaled with the product of the “importances” of the arguments leading to s. Hence, statements which are deeper in the argumentation tree get a smaller weight, and the overall relevance of an argumentation branch is limited to its importance.
To see how the calculation works and that the results match intuition, let us calculate the distance between the graphs G0 (Figure 1), T2, and T3 (Figure 2) for a = 0:5. We can expect that T2 is closer to T3 than to G0, because the opinions on the statements a and b match and only the weights are different. The results confirm the expectation: d(T2; G0) =(1 a) a1 0:5 0:6 ( 0:2) + a1 0:5 0:4 0:4
However, dG is not a metric, because dG(G1; G2) can be 0 even if G1 is not equal to
G2: Consider G1 where all statements are agreed to and every argument weight is 0, and
G2 where all statements have a rating of 0. Because weights and ratings are multiplied,
the distance is 0 though the weighted argumentation graphs are different.
3Remember that r1(root(T )) = r2(root(T )) because root(T ) = I, and r(I) := 0.
3.2. Desiderata for a Metric for Weighted Argumentation Trees
Our pseudometric is only one of many possible metrics for the applications described.
We present a list of intuitive desiderata which, as we think, should be fulfilled by any
metric comparing two person’s argumentations, and thus should be considered when
constructing alternative metric proposals. It is, however, hard to capture intuitive
properties in graphs which can contain circular references and re-used statements. Therefore,
we focus on argumentation trees in this section. Field experiments like [
After each desideratum, we prove that our pseudometric (Formula 5) fulfills it for weighted argumentation trees. We consider those properties important in many realworld application domains of a metric, albeit not everywhere, as pointed out in Section 5.
For each desideratum, we indicate why we think it is intuitive. Most desiderata are followed by a visual example making the choice of variable names clearer. Note that each tree in the examples is considered to be part of a bigger weighted argumentation tree, i.e. not all existing nodes and edges are drawn, and irrelevant statement ratings and argument weights are left out. d(Tk2 ; Tk3 ).
Desideratum 1 (Proportionally bigger overlap is better). Consider trees T1; T2; T3, where T2 is like T1, but uses one additional argument for a statement s, and T3 is like T2, but uses one additional argument for s. Although T2 and T1 differ in only one argument, and T3 and T2 differ in only one argument, we expect d(T1; T2) > d(T2; T3) because T2 and T3 have a greater overlap regarding the used arguments.
More formally: For every statement s in a tree T which only has leaves s1; : : : ; sn (n > 2) as premises for s with r(s1) = = r(sn) 6= 0 and w(s1; s) = = w(sn; s) and 8a 2 rI!s : w(a) 6= 0, consider the trees Tk, n k > 0, which only contain s1; : : : ; sk. Then, given k1 < k2 < k3, we want to have kk12 < kk23 =) d(Tk1 ; Tk2 ) > d(Tk2 ; Tk3 ), i.e. if the relative overlap of the number of arguments used is greater, the distance is smaller. Likewise, we demand kk12 > kk32 =) d(Tk1 ; Tk2 ) < d(Tk2 ; Tk3 ), and kk12 = kk23 =) d(Tk1 ; Tk2 ) =
We require 8a 2 rI!s : w(a) 6= 0 (i.e. no argument (edge) with weight 0 along the path from I to s), because if a user gives an argument a weight of 0, they say the premise is not related to the conclusion, thus not related to the topic of the discussion. This means that the user actually does not care about the opinions underneath that argument, which may be treated as if no opinion has been given.
In the following example, we have k1 = 1, k2 = 2, and k3 = 3, thus 21 < 23 =) dT (T1; T2) > dT (T2; T3):
T1
s 1 k1 r s1
T2
s k12 k12 r s1 r s2
T3
s k13 k13 k13 r s1 r s2 r s3 Proof. Only the argument weights for (si; s) (namely k1 , k1 , k1 , respectively, as used 1 2 3 in (8) = (9)) are different and contribute to the sum, all other summands are zero. The common weight products and common values for r are summarized as wr for readability and are factored out. For (10) > (11), remember that kk1 < kk2 . 2 3 (8) (9) (10) (11) dT (Tk1 ; Tk2 ) = wr(1 adepth(s0)jwk1 (s0; s) wk2 (s0; s)j
The other cases are proven by replacing “>” with “<” or “=”, respectively. Desideratum 2 (Contrary opinion is worse than no opinion). Consider trees T1, T2, T3, where all trees are identical, but T1 has no opinion on a statement s, T2 a positive opinion on s and T3 a negative opinion on s. As we definitely know that T2 and T3 disagree on s, we want to have d(T2; T3) > d(T1; T2).
Formally: For any statement s in a tree T with 8a 2 rI!s : w(a) 6= 0, let T + be like T but with r+(s) = q > 0, T like T with r (s) = p < 0, and T 0 like T with r0(s) = 0. Then d(T +; T ) > d(T +; T 0).
T0 0 s
T+ q s
T p s Proof. The only positive summand is the summand for s (which has rating 0, q or p for T 0, T +, and T , respectively). The argument weights and the a term are common to all summands, can be factored out, and are summarized as wa .
dT (T +; T ) = wa jq pj = wa (q + p) > wa jq 0j = dT (T +; T 0) (12) Desideratum 3 (Deviation in deeper parts has less influence than deviation in higher parts). Consider the trees T1; T2; T3, where T1 has an argument (sA; s) with no children and 8a 2 rI!s : w(a) 6= 0, T2 is constructed from T1 by adding a new statement sB and argument (sB; s) with w2(sB; s) = w2(sA; s) = w1(sA;s) , and T3 is constructed 2 from T1 by adding a new statement sA1 and argument (sA1; sA) with w3(sA1; sA) = 1. If r(sA) = r(sB) = r(sA1) 6= 0, then we want to have d(T1; T2) > d(T1; T3), because arguments deeper in the tree should have a smaller influence since we consider them less important for the overall opinion.
We require r(sA) = r(sB) = r(sA1) 6= 0 because adding a statement with “don’t care” opinion should not actually change the distance. We want equality because this desideratum should cover only differences in the depth of the statements, not their rating.
T1
s w r sA
T2
s w2 w2 r sA r sB
T3
s w r sA
Brenneis et al. / How Much Do I Argue Like You? Towards a Metric on Weighted Argumentation Graphs 9 Proof. The only differences are the summands including s, sA and sA1. Let r := r(sA) = r(sB) = r(sA1) 6= 0 and w := w1(sA; s). As before, we summarize common values for weights and a as wa .
dT (T1; T2) = wa rw > wa ajrwj = wa (a jr 0 w r 1 wj) = dT (T1; T3) (13) (14) Desideratum 4 (Influence of deeper parts depends on weights in higher parts). Consider trees T1; T2; T3, where all trees are identical, have statements sA and sB with conclusion s, and w(sA; s) > w(sB; s), but T1 has an additional argument for sA and T3 has an additional argument for sB. Although the difference in both cases is only one argument, we expect d(T1; T2) > d(T3; T2) because (sA; s) has a larger weight.
Formally: Let T2 be a weighted argumentation tree with arguments (sA; s); (sB; s) and w(sA; s) > w(sB; s), no premises for sA and sB and 8a 2 rI!s : w(a) 6= 0. T1 is constructed from T2 by adding (sA1; sA) and T3 from T2 by adding (sB1; sB), each with a weight of 1 and r1(sA1) = r3(sB1) 6= 0. Then dT (T1; T2) > dT (T3; T2).
T1 w(sA;s) sA 1 r sA1 s w(sB;s) sB
T2 w(sA;s) sA s w(sB;s) sB
T3 w(sA;s) sA s
1 r sB1 w(sB;s) sB Proof. Let r := r1(sA1) = r3(sB1) 6= 0. Only the summand which includes sA1 or sB1, respectively, contributes a value greater than 0.
dT (T1; T2) = wa jw(sA; s) r 0j > wa j0 w(sB; s) rj = dT (T2; T3) (15) Desideratum 5 (Weights of arguments have influence even if they are the only difference). Consider trees T1; T2, where all trees are identical and have the arguments (sA; s) and (sB; s), but the weights are different: w1(sA; s) 6= w2(sA; s) and w1(sB; s) 6= w2(sB; s). We want to have d(T1; T2) > 0 if there exists a statement s0 below sA (or sA itself) with r1(s0) = r2(s0) 6= 0 and 8a 2 rI!s0 : w(a) 6= 0.
We demand r1(s0) = r2(s0) 6= 0 because it makes sense if weights leading only to statements which are rated as “don’t care” are ignored.
In this example, we have sA = s0:
T1 w1(sA;s) r sA s w1(sB;s) sB Proof. It is enough to show that there is at least one summand greater than 0. (16) (17) (18) (19) (20) (21) (22)
Desideratum 6 (Symmetry regarding negation of opinion). Let T1, T2 be any weighted argumentation trees, and T3, T4, respectively, the same trees, but the opinion for each statement is negated, i.e. r3(s) = r1(s) and r4(s) = r2(s) for all s 2 S. We expect that a metric is symmetric regarding negation, i.e. d(T1; T2) = d(T3; T4).
Proof. This holds because jr1(s) r2(s)j = j( r3(s)) ( r4(s))j = jr3(s) r4(s)j. Desideratum 7 (Trade-off between argument weights and agreement). Consider trees T1; T2; T3 which are nearly identical and have leaf statements sA and sB with common conclusion s and 8a 2 rI!s : w(a) 6= 0. We have r1(sA) = r2(sA) = 0:5, r3(sA) = 0:5, r1(sB) = r2(sB) = r3(sB) = 0, and w1(sA; s) > w2(sA; s). Furthermore, w2(sB; s) = w1(sB; s) + w1(sA; s) w2(sA; s), w3(sB; s) = w1(sB; s) + w1(sA; s) w3(sA; s), i.e. sB is neutral and “collects” remaining weight such that the sum is 1.
If w1(sA; s) w2(sA; s) > w2(sA; s) + w3(sA; s), although T1 and T2 have the same opinion on sA, we want to have d(T1; T2) > d(T2; T3), because both T2 and T3 do not care much about their (different) opinions on sA. Likewise, if w1(sA; s) w2(sA; s) < w2(sA; s) + w3(sA; s), we expect d(T1; T2) < d(T2; T3) because the weights w1 and w2 are closer to each other and give a greater weight for opposing opinions on sA.
The following example trees depict the first case with concrete weight values. Because the different opinions on statement sA are underneath an argument edge with small weight, we want to have d(T1; T2) > d(T2; T3).
T1 0 s 0:9 0:1 0:5 sA 0 sB
T2 0 s 0:1 0:9 0:5 sA 0 sB
T3 0 s 0:2 0:8 0:5 sA 0 sB Proof. We proof the first case. For (20) = (21), remember that w1(sA; s) > w2(sA; s). d(T1; T2) = wa j0:5 w1(sA; s) 0:5 w2(sA; s)j = wa 0:5 (w1(sA; s)
w2(sA; s)) > wa 0:5 (w2(sA; s) + w3(sA; s)) = wa j0:5 w2(sA; s) ( 0:5) w3(sA; s)j = d(T2; T3) Brenneis et al. / How Much Do I Argue Like You? Towards a Metric on Weighted Argumentation Graphs 11
The other case follows by replacing “>” with “<”.
Desideratum 8 (Trade-off between statement ratings and agreement). Consider trees T1; T2; T3 which are nearly identical and have a statement s and 8a 2 rI!s : w(a) 6= 0. We have r1(s) > r2(s) > 0 > r3(s) such that jr1(s) r2(s)j > jr2(s) r3(s)j. Although T1 and T2 have the same positive opinion on s, we want to have d(T1; T2) > d(T2; T3), because both T2 and T3 have a weak opinion on s. Likewise, if jr1(s) r2(s)j < jr2(s) r3(s)j, we expect d(T1; T2) < d(T2; T3) because the ratings r1(s) and r2(s) are closer to each other than r2(s) and r3(s).
The first case, d(T1; T2) > d(T2; T3), is shown in the following example: T1 0:4 s
T2 0:1 s
T3 0:1 s Proof. Only the summand for s contributes to the distance, all other summands are 0. Remember that all weights on the path to s are positive.
d(T1; T2) = wa jr1(s) r2(s)j > wa jr2(s) r3(s)j = d(T2; T3) (24)
The other case follows by replacing “>” with “<”.
Desideratum 9 (Weights limit the influence of a path). Consider graphs T1; T2 which are nearly identical, have an argument (sA; s) with w = w1(sA; s) = w2(sA; s) and only the ratings and weights below (and including) sA may differ in any way. No matter how those values are chosen, we want to have d(T1; T2) w, i.e. the maximum influence of the differences below sA is limited by the weight of sA to its conclusion. Proof. It is enough to consider paths which include a, since the summands for all other parts are 0. Let Sa be the set of all statements which have an argument leading to a, including a itself. We abbreviate adepth(s0)(1 a) as a(s0).
dT (T1; T2) = å a(s0) r1(s0) s02Sa
Õ sA0 2rS!I As the factor after w is in [0; 1], we get dT (T1; T2)
Although the proposed pseudometric fulfills several intuitive desiderata, there are some limitations which we will discuss in the following.
If a weight of an argument is 0, the proposed pseudometric ignores all weights which are underneath this argument. When comparing how similar people argue for or against T1 0
I the top-level positions, this is okay, but if also the way how arguments which are not supported are attacked should influence the distance, the metric has to be extended.
Moreover, ordering can be changed by adding an unrelated opinion. Consider trees T1; T2; T3 with d(T1; T3) < d(T2; T3). At first sight, it might seem unexpected that this order can be changed to d(T1; T30) > d(T2; T30) by adding a new position c to T3 and keeping the relative weights of the other positions. Depending on the application context, e.g. a voting advice application (VAA), this might be unwanted. An example is depicted in Figure 2. This is due to the normalization of the argument weights.
Although even end-user friendly systems like D-BAS support for undercuts, i.e. arguments that have an argument as conclusion, undercuts are currently not explicitly modeled in our model. This is no big problem, because in many applications, for instance, in a VAA, arguments can be preselected such that no undercut attack is necessary (since the arguments make sense), or rephrased such that the premise is attacked. For example, consider the argument “We should build more nuclear power plants because cats are cute” and the attack “Cats and nuclear power plants are unrelated”. Though this is technically an undercut, a user interface may present this as an attack on “Cats are cute”.
Calculating the distance between argumentation graphs to compare how similar the
attitudes of two agents are has already been used in other systems. The Carneades opinion
formation and polling tool presented in [
The mobile application described in [
Another application of calculating the similarity of weighted trees is match-making
of agents, which are represented by weighted trees. In [
There are already other definitions of weighted argumentation graphs based upon
Dung’s definition of argumentation frameworks, but many lack the differentiation
between argument relevance and statement ratings (e.g. [
In this paper we proposed a pseudometric to calculate the distance between two argumentation graphs representing the attitudes of different persons, and several desiderata which should be considered when proposing other metrics for the same purpose, and are fulfilled by our pseudometric. Possible next steps include developing other sensible metrics and comparing them regarding theoretical properties and practicality.
In future work, we want to check if the desiderata are not only intuitive for experts in argumentation theory, but are following the intuition of untrained humans. We also want to test the metric in a VAA to compare the argumentation of voters with those of parties. Thereby we see whether the results of the metric are accepted in an application context.