We are interested in employing argumentation for intelligence analysis due to its obvious potential for real-world impact. The Analysis of Competing Hypotheses, a well-known technique for multiple hypothesis evaluation from the intelligence community, includes sensitivity analysis as a task which helps analysts identify diagnostic information. We draw upon this notion of sensitivity analysis in this paper to set out a novel algorithm, called the Diagnostic Argument Identifier, that is able to identify diagnostic arguments. We employ a labelling-based approach to compute acceptance probabilities between partitions of arguments, which are then used to calculate the mutual information between the labels of each partition before and after the sequential removal of each argument from a framework. We present the results from one experiment on an abstract framework to assess whether our method can identify diagnostic arguments, and thus aid intelligence analysts. We argue that our algorithmic approach systematises and, therefore, reduces the subjectivity of sensitivity analysis; thus, yielding benefits to intelligence analysts - or any other expert working within a decision or deliberation setting - who need to objectively reevaluate the dependence of their set of conclusions on observed data present within an analysis.
Our work is conducted within the setting of intelligence analysis where one of the most
wellknown techniques within the intelligence community is the Analysis of Competing Hypotheses
(ACH) [
Sensitivity analysis forces intelligence analysts to think diagnostically in that they must establish whether the likelihood of their conclusions changes after the removal of a row entry. A diagnostic data point is one where its removal from the analysis changes the conclusions drawn. Although the ACH does its best to formalise hypothesis evaluation, sensitivity analysis is a challenging and subjective task because it is difficult to remove an observed data point and act as though it never existed when reevaluating.
To this end, we make two contributions. The first is an evaluation-based approach to
probabilistic argumentation which uses the set of labellings discovered by a semantics to calculate
joint and marginal argument acceptance probabilities of partitions of arguments and their labels.
Second, we introduce the Diagnostic Argument Identifier (DAI), a novel algorithm, which applies
the equations from the first contribution to quantify diagnosticity scores of arguments within an
argumentation framework (AF), measuring the change in evaluation after the sequential removal
of each argument from a framework. Our algorithm emulates the task of sensitivity analysis and
should alleviate the reliance on human effort, through use of an algorithmic approach. In [
The remainder of this paper is structured as follows. We begin by providing a brief overview of abstract argumentation in Section 2. In Section 3, we present our two main contributions: the joint and marginal argument acceptance probability equations and the DAI, which is accompanied by the relevant pseudo-code in Section 3.4. We present and discuss a result from a single experiment on an abstract example in Section 4. In Section 5, we relate the DAI to others’ work, and Section 6 concludes with some avenues for future research.
We consider (finite) Dung AFs (A , R) containing a set of arguments A and attack relations
R ✓ A ⇥ A [
We employ a labelling-based approach for this work because the set of labellings resulting
from semantic evaluation enabled the computation of probabilities. A labelling L of a set
of arguments S ✓ A within an AF G is a total function L (S) : S ! LAB that assigns all the
arguments a 2 S to a label l 2 LAB, where LAB = {IN, OUT, UND} in the case of complete
semantics. We only consider complete labellings in this work as they are the foundation upon
which all other semantics can be defined [
As a consequence of a labelling being a total function, arguments that are neither labelled IN nor OUT are labelled UND.
Arguments labelled IN, OUT, or UND for one or all labellings are referred to as credulously or sceptically IN, OUT, or UND, respectively.
We now introduce our main contributions: the labelling-based argument acceptance probability equations for partitions of argument labels, derived from a set of probability spaces (Sec. 3.1), and then the DAI, which is an algorithm that is capable of identifying the most critical arguments within a Dung AF. We explain how to calculate the mutual information (MI) between partitions of labelling vectors (Sec. 3.2) and show how to conduct sensitivity analysis on an AF (Sec. 3.3). The section closes with our second contribution, the pseudo-code for the DAI (Sec. 3.4).
We let G = (A , R) be an AF which contains a set of N arguments A . We order the set of arguments using a function f (A ) : A ! A, where A = (a1, ..., aN ) is an ordered vector of arguments. We assume we have a function g(A) : A ! LM which assigns all arguments ai 2 A to a set of labelling vectors, such that
M
LM = {Li}i=1 where M is the number of labelling vectors, and Li is the i-th labelling vector containing N argument labels, such that Li = (l1, ..., lN ), where l j 2 Li is the label of the argument a j 2 A and l j 2 LAB (Sec. 2).
We now partition the set of arguments into two sets, named Af and Ay .
Definition 3.1. For an AF G = (A , R), let the partitions of A be Af ✓ A and Ay ✓ A , where Af [ Ay = A and Af \ Ay = 0/, and the dichotomous sets Af c and Ay c are complements such that Af c = A \ Ay and Ay c = A \ Af .
Both the sets Af and Ay are mapped to argument vectors through the function f , such that f (Af ) : Af ! f (Ay ) : Ay !
Af = {(a1, ..., a|Af |) | 8 ai 2 A
where ai 62 Ay } and Ay = {(a1, ..., a|Ay |) | 8 a j 2 A where a j 62 Af }, respectively, where i 6= j.
The partitions Af and Ay are mapped to a corresponding set of labelling vectors through the same function g, such that g(Af ) : Af ! Lf and g(Ay ) : Ay ! Ly , respectively, such that Mf Lf = {Lf ,i}i=1 and
My
Ly = {Ly ,i}i=1 where Mf M, My M, Lf ✓ LM, Ly ✓ for the partitions Af and Ay , respectively.
Example 3.1. Consider a Dung AF G with arguments A = {p, q, r, s, t} and relations R = {(q, p), (r, q), (s, q), (r, s), (s, r), (r, t), (t, r), (s, t), (t, s)} (Fig. 1a). Evaluating the AF G under complete semantics produces four distinct labellings (Tab. 1). Following Def. 3.1, we partition the set of arguments A into dichotomous sets, where Af = {p, q} and Ay = {r, s, t}. Using the (1) (2) (3) (4) (5) p t p t function f , we map the sets of arguments A , Af and Ay to the argument vectors A = (p, q, r, s,t), Af = (p, q) and Ay = (r, s,t), which enables the creation of sets of labelling vectors LM, Lf and Ly , as shown in Eqs. 6, 7 and 8, respectively.
We let (W f , Ff , Pf ) and (W y , Fy , Py ) be probability spaces, where W f and W y are sample spaces, Ff and Fy are event spaces, and Pf and Pf are functions such that Pf : Ff ! (0, 1] and Py : Fy ! (0, 1], respectively. We consider two random variables Xf and Xy that are real-valued measurable functions Xf : W f ! R and Xy : W y ! R that map results from from the sample spaces W f and W y to numerical values; thus, modelling a random experiment which, in our case, is the resulting set of labellings output from semantic evaluation of an AF.
Definition 3.2. (Random Vector). Let (W , F , P) be a probability space where the random vector X : W ! R is a measurable function. The random vector X contains random variables X = (Xf , Xy ) defined on two probability spaces Xf : W f ! R and Xy : W y ! R.
Before semantic evaluation and given the constraints of argumentation, the number of possible elements (or labelling vectors) in the sample space W is the number of unique combinations of argument labels, such that
|W | = |LAB|N where |W | is the number of potential elements in the sample space W , N is the number of arguments, and LAB is defined in Sec. 2. After semantic evaluation of an AF, the set of labelling vectors in the sample space is reduced to a set of M labelling vectors, such that W = LM.
The probability spaces (W f , Ff , Pf ) and (W y , Fy , Py ) are measurable spaces, where Ff ⌦ Fy is the smallest s -field of potential subsets of W f ⇥ W y ◆ W , containing all sets of the form Lf ,i ⇥ Ly , j where Lf ,i 2 Ff and Ly , j 2 Fy , such that Ff ⌦ Fy is the product s -field. Thus, the event space F is a s -algebra containing the powerset of all elements in the product space Ff ⌦ Fy , including the empty set and the set of all events. However, after semantic evaluation, the set of combinations of labelling vectors for each partition is, again, reduced to sets of realised events such that Lf ⌦ Ly ⇢ Ff ⌦ Fy is the smallest subset of events from which we can calculate non-zero probabilities, where Lf and Ly are shown in Eqs. 4 and 5, respectively.
The observation of the i-th outcome is denoted x(i) 2 W which is the i-th labelling vector
LM, such that x(i) = (xf(i), xy(i)) = (Lf ,i, Ly ,i) 2 W . Similarly, we can observe outcomes x(i) 2 from different labellings, where xf(i), xy( j) 2 W are the i-th and j-th labelling vector for the partitions Af and Ay , respectively, where i 6= j. With a slight abuse of notation, the probability of Af and Ay ’s labelling vectors being contained in the i-th and j-th outcome in the sample space is denoted as P(xf(i)) or P(xy( j)) which refers to P(Xf = xf(i)) or P(Xy = xy( j)), respectively.
The joint and marginal probabilities of argument labels in each segment are the only probabilities concern our calculation of the MI for each combination of Af and Ay . (9) (10)
First, the joint probability of the labels of arguments in each partition measured across the i-th and j-th outcome in the sample space W is computed using Eq. 10.
P(xf(i), xy( j)) = 1 M
 Ix(i)=xf(k);xy( j)=xy(k)
M k=1 f where IA is unity if A is true and zero otherwise.
Due to the distinct nature of labellings, there will only be one labelling vector in the sample space W that contains the same arrangement of argument labels as the labels for arguments in each partition. Thus, there will be M pairs of labellings vectors which produce non-zero joint probabilities across the product space of realised argument labels in each segment. Consider a list M containing |M | distinct i- j pairs, where |M | = Mf ⇥ My , from the observed product space Lf ⌦ Ly , such that we compute |M | joint probabilities. Each joint probability across the product space is the reciprocal of the number of labellings, if and only if both observed outcomes feature in the same labelling vector across the sample space W , and for any other i- j pair the probability is zero, as shown in Eq. 11.
P(xf(i), xy( j)) = ( M1 , iff 9 m 2 { 1, . . . , M} such that i = i(m) and j = j(m) 0, otherwise.
(11)
It follows that the list M contains M i- j pairs of labelling vectors with a joint probability greater than zero, corresponding to the number of times that the sample space W contained those distinct segments of argument labels, or outcomes x(m) = (xf(m), xy(m)) 2 W .
Example 3.2. Continuing our running example, we employ the sets of labelling vectors Lf and Ly , presented in Eqs. 7 and 8, to compute the joint probability of labels for arguments in each partition. The product space Lf ⌦ Ly contains |M | = Mf ⇥ My combinations of potential labelling vectors based on the unique vectors in each partition, where |M | = 12 in this example. Using both Eqs. 10 and 11, it is easy to see that the first, second and last i- j pair from the product space Lf ⌦ Ly , shown in Eq. 12, is equal to 14 , which is M1 in this example. The joint probability of the third i- j pair in Eq. 12 is equal to zero in both Eqs. 10 and 11 because that event was not observed as a labelling vector in the sample space W (see Tab. 1). (12) (13) (14)
The second probability that we wish to measure is the marginal probability of each unique labelling vector in the sets Lf and Ly , which turns out to be a normalised count across a vector subspace, counting how many times Xf = xf(i) and Xy = xy(i) occurred in the sample space W , as shown in Eqs. 13 and 14, respectively.
P(xf(i)) = Example 3.3. Consider again the running example from Fig. 1a and the partition Af with its set of distinct labelling vectors Lf , as presented in Eq. 7. Using Eq. 13, there are three events within the space Lf which we can calculate marginal probabilities for. The marginal probability of the first Lf ,1 2 W , second Lf ,2 2 W and third event Lf ,3 2 W are equal to 12 , 41 and 14 , respectively.
We wish to quantify the amount of information in the initial AF and the extent to which removing
an argument affects the distribution of labels between partitions of other arguments. We measure
this change through the computation of the MI between partitions of argument labels before and
after the removal of an argument. The MI is a symmetric function that quantifies the conditional
dependence between two random variables and is able to determine the amount of information
communicated, on average, about one random variable through observation of another [
The units of MI depend on the base of the logarithm used in the calculation. For the purposes of this paper, information will be measured in bits which is the logarithm to the base of two.
We simplify the MI calculation presented in Eq. 15 by conducting one summation over |M | pairs of outcomes xf(i) and xy( j) which feature in the realised product space Lf ⌦ Ly , where |M | = Mf ⇥ My , as shown in Eq. 16.
I(Xf ; Xy ) = Â Â xf(i)2 Lf xy( j)2 Ly I(Xf ; Xy ) =
 (xf(i),xy( j))2 Lf ⌦ Ly 0 0
P(xf(i), xy( j)) 1 P(xf(i))P(xy( j)) A
P(xf(i), xy( j)) 1 P(xf(i))P(xy( j)) A
We note that many of the joint probabilities in this sum will be zero for i- j pairs of outcomes in the product space that do not feature in the sample space. So, the only non-zero contributions to the MI summation will be i- j pairs that are in both the observed sample space W and the product space Lf ⌦ Ly . Thus, we restrict the summation to only include outcomes xf(i) and xy( j) that have a joint probability greater than zero, as in Eq. 17.
I(Xf ; Xy ) =
 (xf(i),xy( j))2 Lf ⌦ Ly ,P(xf(i),xy( j))>0 0
P(xf(i), xy( j)) 1 P(xf(i))P(xy( j)) A
In light of this constraint and using Eq. 11, it is easy to see that the only non-zero addends to the MI will be from contributions where the joint probability of pairs of outcomes from the product space feature in the same labelling from the sample space. For the sake of computational efficiency, we substitute Eq. 11 into Eq. 17 so that the MI calculation is, therefore, reduced to a single sum over the set of labellings for our problem setting, such that
M 1 I(Xf ; Xy ) = x(mÂ)2 W M log @ 0
1 M P(xf(m))P(xy(m)) A 1 where x(m) = (xf(m), xy(m)) and W ⇢
Lf ⌦
The MI is a symmetric function which means there will be a total of 22A distinct combinations of Af and Ay which produce unique MI calculations. We, therefore, compute the MI between divisions of arguments for up to half the powerset to completely explore the distribution of information communicated across the sets of labelling vectors within the initial AF. Example 3.4. Turning back to our running example and remembering that Af = (p, q) and Ay = (r, s, t). The MI between each segment’s set of labelling vectors was found to be I(Xf ; Xy ) = 1.5 bits (1 d.p.), under complete semantics. We see that observing the labels of, say, the arguments in Af tells us 1.5 bits of information about the labels of arguments in Ay , and vice versa. (15) (16) (17) (18)
is the set of sensitive relations and R removed, where Ra ✓
R.
Now that we have explained how to determine the MI between segments of labelling vectors within the initial AF, we explain how to conduct sensitivity analysis. To start this task, we sequentially remove each argument a 2 A and the relations containing that argument from the initial AF – creating a sensitive AF G a – which we evaluate using the same semantics chosen earlier, and compute the MI between divisions of labelling vectors in the sensitive AF. Definition 3.3. For an AF G = (A , R) undergoing sensitivity analysis, we refer to G a = (A a, Ra) as a sensitive AF which does not include the argument a1, the argument of interest, where A a = A \ a such that A a ⇢
A and a 26
A a. For a relation r 2 a 2 r features in that relation, then it is removed from the set of relations such that Ra = R \ R
= { r | 8 r 2
2 r} are the relations to be Notation 3.1. When we say Af a or Ay a, we are referring to the two partitions of A a within the sensitive AF G a that obey Def. 3.1, however they use the set of sensitive arguments A a, as outlined in Def. 3.3, instead of the set of all arguments A in the initial AF. a set of labelling vectors through the function g(Aa) : Aa ! LMa , such that Again, we order the set of sensitive arguments A a using a function f (A a) : A a
that A a = (a1, ..., aN 1) is a vector of arguments. We map the vector of sensitive arguments Aa to ! a Ma
LMa = {Li }i=1 where Ma is the number of labelling vectors output from semantic evaluation of the sensitive AF G a, and Lia is the i-th labelling vector containing N 1 argument labels, such that Lia = (l1a, ..., lNa 1), where l aj 2 Lia is the label of the argument a j 2 Aa and l aj 2 LAB (Sec. 2).
The sets Af a and Ay a are mapped to the argument vectors Afa and Afa through f (Af a) : Af a ! f (Ay a) : Ay a !
A A fa = {(a1, ..., a|Af a|) | 8 ai 2 A a where ai 62 Ay a} and ya = {(a1, ..., a|Ay a|) | 8 a j 2 A a where a j 62 Af a}, respectively, where i 6= j.
The sets of distinct labelling vectors corresponding to the partitions Af and Ay are found using g(Afa ) : Afa ! g(Aya ) : Aya !
Lf a = {Lfa ,i}i=f 1 and Ly a = {Lya ,i}i=1
Ma Mya where Mfa
Ma, Mya
Ma, Lf a ✓ for the partitions Afa and Aya , respectively.
LMa , Ly a ✓
LMa , and Lfa ,i and Lya ,i are the i-th labelling vectors
Using the same semantics as earlier, we evaluate the sensitive AF to observe two sets of labelling vectors for the arguments in the vectors A fa and Aya ; thus, enabling us to compute argument acceptance probabilities for partitions of arguments within the sensitive AF. 1It is important for the reader to note that this paper only considers the removal of one argument from an initial AF while conducting sensitivity analysis. However, we note this approach could be extended to remove more than one argument from an initial AF to understand how this affects the results output from sensitivity analysis. (19) (20) (21) (22) (23) Notation 3.2. Let G a be a sensitive AF with the argument a removed (Def. 3.3) and Afa and Aya be two sensitive argument vectors, mapped from the sets Af a and Ay a, which follow Def. 3.1. The random vector X˜ = (X˜ f , X˜ y ) is a measurable function from a probability space (W a, F a, Pa) where X˜ maps elements from the sensitive sample space W a = LMa to events Lf a ⌦ Ly a ⇢ F a. Example 3.5. Back to our running example. We present an MI calculation for the sensitive AF G t . To start, we employ Def. 3.3 to create a sensitive AF G t with arguments A t = {p, q, r, s} and relations Rt = {(q, p), (r, q), (s, q), (r, s), (s, r)} (Fig. 1b). We let Af t = {p, q} and Ay t = {r, s} because that is the same as the original partitions (i.e., Af and Ay in Ex. 3.1) where neither Ay t nor Ay t contain the argument t. We map the dichotomous sets of arguments Af t and Ay t to the argument vectors Atf = (p, q) and Aty = (r, s), respectively, through the function f . We, again, use complete semantics to evaluate G t to observe the set labellings for each partition (Tab. 2), allowing the realisation and mapping of each argument in Atf and Aty to their respective sets of labelling vectors Lf t and Ly t , through the function g. The MI I(X˜ f ; X˜ y ) between the labelling vectors for this partition of sensitive arguments is equal to 0.9 bits (1 d.p.).
For a sensitive AF G a, the total number of combinations of Af a and Ay a that produce unique MI calculations is equal to 2A2 a , which is equal to half the number of unique MI results (i.e., 22A ) that we calculate in the initial AF G .
Once we have computed the set of distinct MI results using half the possible combinations of Af and Ay in the initial AF and Af a and Ay a in the sensitive one, we can quantify a diagnosticity score, which describes how much information was lost or gained between either Af and Ay ’s set of labelling vectors after the removal of the sensitive argument, as stated in Def. 3.4. Definition 3.4. The diagnosticity score is defined as the change in MI before (calculated using the labelling vectors Lf and Ly for all arguments in Af and Ay from G ) and after (calculated using Lf a and Ly a for all arguments in Af a and Ay a from G a) the removal of the argument of interest. Eq. 24 computes the diagnosticity score when the argument of interest a was removed from either Af or Ay .
D (Af , Ay ; Af a _ Ay a) = I(Xf ; Xy )
I(X˜ f ; X˜ y ) (24)
An interesting point to note about Eq. 24 is the sign. A positive diagnosticity score infers that there was more information communicated, on average, between partitions in the initial AF G , whereas a score below zero indicates that there was more information transferred between partitions of argument labels within the sensitive AF G a.
Example 3.6. Coming back to our running example for the last time, we consider the impact that removing the argument t had on the acceptability of arguments within each partition in the initial and sensitive AFs. We employed the set of labelling vectors of Af and Ay to calculate the MI between Xf and Xy in the initial AF (Ex. 3.4), and used the set of labelling vectors of Atf and Aty to compute the MI between X˜ f and X˜ y in the sensitive AF G t (Ex. 3.5). We now show the use Def. 3.4 by presenting the diagnosticity score D (Af , Ay ; Ay t ) for the aforementioned partitions, which turns out to be 0.6 bits (1 d.p.).
We present the pseudo-code for the DAI in Algorithm 1. The algorithm takes as input an AF G and a semantics S capable of producing more than one labelling, and returns a diagnosticity vector D[A ][ 22A ], which contains 22A diagnosticity scores for every argument in the graph.
We have introduced a novel algorithm that uses an evaluation-based approach to quantify how much the removal of an argument changes the semantic evaluation of an AF (Algo. 1). To evaluate the effectiveness of our method, we now present the results of an experiment conducted using the AF in our running example (Fig. 1). One of the reasons we chose to use the AF presented in Fig. 1 was because we assumed it possessed a similar topology to graphs found within an intelligence setting (i.e., many symmetric attacks as a result of conflicting information).
Results and Discussion. We computed 22A (i.e., 16) diagnosticity scores, corresponding to the total number of segments of labelling vectors, for all arguments in the running example (Fig. 2).
Looking to Fig. 3, we present a violin plot which groups together the individual diagnosticity scores in Fig. 2 to show the distribution of change in MI for partitions of labelling vectors after the sequential removal of each argument in the AF G within the running example. We also included the median and mean diagnosticity scores, as well as the average absolute values, to better understand and compare the distribution and sign (i.e., positive or negative) of scores for all the removed arguments.
I(Xf ; Xy ) += M1 log M P(xf(m1))P(xy(m)) m-th labelling vector of the initial AF (Eq. 18).
MIs { Af , Ay , I(Xf ; Xy )} ✓
◆ for a 2 A do
G a = (A a, Ra) L (A a) Aa = f (A a) : A a ! Aa W a = g(A) : A ! LMa Ma = |LMa | for Af , Ay , I(Af ; Ay , ) 2 MIs do if a 2 Af then
Af a = Af \ a; Ay a = Ay else if a 2 Ay then
Ay a = Ay \ a; Af a = Af
. Append the result to the array of initial MI calculations. . Create Af a and Ay a by removing a from Af (Defs. 3.1 & 3.3).
. Create Af a and Ay a by removing a from Ay (Defs. 3.1 & 3.3).
Input: An AF G = (A , R) and a semantics S
Output: D[A ][ 22A ] 1: function DAI(G , S) 2: L (A ) 3: A = f (A ) : A ! A 4: W = g(A) : A ! LM 5: M = |LM | 6: MIs[ 22A ]
D[A ][ 22A ]
22A do for Af 2
Ay = A \ Af Af = f (Af ) : Af ! Af ; Lf = g(Af ) : Af ! Lf ; I(Xf ; Xy ) = 0 for (xf(m), xy(m)) 2 W do
P(xf(m)); P(xy(m)) 7:
The argument t was arguably the most diagnostic as it had the largest median, mean and average absolute diagnosticity scores (Fig. 3). This result is intuitive because t has symmetric attacks between the arguments r and s such that removing t reduces the number of complete labellings from four (Tab. 1) to three (Tab. 2).
The argument q had the second largest set of diagnosticity scores (Fig. 3). Even though q only attacked one argument (Fig. 1) and its removal does not reduce the number of labellings (Tab. 1), the conditional dependence between t and q meant that q produced greater median and mean diagnosticity scores. An interesting point to note is that the removal q from the initial AF would change the label of the argument p so that it was sceptically labelled IN within the sensitive AF G q. Thus, for the sensitive AF G q, the MI scores would equal zero for partitions where p was the only argument in a segment (i.e., Af q = {p} or Ay q = {p}) because the MI is always zero for partitions that only contain arguments which are sceptically labelled.
While the arguments r and s are not the most diagnostic, their removal from the initial AF results in the same distribution of change in MI (Fig. 3), which can be attributed to the symmetry between them in the initial graph. The median change in MI was below zero (Fig. 3) which indicates that, on average, more information was communicated between segments of labelling vectors in the sensitive AFs G r and G s after the removal of r and s, respectively.
The preliminary result presented in this paper is a first attempt within the literature to combine abstract argumentation and a technique from the information-theoretic literature for sensitivity analysis. We were able to quantify the sensitivity, dependence and robustness of an AF’s set of conclusions based upon the arguments it was comprised of.
The prior literature covers work on probabilistic argumentation, namely the epistemic [
To the best of our knowledge, there has only been two proposals before this one which use
argumentation for sensitivity analysis. The first employs argumentation and Markov random
fields to quantify the sensitivity of items of information [
Aside from the two above works on sensitivity analysis, we consider the argument strength
literature to be the most closely related to the DAI. Strength has been represented by arbitrarily
assigning a weight to an argument [
The novel algorithm presented in this paper is one of the first attempts within the literature to use of argumentation for sensitivity analysis. We use the labellings output from the evaluation of an AF to compute joint and marginal argument acceptance probabilities which are employed in MI calculations before and after the removal of an argument. We argue that the diagnosticity scores output by the DAI provide a holistic quantification of the sensitivity, dependence and robustness of an AF’s evaluation. We contend that such a tool would provide benefit to intelligence analysts by algorithmically identifying sensitive arguments found within an analysis.
Future work could involve intelligence analysts, comparing the outputs from the DAI with the outcome from sensitivity analysis conducted by a set of analysts. And, of course, extending the DAI to include more flavoursome AFs is an obvious extension. Another avenue for future work could be to investigate different domains in which the DAI could be applied. For instance, the DAI could aid in decision and deliberation problems, allowing users to utilise the rational logic of argumentation and probability theory to identify and focus on crucial arguments with their task, or by providing reasoners with an indication of which arguments are the most important to attack when analysing a debate. Adding or removing multiple arguments is an interesting idea to pursue as it might better explain the diagnosticity of sets of arguments. Finally, one could argue that the DAI, in its current state, is computationally expensive. Cheaper approaches, such as counting the number of attacks from and to an argument or whether an argument is in a dense area of an AF, should be explored to identify whether they are capable of producing the same results as the DAI.
This work was funded by both the Defence Science and Technology Laboratory and the United Kingdom’s Engineering and Physical Sciences Research Council (Grant number: EP/S023445/1).