Explainable methods have received increased attention within artificial intelligence. Wherever an automated system makes a decision an explanation is required to convince a user about the decision. Furthermore, online information quality assessment is crucial to help users navigate information. However, explaining the assessment of online information had not been clarified well. The current work provides explanations to a user about the assessment of online information and specific, provides explanations for the quality assessments of online reviews. We construct an abstract argumentation framework (AF) based on a set of given reviews. We consider the grounded semantics of AFs to assess each topic. Then, we discuss the question of why a score can be assigned to a topic of a product. Furthermore, we indicate a proper score of a review based on the scores of topics within the review in question. We also collect arguments that can support the chosen score of a review.
Argumentation is one research area that is frequently mentioned in explainable AI [
Abstract argumentation frameworks (AFs for short), as introduced by Dung [
In the work presented in this paper, we aim to model a set of reviews with AFs and use the
grounded semantics of AFs as a means of evaluation and explanation. That is, this model allows
reasoning on the ‘acceptability of topics within reviews’ when reviews are conflicting with each
other. Such conflicts are solved by defining a weight for topics within reviews, which extends
and generalizes a review weight we defined in previous work [
Thus, we do not miss any information presented in a review. 2. By our method, after the assessment, a user can ask about the score (strength) of topics of a product, instead of just asking about the scores of the reviews. 3. Our method is solid enough to explain to a user the reason of assigning a score to a topic.
Also, explaining why an argument does not have a roll in the assessment of the score of a topic. 4. We also accumulate the scores of topics within a review to assess a review score. Furthermore, we present an explanation for a review score from a machine point of view, which is a subset of the grounded semantics of AFs. Thus, this explanation is beyond any doubt. 5. For indicating the scores of topics about a product within a set of reviews, based on a system’s points of view, we do not need to consider any generalization of AFs. Since the weight of each argument is used to indicate the direction of the attack relation.
The rest of the paper is structured as follows. Section 2 presents related work, and Section 3, introduces abstract argumentation frameworks. Section 4 presents a model to formally represent and reason on reviews. Section 5 concludes and outlines future work directions.
Explainable artificial intelligence (XAI) has received increased attention to explain decisions of
automated systems [
On the other hand, a significant amount of research investigates the quality of reviews. In [
We start the preliminaries of our work by recalling the basic notion of Dung’s abstract
argumentation frameworks (AFs) [
Definition 1. [
Let = (, ) be a given AF. For each , ∈ , the relation (, ) ∈ is used to represent that is an argument attacking the argument . An argument ∈ is, on the other hand, defended by a set ⊆ of arguments (alternatively, the argument is acceptable with respect to ) (in ) if for each argument ∈ , it holds that if (, ) ∈ , then there is a ∈ such that (, ) ∈ ( is called a defender of ).
Diferent extension-based semantics of AFs present which sets of arguments in a given AF
can be accepted jointly [
Set ⊆ is called a conflict-free set (extension) (in ) if there is no , ∈ such that (, ) ∈
. The characteristic function : 2 ↦→ 2 is defined as () = { | is defended by }. A
set ∈ cf( ) is the grounded extension in if is a unique fixed point of (∅).
Example 1. Let = ({, , }, {(, ), (, )}) be an AF. In , (, ) means that argument
attacks , and (, ) means that attacks . Here, argument is defended by set {} (alternatively,
is acceptable with respect to {}), since attacks the attacker of , namely . The set of conflict-free
sets of is cf( ) = {∅, {}, {}, {}, {, }}. A unique grounded extension of is {, }. The
intuition is that is not attacked by any argument, thus no one has any doubt about accepting
argument . Argument is attacked by , however, it is defended by which was accepted by
everyone.
1The reader interested in semantics of AFs can see [
Here, we first model reviews in a formal manner by means of abstract argumentation frameworks (AFs), and then show how to use AFs to evaluate the score of a topic of a product based on given reviews (see Section 4.2). To this end, we collect the reviews with the same score containing the same topic and we consider them as a single argument. After indicating the attack relations between arguments, we construct an AF. Then, we use the grounded semantics of AFs for indicating the most trusted arguments and for a reason of explaining why a score is assigned to a topic and a review in our automated method.
Let be a product (target) and let {r()} be a set of reviews of (alternatively, {r}). Each review r consists of a numerical score (r) (e.g., in 5-level Likert scale) and a textual description. The description characterizes the product focusing to specific topics. E.g., in the case of a pair of shoes, the topics can be ‘sole’, ‘upper’ or ‘comfort’. We represent the list of all topics relevant to the product as , where = {1, . . . , }.
Each review r contains a finite set of topics r ⊆ . E.g., review ‘r1 : the color of the shoes are not my favorite but I use it for a long time and they are still looking good’ contains two topics, i.e., r1 = {1 = color, 3 = quality}. The review score represents the actual reviewer’s opinion on the product, while the text aims at motivating such judgment. An agreeing score of two reviews containing the same topic about the same product indicates a support between reviews while disagreeing score indicates a conflict.
Our goal is to identify the score of each topic of a product, based on the set of given reviews, i.e., which score is proper for a topic in question. We address this by constructing an AF based on a set of given reviews. Then, we use the grounded semantics of AFs to indicate a score to a topic in question and explain why it is the case. In the end, we further update the scores of reviews based on their topics. We also, prepare an explanation for the updated score of the reviews by using the grounded semantics of AFs.
In review r in Table 1, index indicates the product and indicates the reviewer. Reviews can support one another if they have the same score and they have a common topic. If a set of reviews has the same score and has a common topic, then it means that these reviews support one another. Thus, we consider them as a single argument. In Table 1, topic ’ : quality’, indicated by a topic detection, is a common topic between r21 and r22, however, these reviews do not have the same score. Reviewer r21 gave a score of 4 out of 5 to this product. The last sentence in r21 presents the main reason why the reviewer gave a score of 4 out of 5 to this product because she/he was satisfied with the quality of this product. However, the reviewer of r22 is not as satisfied as the reviewer of r21 with the quality of this product. Thus, there is an attack relation between reviews r21 and r22, because (r21) ̸= (r22).
Reviews can be classified based on their topics. Assume that is a topic of a set of reviews = {r1, . . . , r }, if there is ′ ⊆ , such that for any r, r ∈ ′, it holds that (r) = (r ), then we say that reviews in ′ support one another and we consider all reviews as a single argument. This leads to the classification of based on the scores of reviews. Between arguments with diferent scores, there is a symmetric attack relation. If there is no intersection Id r11 r12 r21 r22
Score *****
* **** ***
Review Text
Comfortable
Waaaay too BIG Fit perfectly. I bought dark grey, and they didn’t fade
They fit great. But they fade bad between topics within r and r , then there is no relation between these two reviews.
Let {r1, . . . , r} be a finite set of reviews on product . Let = {} be the set of topics relevant to the product . Each review r consists of a numerical score (r) (e.g., in 5-level Likert scale) and a textual description. Let ,r be the set of all topics presented in review r. Diferent topics have diferent importance in each review, to indicate this importance we introduce a weight function in Definition 2 which shows the initial weight of topic in review r with score (r).
Definition 2. Let r be a review, let be a topic in r, and let (r) be a score of r.
((r), , r) is called the initial weight of in review r and score (r), where is
a value in [0, ∞] such that is computed by aggregating one or multiple factors meeting the
following criteria:
1. at least one of such factors is the result of an abstraction function computed on the
review itself. Such abstraction functions should be computable over any review and allow
establishing a total order of reviews. Example of such abstraction functions include, for
instance, readability scores (e.g., Dale-Chall readability [
We aim to use a formalism of argumentation, i.e., abstract argumentation frameworks (AFs), to assess the score of the topics within reviews of a product . To this end, we construct an AF based on a set of reviews. Our method associates each argument to the set of reviews with the same score about a common topic. That is, we consider all reviews with the same score that presents the same topic as a single argument. Formally, let [] = {r | r contains topic and (r) = }, i.e., [] collect all the reviews that contains topic and have score . We consider [] as a single argument, that contains all the reviews that have topic and score . We are interested in evaluating the weight of [] as a single argument. To this end, for each with ∈ {1, 2, 3, 4, 5}, we sum up the initial weights of topic in review r and score . Note that in Definition 3 if is not a topic of r then (, , r) = 0. Definition 3. Let be a topic of a product , let be a natural number between 1 and 5. The weight of topic with respect to score , alternatively, the weight of argument [] is as follows: ([]) = Σ =1(, , r) where is the number of reviews about product , and (, , r) is the initial weight in review r and score as introduced in Definition 2.
To construct an AF based on a set of reviews, we consider an attack relation from an argument [] to [] if ([]) > ([] ).
Definition 4. Let be a product, let = {} be a set of topics, and let be a natural number between 1 and 5. For each , with ∈ {1.2, 3, 4, 5}, and for each ∈ , we introduce an argument , = []. Furthermore, (,) is the weight of argument ,, equal to ([]), as introduced in Definition 3. An AF constructed based on topics is = (, ) where, • = {,} • = {(,, ,) | ,, , ∈ and (,) > (,)}.
An AF, constructed based on topics of a product, is a directed graph. Each node, indicated by , contains all sets of reviews that contain topic with score . The reason for collecting all such reviews is that if two reviews contain the same topic and give the same score to the product, then their content support one another. Thus, we accumulate them in a single node and we consider them as one argument. After indicating the set of arguments from a given set of reviews, we designate attack relations. If two arguments give diferent scores to product , but contain the same topic, it means that there may exist a conflict between these two arguments with respect to the topic in question. We consider an attack relation between , and , when their weights are not the same. In order to indicate the direction of the attack relation between two arguments we consider the weight of the topic with respect to the score, presented in Definition 3. Note that in Definition 4 we only consider relations among arguments with the same topics. Thus, for , ′ ∈ , if ̸= ′, then there is no relation between any arguments of , and ,′ , for , ∈ {1, 2, 3, 4, 5}. That is, the associated graph to AF , constructed based on the topics of a product, is a forest, presented formally in Lemma 1. Note that in graph theory a graph is called connected if for every pair of vertices and , there is a path between and . Furthermore, a connected component is a maximal connected subgraph of an undirected graph. Definition 5. Let be a product, and let = {} be a set of topics of . Let = (, ) be an AF constructed based on topics. Let ∈ , set = {,} is called a component contains in . Furthermore, component is called connected component if for every ,, , ∈ it holds that (,, ,) ∈ .
Lemma 1. Let be an AF, constructed based on topics of product . If the reviews of the product contain at least two topics, then the graph associated with is disconnected. Furthermore, If is the number of topics presented in the reviews, i.e., || = , then the associated graph of contains at least connected component. Proof. If the reviews contain more than one topic, for instance and ′, then it is clear that there is no relation between arguments containing and ′. Thus, the associated graph is disconnected. That is, there is no link between the arguments of components and ′ . Thus, if we have at least two topics in the set of reviews, then the associated graph is disconnected.
We now show that the associated graph contains at least connected components if the set of reviews contains number of diferent topics, by induction on .
Base case: assume that = 1, that is all reviews contain the single topic . If all reviews give the same score to the product, then we only have one argument in the AF constructed based on these reviews. Thus, we have exactly one component. Note that if we have diferent arguments with the same weight, i.e., || > 1 and (,) = (,) for every ,, , ∈ , then there are more than one connected component.
Inductive hypothesis: Assume that = , then the associated graph contains at least connected components.
Inductive step: Assume that = + 1, then we have to show that the associated graph contains at least + 1 components. Let be diferent topics of , i.e., ⊆ and || = . By the inductive hypothesis, the associated graph contains at least connected components. Let be a topic such that ̸∈ , and let ′ ∈ . Thus, there is no relation between arguments of and ′ . Thus, if = + 1 the associated graph has at least + 1 connected component.
An AF = (, ) is called acyclic (or well-founded) if there is no infinite sequence of
arguments 1, . . . , , . . . such that (+1, ) ∈ . By Definition 4, in an AF = (, ),
it holds that (,, ,) ∈ if (,) > (,). Furthermore, because of the transitive
property of relation < in real numbers, any AF constructed based on a set of reviews is acyclic.
It is proven in [
Corollary 1. Let be an AF, constructed based on topics of the product . Every connected component in the associated graph of is acyclic.
Proof. This corollary is the direct result of the fact that is acyclic. Thus, any connected component of is also acyclic.
Since, by Corollary 1 every connected component in is acyclic, every component has an initial argument, i.e., an argument that does not have any parents. Proposition 1 is the direct result of Definition 4.
Proposition 1. Let be an AF constructed based on a set of topics of the product . Let , be an initial argument of . It holds that (,) is maximum among the weights of other arguments. Lemma 2. Let = (, ) be an AF constructed based on a set of topics of the product . Let be the set of initial arguments of , i.e., = { | there is no ∈ such that (, ) ∈ }. The grounded extension of is none empty and it is equal with .
Proof. Since every AF constructed based on topics is acyclic and since each acyclic AF has a none empty grounded extension, has a none empty grounded extension. By the definition of grounded semantics, every initial argument is in the grounded extension, i.e., ⊆ grd( ). It remains to show that grd( ) ⊆ . Toward a contradiction, assume that grd( ) ̸⊆ . That is, there exists an argument in grd( ) which is not an initial argument. Assume that , is an argument such that , ∈ grd( ) but , ̸∈ . Since grd( ) ̸= ∅ and ⊆ grd( ), there exists , ∈ grd( ) ∩ . By Proposition 1, (,) is the maximum among the weights of other arguments, in specific, (,) > (,). Thus, by Definition 4, (,, ,) ∈ . That is, , is attacked by , in . This is a contradiction of the assumption that , ∈ grd( ). Thus, the assumption that there exists an argument in the grounded extension which is not an initial argument is wrong. Hence, grd( ) = .
In this section, we explain how AFs as an AI system can be used to indicate the score of each topic of the product , based on the set of reviews.
Proposition 2. Let = (, ) be the constructed AF based on topics of product . Let be a topic of product . There exists ∈ {1, 2, 3, 4, 5} such that , is in the grounded extension of . Proof. Let be an AF, constructed based on topics of the product . Let be a topic and let be the connected component of , as introduced in Definition 5. By Corollary 1, every connected component in the associated graph of is acyclic. Thus, for each , contains an initial argument. By Lemma 2, the grounded extension of and the set of initial arguments of coincide. Thus, for each an argument of is in the grounded extension. Hence, for any , there exists such that , is in the grounded extension of .
We now define basic explanation in terms of functions. The function (− ) is a unary function that takes a topic as an input and returns an appropriate score to that topic by considering all the reviews containing that topic, presented in Definition 6.
Definition 6. (Score of a topic) Let = (, ) be an AF constructed based on topics of product . Let be a topic of product . The score based on an AI system is denoted by (), defined as follows: () = round(
Σ ,∈grd( ) |{ | , ∈ grd( )}| ) In () the output of the function round is the nearest integer to |{Σ| ,,∈∈ggrdr(d())}| .
Note that by Proposition 2 for each topic there exists at least an such that , ∈ grd( ). Hence, Definition 6 is well-defined. Note that for each , if component is connected, then there exists exactly one such that , ∈ grd( ). Intuitively, for a , if component is connected and , is an initial argument, then it holds that () = . In this case, the choice of machine, i.e., () = can be explained that the score of topic is because argument , is an initial argument, i.e., (,) is maximum among all other arguments in component . The highest weight of , among the arguments of means that sum of the weights of reviews containing with score is the highest weight.
Furthermore, in the following, we try to explain to a user why from a machine point of view topic has the score sc (). To this end, we define an explanation function, denoted by Exp(, sc ()) in Definition 7. Function (, sc ()) is a binary function that takes a topic and its machine score sc () as inputs and returns the set of arguments that their scores have a role in the machine decision in Definition 6.
Definition 7. (Explanation of a score of a topic) Let be an AF, constructed based on topics {} of product . Let be a topic and let sc () be the score of from a machine point of view, as introduced in Definition 6. An explanation of why the score of topic is sc () is as follows:
Exp(, sc ()) = {, | , ∈ grd( )}
Let be a topic, Exp(, sc ()) collects all , that is in the grounded extension of , i.e., the set of arguments, with respect to topic , that the acceptance of them are beyond of any doubts. Assume that sc () = , the notation ¬sc () is a number such that ̸= .
Furthermore, the function NotDef() collects all the arguments that contain topic but do not have any role in the computation of the score of topic , from a machine point of view. In other words, NotDef() contains all arguments which are attacked by Exp(, sc ()). Definition 8. Let be an AF, constructed based on topics {} of product . Let be a topic and let sc () be the score of from a machine point of view, as introduced in Definition 6. Function NotDef() is a unary function that takes a topic as an input and returns the set of arguments that do not have any role in the evaluation of sc ().
NotDef() = {, | , ̸∈ grd( )}
For each topic , Exp(, sc ()) and NotDef() are disjoint functions, i.e., Exp(, sc ())∪ NotDef() = . For each , if , ̸∈ Exp(, sc ()), then , ̸∈ grd( ), that is, there exists , ∈ grd( ) such that , attack ,. That is, (,) > (,), thus, it is reasonable that the machine neither consider the score of , in the evaluation of sc () nor argument , in Exp(, sc ()). Furthermore, the function SC (− ) gives a score to a review r based on the score of each topic within r. That is, SC revise the score of each review given by a reviewer. Definition 9. (Score of a review) Let r be a review of product . Let ,r be a set of topics presented in review r. The score of the review r based on a machine assessment is the output of function SC (r), evaluated as follows:
SC (r) = round( Σ ∈,r sc () |,r | )
The score of review r, from a machine point of view, is the round of the average of scores of the topics presented in r, where the score of each topic from a machine point of view is presented in Definition 6.
In Definition 10 grounded semantics of AFs are implicitly used to explain the reasons for the score given to a review from a machine’s point of view. Definition 10. (Explanation of a score of a review) Let r be a review of product . An explanation of why the score SC(r) is given to a review r is as follows:
Exp(r, SC(r)) =
Exp(, sc ()) ⋃︁ ∈,r
In Definition 10, function Exp(r, SC(r)) collects all the explanations for all the topics within r. In other words, this function collects all the initial arguments of components , where ∈ ,r . That is, Exp(r, SC(r)) ⊆ grd( ), thus, there is no doubt on the acceptance of these arguments.
We present a method to consider reviews containing a comment topic with the same score as a single argument. We aggregate the initial weights of the reviews in an argument to indicate the main weight of the arguments. We introduce an attack relation between arguments by considering the weight of arguments. Then, we use the notion of grounded semantics of AFs to evaluate the most trusted arguments. Since each argument , in the constructed AF is attached by , if ̸= and (,) > (,), we evaluate the score of topics in the grounded extension. Next, we introduce a function to explain the reason for choosing the associated score of a topic by a system. Then, we present a function to accumulate the scores of topics within a review to assign a score to a review from a machine point of view. In the next step, as an explanation method, we also introduce a function to give a review and its score from a system point of view as inputs and returns all the arguments that have an efect on the assessment of a review in question from a system point of view.
In our approach, we identify an attack relation between arguments if they are about the same
topic but their weights are diferent. As future work, we are eager to present relations among
arguments that do not have a common topic (we are eager to consider some other features of
arguments for presenting relations among them). Furthermore, we are interested in using the
non-monotonic nature of argumentation theory for working on a temporal way of reasoning
instead of considering a fixed set of reviews. That is, we aim to show how we can evaluate the
quality of reviews if we consider the order of time of presenting of reviews. To have further
human-machine interaction we aim to consider user preferences over the topics of a product to
evaluate the score of a review. It is a possible topic for future work to use the generalizations of
AFs, namely, valued-based AFs [
Note that the tasks of argument mining from a given KR and automatically constructing of the associated AF are not within the scope of this work because we focus on studying the use of the grounded semantics of AFs as a means to assess product reviews. Given that this shows to be a promising direction, future work will focus on optimizing the task of AF construction by combining human and automated computation.
Acknowledgments This research has been supported by the Centrum Wiskunde & Informatica (CWI) and Supported by the Netherlands eScience Center project “The Eye of the Beholder” (project nr. 027.020.15).