Product rating systems are very popular on the web, and users are increasingly depending on the overall product ratings provided by websites to make purchase decisions or to compare various products. Currently most of these systems directly depend on users' ratings and aggregate the ratings using simple aggregating methods such as mean or median [1]. In fact, many websites also allow users to express their opinions in the form of textual product reviews. In this paper, we propose a new product reputation model that uses opinion mining techniques in order to extract sentiments about product's features, and then provide a method to generate a more realistic reputation value for every feature of the product and the product itself. We considered the strength of the opinion rather than its orientation only. We do not treat all product features equally when we calculate the overall product reputation, as some features are more important to customers than others, and consequently have more impact on customers buying decisions. Our method provides helpful details about the product features for customers rather than only representing reputation as a number only.
Many websites nowadays provide a rating system for products, which is used by customers to rate available products according to their own experience. Reputation systems provide methods for collecting and aggregating users’ ratings to calculate the overall reputation for products, users, or services [2]. This final rate is very important, as it represents the electronic ‘word of mouth’ that customers build their trust in a product on. On the other hand, most websites allow customers to add textual reviews to explain more about their opinion to the product. These reviews are available for customers to read, to the best of our knowledge, they are not analyzed and counted in the product overall reputation. Many reputation models have been proposed, but most of them concentrated on user’s reputation in C2C (Consumer to Consumer) websites such as eBay.com, while service and product reputation has received less attention. Besides, most of the literature about product reputation models neglected users’ reviews and counted users’ ratings only. Therefore, their reputation systems did not provide any summaries and details about the weakness and strength points in the product.
In this work we will provide a reputation model for products using sentiment analysis methods. The proposed model generates reputation for a specific product depending on the textual reviews provided by users rather than depending on their ratings because users’ ratings do not reveal an actual reflection for the products’ features, and they do not provide details for customers about features reputation and about “why” the reputation is high or low. For example, a strict user might give three stars for the product although he does not have a clear negative opinion about the product. On the other hand a more generous customer might have a couple of negative opinions about the product but still give four stars. Additionally, textual reviews can be used to provide summaries about product features reputation in addition to the aggregated value for the product reputation, which can make the reputation system more meaningful rather than being just a number. We calculate features impact by counting how many times every feature is mentioned explicitly in the text reviews, assuming that features that are mentioned more by users are more important for them.
In the rest of this paper, we will demonstrate couple of existing product reputation model in the section II, and in the following sections we will explain equations we use to calculate the reputation value for a product. We will also provide diagrams to show the difference between the results of our reputation calculation method and the regular average method used by most websites to represent the overall product reputation. 2 2.1
Related Work
Reputation Models
Reputation models have been studied intensively by many researchers in the last decade, many of these researches concentrated on user’s reputation and some of them have discussed product reputations. One of the most basic works on ratings aggregation analyzed robustness of different aggregators, in particular the mean, weighted mean, median and mode, and proposed that using median or mode is more efficient than using mean [1]. Cho et al. [3] proposed a more sophisticated model, they calculated user reputation and used it in order to calculate weights for different ratings. Moreover, they assumed that some users tend to give higher ratings than others, hence, they calculated rating tendency for users and deducted it from user rating. They used the user’s accurate prediction and the degree of his activity to define his level of expertise, and then they used this value to represent user’s reputation. This method might not be an accurate way to give different weights for ratings, because a user’s reputation should not reduce the weight of his opinion about a product. On the other hand, another promising work introduced by Leberknight et al. [4], discussed the volatility of online ratings, where authors aimed to reflect the current trend of users’ ratings, they used weighted average where old ratings have less weight than current ones. They introduced a metric called Average Rating Volatility (ARV) that captured the extent of fluctuation present in the ratings, and then they used it to calculate discounting factor, which is used in weighting older ratings. 2.2
Opinion Mining
Many literatures have focused on extracting useful information from the huge amount of available users' opinions in the internet. Opinion mining was used in many different domains. Business Intelligence is the most popular one, where many studies concentrated on mining customers' reviews for better market understanding [5]. Researchers focused on the sentiment analysis part and represented product reputation as a simple count of positive and negative sentiments [6] [7]. Turney [8], Pang et al. [9], and Kamps et al. [10] provided different methods to determine the orientation of a word as positive or negative. In contrast, Hu & Liu [6] proposed a set of techniques for mining and summarizing product reviews to provide a feature based summary of customer reviews, they searched for frequent noun and noun phrases as candidate features. While Popescu et al. [11] identified parts and features of a product depending on finding relation between noun words and the product class using PMI algorithm [8]. Morinaga et al. [12] were one of the first researchers to introduce a general framework for collecting and analyzing users’ reviews in order to find the overall product reputation. They used two dimensional positioning Maps, which contained the extracted opinion phrases and associate products with them. The distance between opinion-phrases and products represents closeness. Their proposed method does not mine product features [6], which might be crucial element in the product reputation analysis. In contrast, Hashimoto & Shirota [13] depended on buzz marketing sites to provide a framework for reputation analysis considering product’s features. They attempted to discover the topic of each review as initial step, and then they determined important topics depending on the contribution rate of each topic and the polarity of the messages. Finally, the results are visualized for users. However, the effectiveness of their framework has not been evaluated, and the visualization method used to represent the results has not been perfected. Moreover, they neglected topics with lower contributions which might affect the overall product reputation.
To the best of our knowledge none of the previous work has proposed a convenient method to calculate product reputation, depending on the outcome of mining users’ reviews. Most of the available methods represent the reputation as a simple count or average of positive and negative opinions in the reviews. While the convenient represented models depended on users’ ratings rather than users’ textual reviews. 3 3.1
The Proposed Approach
Definition
A product can be described by a set of features representing its characteristics. Some of the features may be more specific or more general than others. For example, for a specific mobile phone product, the “Mobile Camera” is considered as a general feature, while Resolution, Optical Zoom, Flash Light, Video Recording are more specific features of Mobile Camera. In this paper, we define product features as a hierarchy.
Definition 1 (Feature hierarchy): A feature hierarchy consists of a set of features and their relationships, denoted as FH # !F, L" , F is a set of features where F # !f1, f2 ,..., fn "and L is a set of relations. In the feature hierarchy, the relationship between a pair of features is the sub-feature relationship. For f i , f j $ F , if f j is a sub-feature of f i , then ( f i , f j ) $ L , which means, f j is more specific than f i . The root of the hierarchy represents the product itself, and the first level children are the generic features. In this paper, we assume that the feature hierarchy is available.
Definition 2 (User’s Review): R is a set of reviews where R # !r1 , r2 ,..., rm " . Every review consists of a number of opinions about different features, denoted as %ri $ R ri # {( f i1 , oi1 , si1 ),..., ( f in , oin , sin )} . oij is the orientation of the opinion; oij $!Pos, Neg, Neu" , which represents positive, negative, and neutral respectively. si is the strength of the opinion, si $!1,2,3", where 1 represents “weak opinion”, 2 for “moderate”, and 3 for “strong opinion”.
In this paper, we assume that the product features and the opinion orientation and strength to the features in each product review have been determined by using existing opinion mining techniques. The proposed reputation model will generate product reputation based on the opinion orientation information, i.e., this information is available, and is the input to the reputation model. There are different methods that can be used to extract this information [14] [15].
For a specific feature f j , the set of negative reviews are denoted as R njeg # !ri oij # Neg" ,
R njeg # *+)r1neg , r2neg ,..., r Rnenjgeg '&( , ri neg # {( fi1neg , oin1eg , sin1eg ),..., ( fijneg , oinjeg , sinjeg ),...} where %rineg $ R neg j
The same definitions also apply for the set of positive reviews R jpos . The neutral orientation reflects the lack of opinion about the specific feature and consequently will not be considered in the reputation model.
Our proposed product reputation model consists of three stages: , Feature Reputation: the reputation of every feature is calculated based on the frequencies of positive and negative opinions about the features and its sub features. , Features’ Impact: feature impact is used to give a different weight for every feature depending on the number of opinions available in users’ reviews about this feature. , Product Reputation: the final product reputation is the aggregation of features’ reputations.
In the following sections we will describe them in details. 3.2
The basic idea of the proposed model is to generate the reputation of a product based on the reputation of the product’s features. The reputation of each feature is generated based on the opinion orientation and strength of its sub features. For a feature f i , the reputation of f i will be the aggregation of the positive and negative opinions weights for all of its sub-features f j , where ( f i , f j ) $ L as mentioned in Definition 1. This section will discuss how to derive feature reputation based on sub features’ opinion information.
In this part we suggest a formula to give more weights for frequent negative
opinions about a specific feature. By “frequent”, we mean that the negative opinion about
a feature has occurred in many reviews. Frequent negative opinions may indicate a
real drawback in the product, where there is a larger probability that a customer will
have the same problem if he buys this product. Thus, when more reviews share a
negative opinion about the same feature, the risk of facing the same problem becomes
higher. These kinds of problems must appear in the reputation model in order to
reflect a true evaluation for the product in use, and to draw user’s attention so that he
can look for more details and have a rational decision about buying the product.
Therefore, we suggest giving these types of negative opinions more weight to draw
the user’s attention to problems in the products. If we have some negative opinions
about different sub-features f j for the feature f i , we do not consider them as frequent
for the feature f i . For example, if we have negative opinions about a mobile phone
camera as follows “Low video recording quality”, “The flash light give a very harsh
light”, and “No zoom available”, these negative opinions about the camera cannot be
considered frequent in terms of “camera” because they are about different
subfeatures (video recording, flash light, and zoom) of the generic feature “camera”.
Equation (
Rneg j 3 N j # - 11 sinjeg 5 i#1 2 i 5 ! 41 !
0 41..
/
(
R njeg : is the number of reviews that contains negative opinions about the feature sinjeg : is the strength of negative opinion in review (i) about the feature (j). ! : is a positive integer that is used to define the interval of weight increment for the subsequent opinions, where
Interval # 1 !
The value of ! is subject to change, higher ! values will furnish higher feature
reputation values, and that is because the Interval value in (
N j # s1njeg 5 (s2nejg 5 0.33) 5 ... 5 (s nRenjgeg j 5 R njeg 4 1 3 )
For a feature f i which has sub features {f1, f2, …, fk}, Equation (
WNi # 112 -j#1 N j ../ 5 Ni
WN i : is the weight of all negative opinions about a generic feature f i in the hierarchy FH.
k : is the number of sub-features of feature f i .
Ni : represents the weight of negative opinions about the generic feature f i itself
and not about one of its sub-features. It is calculated using Equation (
For the positive opinions, we propose to calculate the positive weight for a feature
f j by adding opinion strength values si given in Equation (
R jpos - s pos
ij
i#1
3 k 0
WPi # 11 - Pj .. 5 Pi
2 j#1 /
(
WPi : is the weight of all positive opinions about a generic feature f i in the hierarchy FH.
Pi : represents the weight of positive opinions about the generic feature f i itself and
not about one of its sub-features. It is calculated using Equation (
Calculating Feature Reputation
In this paper, we propose to calculate the reputation of a feature based on its
overall positive and negative weights as showed in Equation (
FREPi #
WPi WPi 5 WNi 8100
An example is given in Table 1 to demonstrate the proposed method. In the table,
for simplicity, each feature listed on the left most column has three sub features;
NOf1, NOf2, and NOf3 are the number of reviews which contain negative opinions to
the corresponding sub features; N1, N2, and N3 are the negative weight of
corresponding sub features; NOFi and WNOFi are the number of reviews containing negative
opinions about feature Fi and its negative weight respectively. It also shows the total
number of positive reviews (PO), the total number of negative reviews (NO), overall
weight for positive (WPi) and negative (WNi) opinions, and the aggregation (FREPi)
using the proposed method and the (PPR) which is the percentage of positive reviews
among all reviews without considering the strength of opinion and it can be calculated
using Equation (
The example shows the detailed calculations for both positive and negative opinions weights. In the last two columns we can see the differences between the feature reputation value using our method (FREPi) and the simple average method (AVG). Our method results in lower reputation in all cases, this is logical as we give more weight for negative opinions. For example, the total number of negative opinions (NO) for both F2, and F7 are the same which is equal to 21. Nevertheless, the overall weight for negative opinions (N2) for F2 is 63.33 and for F7 is 53.00, which is totally different. This difference is due to; first, the large frequency for the second subfeature (NOf2 # 11) for F2, second, higher values for opinions’ strength (which was not provided in the table). Fig. 1 shows the relation between (FREPi) and (AVG) where the difference between the two values is the most when the percentage of negative opinions to positive ones is higher. And this complies with our purpose of giving negative opinions more weight. 3.3
Feature Impact
Depending on the fact that product features are not equally important to customers,
we will calculate feature’s impact, which is a value that reflects a feature’s influence
between users. Some of the features are essential for a product to work, but they do
not inspire customers to buy the product, as they become consistent over time. On the
other hand, some hot features, that are improved continually or new features have
high influence on customers to be more interested in the product. Thus, these features
should have more impact on the product overall reputation. Features impact will be
used to give different weights for every feature in the final product reputation
aggregation formula. We suggest that features that frequently occurred in users’ reviews
have more impact than other features. Let Mj denote the number of reviews that have
opinion about this feature, whether positive or negative, the impact of a feature f j ,
denoted as Ij, is defined in Equation (
M j # R njeg 5 R jpos
I j #
M j
M Max
(
All feature impacts will be given values between 0 and 1; 1 for the feature that was mentioned the most in the users’ reviews, and thus has the most influence on users. 80.00 60.00 40.00 20.00 0.00 FREP PPR F1
F2
F3
F4
F5
F6
F7
F8
Many opinions in customers’ reviews targeted the product itself rather than mentioning a specific feature in the product, these opinions are also considered in our model. We propose to calculate the product reputation by integrating the reputation calculated based on the reviews which are directly about the product and the reputations of the product’s direct features.
Assume that a product has h direct sub features, FREPj and Ij are the reputation and
the impact of each sub feature, respectively. Let GOP denote the product reputation
calculated using Equation (
PR # j#1
h
- (FREPj 8 I j ) 5 GOP
h
- I j 5 1
j#1
(
As we mentioned before, our model reveals a final reputation lower than the average method. One of the strength points in our model is data representation, as we are able to provide details for customers about every specific feature. Fig. 2 shows the reputation of every feature, which can be more inspiring for customers than the one value reputation representation. Furthermore, more detailed information can also be provided as showed in the example in Fig. 3. For example, if the user is interested in a specific feature and he wants to see more about it, a second level will show the details of negative opinions about sub-features and the frequency of each one. 100.00 80.00 FREP
PPR
12 8 4 0
Resolution
Sub Features
In this paper we have presented a new reputation model for products, our model used text reviews rather than users’ ratings. We extracted opinions about hierarchy of features and calculated the frequencies for positive and negative opinions assuming that frequent negative opinions about features and sub-features should get more weight in the reputation calculation, as they indicate a problem in a product a customer may face if they buy it. In Addition, we calculated the impact of features, hence certain features in some products are more inspiring for users, and therefore they are more important in the reputation model. Our model integrates the strength of opinions and provides summary about users’ opinions for customers rather than representing reputation as a number of stars. For future work, the reputation model may be modified to consider age and validity of reviews, and also detect malicious users’ reviews which aim to sabotage the reputation of a product.