One of the most recent opinion mining research directions falls in the extraction of polarities referring to specific entities (called “aspects”) contained in the analyzed texts. The detection of such aspects may be very critical especially when the domain which documents belong to is unknown. Indeed, while in some contexts it is possible to train domain-specific models for improving the effectiveness of aspects extraction algorithms, in others the most suitable solution is to apply unsupervised techniques by making the used algorithm independent from the domain. In this work, we implemented different unsupervised solutions into an aspect-based opinion mining system. Such solutions are based on the use of semantic resources for performing the extraction of aspects from texts. The algorithms have been tested on benchmarks provided by the SemEval campaign and have been compared with the results obtained by domain-adapted techniques.
Opinion Mining is a natural language processing (NLP) task that aims to classify
documents according to their opinion (polarity) on a given subject [
Let us consider the following example:
Yesterday, I bought a new smartphone.
The quality of the display is very good, but the buttery lasts too little.
In the sentence above, we may identify three aspects: “smartphone”, “display”, and “battery”. Each aspect has a different opinion associated with it, in particular: – “display” ! “very good” – “battery” ! “too little” – “smarthphone” ! no explicit opinions, therefore its polarity can be inferred by averaging the opinions associated with all other aspects.
Another important consideration related to this example is that it is easy to detect which is the domain of the analyzed text. In this case, by assuming to have a training set, it should be possible to build domain-specific models for supporting the extraction of the aspects. However, this strategy is in contrast with two considerations coming from real-world scenarios: (i) it is difficult to find annotated dataset related to all possible domains, and (ii) in the same document, it is possible to have sentences belonging to many domains by making the adoption of a domain-specific models not feasible.
To overcome these issues, we propose a set of unsupervised approaches based on natural language processing approaches that do not rely to any domain-specific information. The goal of this study is to provide techniques that are able to reach an effectiveness comparable with supervised systems.
The paper is structured as follows. In Section 2, we provide an overview of the opinion mining field with a focus on aspects extraction approaches. Section 3 presents the natural language processing layer built for supporting the approaches described in Sections 4 and 5. Section 6 discusses the performance of each algorithm; while, Section 7 concludes the paper. 2
The topic of opinion mining has been studied extensively in the literature [
All the approaches presented so far operate at the document-level[
The growth of online product reviews was the perfect floor for using opinion mining
techniques in marketing activities. The issue of detecting the different opinions
concerning the same product expressed in the same review emerged as a challenging problem.
Such a task has been faced by introducing aspect extraction approaches aiming to
extract, from each sentence, which is the aspect the opinion refers to. In the literature,
many approaches have been proposed: conditional random fields (CRF) [
At the same time, the social dimension of the Web opens up the opportunity to
combine computer science and social sciences to better recognize, interpret, and process
opinions and sentiments expressed over it. Such multi-disciplinary approach has been
called sentic computing [
Above, we mentioned approaches that do not consider the domain analyzed documents belong to. The use of domain adaptation demonstrated that opinion classification is highly sensitive to the domain from which the training data is extracted. A classifier trained using opinionated documents from one domain often performs poorly when applied or tested on opinionated documents from another domain. The reason is that using the same words and even the same language constructs can carry different opinions, depending on the domain.
The classic scenario is when the same word in one domain may have positive
connotations, but in another domain may have negative one; therefore, domain adaptation
is needed. In the literature, different approaches related to the Multi-Domain sentiment
analysis have been proposed. Briefly, two main categories may be identified: (i) the
transfer of learned classifiers across different domains [
While on one side such approaches demonstrated their effectiveness in working in a multi-domain environment, on the other one, they suffer by the limitation in abstracting their usage within any domain different from the ones used for building the model. 3
A number of different approaches has been tested in order to accomplish aspect extraction task. Each one uses different functionalities offered by the Stanford NLP Library but every technique is characterized by a common preliminary phase.
First of all, WordNet3 [
The entire text is then fed to the co-reference resolution module to compute pronoun references which are stored in an index-reference map.
The next operation consists in detecting which word expresses polarity within each
sentence. To achieve this task SenticNet4 [
While SenticNet expresses polarity values in the continuous range from -1 to 1, the other two resources been normalized: the General Inquirer words have positive values of polarity if they belong to the “Positiv” class while negative if they belong to “Negativ” one, zero otherwise, similarly, MPQA “polarity” labels are used to infer a numerical values. Only words with a non-zero polarity value in at least one resource are considered as opinion words (e.g. word “third” is not present in MPQA and SenticNet and has a 0 value according to General Inquirer, consequently, it is not a valid opinion word; on the other hand, word “huge” has a positive 0.069 value according to SenticNet, a negative value in MPQA and 0 value according to General Inquirer, therefore, it is a possible opinion word even if lexicons express contrasting values). Every noun (single 3 https://wordnet.princeton.edu/ 4 http://sentic.net/ 5 http://www.wjh.harvard.edu/ inquirer/spreadsheet guide.htm 6 http://mpqa.cs.pitt.edu/corpora/mpqa corpus/ or complex) is considered an aspect as long as it’s connected to at least one opinion and it’s not in the stopword list. This list has been created starting from the “Onix” text retrieval engine stopwords list7 and it contains words without a specific meaning (such as “thing”) and special characters.
Opinions associated with pronouns are connected to the aspect they are referring to; instead, if pronouns reference can’t be resolved, they are both discarded.
The main task of the system is, then, represented by connecting opinions with possible aspects. Two different approaches have been tested with a few variants. The first one relies on the syntactic tree while the second one is based on grammar dependencies.
The sentence “I enjoyed the screen resolution, it’s amazing for such a cheap laptop.” has been used to underline differences in connection techniques.
The preliminary phase merges words “screen” and “resolution” into a single word “Screenresolution” because they are consecutive nouns. Co-reference resolution module extracts a relation between “it” and “Screenresolution”. This relation is stored so that every possible opinion that would be connected to “it” will be connected to “Screenresolution” instead. Figure 1 shows the syntax tree while Figure 2 represents the grammar relation graph generated starting from the example sentence. Both structures have been computed using Stanford NLP modules (“parse”, “depparse”). These typologies of approaches are based on syntax tree structures created by Stanford NLP library. In order to explain how the algorithms connect opinion with aspects a few definition are needed: – “Intermediate node”: tree node which is not a leaf; – “Sentence node”: intermediate node labeled with one of the following: 7 The used stopwords list is available at http://www.lextek.com/manuals/onix/stopwords1.html
S - Sentence SBAR - Clause introduced by a (possibly empty) subordinating conjunction SBARQ - Direct question introduced by a wh-word or a wh phrase SQ - Inverted yes/no question or main clause of a wh-question SINV - Inverted declarative sentence PRN - Parenthetical
FRAG - Fragment – “Noun Phrase node”: intermediate node labeled with NP tag
Approaches differ in rules adopted for associating intermediate nodes that define how aspects are extracted by starting from their child nodes.
Approach 1.1 Each polarized adjective is connected with each possible aspect in the same sentence.
Figure 3 shows she propagation of aspects and opinion in the tree with red lines representing propagation of aspects, blue lines for opinions and purple ones when both are propagated to the upper level.
Within the sub-sentence “I enjoyed the Screenresolution” only aspects are detected, consequently, once the Sentence Level node is reached, no connection is done. On the other hand, both polarized adjectives “cheap” and “amazing” are propagated until they reach the top sentence node together with “it” and “laptop” aspects, then, they are connected with each other.
The results are shown in Figure 4.
Approach 1.2 Each polarized adjective is connected to each possible aspect within the same sentence or noun phrase.
Influences of this variant are underlined in Figure 5 with the same notation.
Even if extracted aspects are the same, the opinion “cheap” is associated only with the name “laptop” as shown in Figure 6.
Approach 1.3 When both aspects set and opinion words set related to a node are not empty, each opinion word is connected to the related aspect and removed from the opinion words set. Opinion words and possible aspects are removed anyway in sentence nodes.
Figure 7 shows the effects of the association rules mentioned above.
Once again, even if aspects extracted are the same, the connections are different (Figure 8).
The other set of approaches proposed in this paper exploits grammar dependencies instead of syntax tree to detect aspect-opinion associations. Grammar dependencies computed by Stanford NLP modules (which are represented by the labeled graph in picture [1.2]) can be expressed by triples: fRelationtype; Governor; Dependantg. One of the most important difference with the previous methodology is represented by the possibility of detecting opinion expressed by word that are not adjectives (such as verbs that are considered by approaches 2.2 and 2.3). Different approaches have been tested in order to detect which kind of triple can be interpreted as a connection between an opinion word and a possible aspect.
Rule 2: Each nominal subject (nsubj) relation expresses a connection between an aspect and an opinion word if and only if the governor is a polarized opinion and the dependant is a possible aspect.
Figure 9 underlines aspect-opinion connections mined through the process. Resulting aspects are shown in Figure 10.
Approach 2.3 The Rules “1” and “3” are both used, while Rule “2” is changed as follows:
Rule 2.1: Each nominal subject (nsubj) relation expresses a connection between an aspect and an opinion word if and only if the governor is a polarized word and the dependant is a possible aspect.
Figure 13 shows results of the modification of the rules. Even if the relation between “enjoyed” and “I” is detected, “I” is not considered as a valid aspect since it’s has an unresolved reference in the current context.
Results are the same as the previous example (Figure 14). 6
Each approach has been tested on two datasets provided by the Task 12 of SemEval 2015 evaluation campaign, namely “Laptop” and “Restaurant”. To evaluate results a notion of correctness has to be introduced: if the extracted aspects is equal, contained or contains the correct one, it’s considered to be correct (for example if the extracted aspect is “screen”, while the annotated one is “screen of the computer” or vice versa, the result of the system is considered to correct).
Tables 1 and 2 shows the number of “True Positive”, “False Positive”, and “False Negative” computed on mentioned datasets. The rationale behind this choice is to support the error analysis provided later and for showing strong and weak points of each aspect.
To evaluate performance on the “Restaurants” dataset, “null” aspect has not been considered in false negatives count.
Restaurants True Positives False Positives False Negatives complex expressions “tech support would not fix the problem unless I bought your plan for $150 plus”.
Other issues are correlated to the association algorithm. Figures 16 and 17 show error categories in approaches 1.3 and 2.1 respectively, always computed on the same 100 sentences of the “Laptop” dataset.
Even if the syntax-tree-based approach tends to produce a significant number of true positives, relationships are often imprecise. A relevant example is represented by the sentence “I was extremely happy with the OS itself.” in the “Laptop” dataset. Approach 1.3 connects the opinion adjective “happy” with the potential aspect “OS”, correctly recognized as an aspect in the sentence, while approach 2.1 does not detect such a relation because the word “happy” is connected to “I” which is not a potential aspect.
A relevant part of false positives are generated by approaches that are not able to discriminate aspects from the entity itself. In facts, almost half of them consists in associations between opinion words and the entity reviewed that are correct. However, they must not be considered during the aspect extraction task (for example the aspect “laptop” in the example sentence should not be considered according to the definition of aspect). 7
In this paper, we presented a set of unsupervised approaches for aspect-based sentiment analysis. Such approaches have been tested on two SemEval benchmarks: the “Laptop” and “Restaurant” datasets used in the Task 12 of SemEval 2015 evaluation campaign. Results demonstrated how without using learning techniques the results can be comparable with the ones obtained by trained systems. Future work includes refinement of the proposed approaches in order to make them suitable for real-world implementation. 2. Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM (2004) 168–177