In recent years, the Web and social media are growing exponentially. We are provided with documents which have opinions expressed about several topics. This constitute a rich source for Natural Language Processing tasks, in particular, Sentiment Analysis. In this work, we aim at constructing a sentiment dictionary based on words obtained from web pages related to a specific domain. To do so, we correlate candidate opinion words, seed words and domain using AcroDefMI3 and TrueSkill methods. This dictionarybased approach is compared to the SentiWordNet lexical resource. Experimental results show suitability of our approach for multiple domains and infrequent opinion words.
In recent years, the Web and social media
are growing exponentially, this constitute a rich
source for Sentiment Analysis tasks. Companies
are increasingly using the content in these media
to make better decisions
On the other hand, dictionary based approaches
are unsupervised in nature. In general, these
methods assume that positive (negative) adjectives
appear more frequently near a positive (negative)
seed word
SO(phrase) = log hits(phrase NEAR excellent) · hits(poor) hits(phrase NEAR poor) · hits(excellent)
(1) In this work, words used to express opinions are learned. To do so, positive and negative seed words (e.g. good, excellent, bad) are used to extract adjectives near seed words. To correlate candidate words, seed words and domain, AcroDefMI3 and TrueSkill methods are proposed. Experimental results show suitability of our proposal. Several domains (e.g movies, agricultural) were used to compare our approach to SentiWordNet.
The paper is organized as follows. The Methodology is presented in Section 2. Experimental setup is described in Section 3. In Section 4, we present and discuss the obtained results. Concluding remarks are presented in Section 5. 2
Methodology The proposed process is depicted in Figure 1. The steps are summarized in the following steps: 1. A corpora for a specific domain, containing positive and negative opinions is acquired from the Web. 2. Each document is pre-processed to get text,
remove HTML tags and scripts. 3. Opinion adjectives and nouns are extracted using POS-Tagging and the Window Size algorithm. 4. The correlation score of a given word with a seed word and domain is computed using AcroDefMI3 and TrueSkill. lexicons are inferred based on these correlation scores that identify semantic orientation for each extracted word. High correlation score words are selected.
We perform experiments over two domains:
Agricultural domain (opinions extracted from Twitter)
and a Movie domain1 (data set introduced in
Corpus Acquisition Some words can express neutral, positive or negative opinion in specific domain such as: Domain, Seed Word Pre-processing
Text
Word Extraction Nouns, Adjectives
Word Selection
Score
POS-Tag Window Size(N)
MI3
TrueSkill
SentiWordNet Dictionary+
Dictionary
These examples show that a given word, for
instance scientific, can be highly correlated to a
particular domain
Lexicons are built using selected words from web page corpus. Web pages are retrieved using Bing search engine. Queries used to retrieve this web pages combine seed words and domain keywords. We have positive and negative seed words, P = {good, nice, excellent, positive, fortunate, correct, superior} , Q = {bad, nasty, poor, negative, unfortunate, wrong, inferior}, respectively.
A positive (negative) seed word ensure a positive (negative) web page about a query domain, due to all opposite seed words are excluded from that query. For example, the following query can be used for retrieving positive pages: query+ = +opinion + review + gmo + good bad nasty poor negative unfortunate wrong inferior Neutral ! I attend scientific conferences. Thus, we have positive and negative web pages denoted by corpus+, corpus respectively. Each Positive ! The list shows the scientific discoveries. corpus is related to a seed word and a given doPositive ! He made a good scientific discovery. main. In the next section we will extract words 1http://www.cs.cornell.edu/People/pabo/movie-review- near seed words for each web page corpus using data/ POS-Tagging and the Window Size algorithm. 2.2
Word extraction
Opinion words near a seed word can have the same
polarity
Algorithm 1 The Window Size Algorithm Require: seed words, corpus, K Ensure: opinion words 1: words TreeTagger to each corpus 2: words filter adjectives and nouns 3: for index= 0 until total of words do 4: if words{index} in seed word then 5: for k = 1 until K do 6: left word words[index - k] 7: right word words[index + k] 8: opinion words
left word and right word In Figure 2 adjectives (JJ) and nouns (NNS) are retrieved using TreeTagger. The good word is a positive seed word and its nearest adjective is safe given k = 1 distance. Likewise, scientific and studies words are retrieved with distance k = 2.
In addition, safe is a positive opinion word
candidate because it occurred near a positive seed word
(good). In this sense, we can have a set of
opinion words (positive and negative), that can be
candidates to include into the resulting lexicon. To
get the correlation score of each extracted word
given a seed word, two measures are employed:
AcroDefMI3 and TrueSkill which are described
in the next section.
as gmo, can be used to express a domain
opinion. Hence, we need to measure the correlation
of a given extracted word with domain and seed
word to build a lexicon. In order to get
candidate opinion words we propose to use the
statistical measure AcroDefMI3 (equation 2)
The AcroDefMI3 measure takes each word extracted using the Window Size algorithm and computes the following equation 2, which is based on web mining.
The total web page results, based on queries that combine candidate words, seed words and domain keywords, are used in the AcroDefMI3 measure to get the correlation score for each extracted word.
AcroDefMI3 = 0 (nb(sw word AND domain)+ 1
nb(word sw AND domain))3 C log BB@ nb(sw AND domain) CA · nb(word AND domain)
(2) where sw is a seed word, nb(x) function is the number of total result pages, x is the query used to retrieve pages in the search engine, and word is the word extracted using the Window Size algorithm. This process is detailed in Algorithm 2.
Algorithm 2 Word selection algorithm using AcroDefMI3 Require: corpus, seed words = P, keywords of
domain Ensure: correlation score values for each word 1: for each corpus do 2: words+ = window size(corpus+, P ) 3: for word in words+ do 4: given each seed word and keywords of
domain compute correlation score: 5: score max(AcroDefMI3) 2.3 As seen in our previous example (Figure 2), the scientific word was retrieved using window size distance = 2. However, specific words, such Scientific studies have frequently found that GMO’s are safe to eat and even good.
JJ NNS VHP RB VVN IN NNS VBP JJ TO VV CC RB JJ window size = 1 window size = 2 words. The process is detailed in Figure 3, where
S = {s1,1, s1,2, , s1,n} and S = {s2,1, s2,2, , s2,n}, s are learning values for each word in positive and negative web page respectively. p is the learning performance for each word, t is the sum of total performance for each word in corpus.
As T rueSkill learns s according its match outcome, we set a high punctuation for corpus+, and less punctuation for corpus . Therefore, we have d = t1 t2. Due to difference (d) is important, we set t1 = 1 to a positive corpus and t2 = 2 to a negative corpus, where 1 denotes first. This process is detailed in Algorithm 3.
The following example shows how TrueSkill measures two collected web pages: corpus+= By the way a New York Times excellent job · · · bioengineered food · · · . corpus = Roundup Ready cotton · · · solution · · · at any economic advantage.
· · · wrong
Require: corpus, seed words(P, Q) Ensure: correlation score values for each word 1: k = 10 number of match for each corpus. 2: for each corpus do 3: words+ = window size(corpus+, P )
5: words = window size(corpus , Q)
score: 7:
score
T rueSkill(words+, words , t = [1, 2])
word+ word
Si 22, 738 0, 001
Si+1
Where: Si denotes current correlation score for each word, and Si+1, the updated value after matching pages (positive against negative page), bioengineered is a word near excellent, a seed word 2 P , and economic is near wrong, seed word 2 Q when the Window Size algorithm has distance k = 1. Thus, when the same corpus+ has a match with other corpus : corpus = Various studies · · · poor agricultural income · · · .
word+ word
Si 22, 738 0, 108
Si+1
It is worth noting that agricultural becomes a more negative word than economic because its value decreases more after the match using the same positive word: bioengineered. On one hand, if a word is often found in a corpus its value tends to decrease. On the other hand, if it is in a corpus+ its value will increase. If the word is found in both corpus it tends to be constant. In the next section, experiments results are showed. 3
Experiments In order to validate our approach experiments over two data sets were conducted. The polarity of each opinion from domains (Agricultural tweets and Movie reviews) is predicted using the inferred lexicons, AcroDefMI3 and TrueSkill measures. Precision, recall and f-score were measured in order to compare to the SentiWordNet approach. Data sets used are described in the next section. 3.1
The domains keywords used in queries were:
Agriculture domain = {gmo, agricultural
biotechnology, biotechnology for agriculture}, and
Movie domain = {cinema, film, movie}. In order
to test the agricultural domain, tweets using these
keywords were collected and manually classified.
There were 50 positive and 61 negative tweets.
The Movie domain 4 is based on
A simple classification procedure was used. In order to do so, the number of positive and negative words in each tweet or review is computed using the inferred lexicons. If the difference is greater than zero then it is classified as positive, otherwise is negative. The following kind of lexicons were used to sentiment classification: • M I3: seed words + W S with AcroDefMI3. • T S: seed words + W S with T rueSkill. • SW N : SentiWordNet. where W S denotes words extracted with window size. Finally, the number of web pages retrieved during the corpus acquisition for each seed word was k = 20.
In the next, we show word distributions for each type of lexicon. 3.2
superior
good positive fortunate excellent correct nice poor negative wrong
bad unfortunate nasty inferior Using web pages number k = 20, a high number of low frequency adjectives are retrieved as shown in Figure 5a. To get a word near a seed word with window size= 1, the maximum distance allowed is 10 words per window size. SentiWordNet5 is a lexical resource for opinion mining. It assigns to each synset of WordNet three sentiment scores, positive, negative and neutral. We compute differences between positive and negative scores. If the result is greater than zero then the polarity of the word is positive, otherwise negative. SentiWordNet assigns a different score for each word according its context. As context is not considered, higher positive and negative word scores are obtained. Finally, SentiWordNet comprises 21479 adjectives and 117798 nouns. top 10 new words ordered by their correlation score value. In order to validate the algorithms we calculate recall, precision and f-score. Figures 7, 6 show the recall, precision and fscore using each word type(noun, adjectives), and the results using MI3, SentiWordNet and TrueSkill.
Discussion of the results When the inferred lexicon for the Movie domain is considered, TrueSkill performs better (Recall, Precision and F-Score) than SentiWordNet and AcroDefMI3 for positive reviews using adjectives and nouns. When negative reviews are con
Domain Agricultural chocolaty glyphosate phosphonic carfentrazone sporogene
kalu protato adeed phthalates genotoxicity
Movie configurable
updated readymade
nature directorial spendidly cartoonish
mic showreel coverup AcroDefMI3 and TrueSkill methods, which are not in SentiW ordN et
Noun
Positive
Negative Agricultural
Movie SentiWordNet performs better than AcroDefM I3 and TrueSkill. This is due to the agricultural domain was collected from Twitter. Tweets are short texts that usually have more seed words and common words as shown in Table 1. The agricultural domain has frequent seed words. 5
Conclusion Most of the dictionary-based algorithms for sentiment analysis consider word frequency in documents. However, this research has shown that collected corpus words with low frequencies can be useful to set polarities. Thus, We propose a dictionary-based algorithm for sentiment analysis that uses AcroDefM I3 and TrueSkill methods so as to compute correlation word scores that allow us to differentiate between positive and negative polarities. This is particularly useful for low frequency words obtained from corpus. In addition, by using the Window Size Algorithm, it is possible to obtain new adjectives entries in both agricultural and movie domains when compared to SentiWordNet.
Acknowledgments This work has been supported and financed by FONDECYT.