The focus of this paper is describing a natural language processing methodology for identifying opinion diversity expressed within text. We achieve this by building a domain-driven opinion vocabulary, in order to be able to identify domain specific words and expressions. As a use case scenario, we consider Twitter comments related to movies, and try to capture opinion diversity by employing an opinion vocabulary, which we generate based on a corpus of IMDb movie reviews.
In this work we are describing a natural language processing methodology for discovering the diversity of opinions expressed within text, which we deem to be an essential step to expressing and presenting diverse information on the Web. In this context, we consider an opinion as a subjective expression of sentiments, appraisals or feelings, and opinion words as a set of keywords/phrases used in expressing an opinion. As such, the orientation of an opinion word indicates whether the opinion expressed is positive, negative or neutral, while the totality of opinion words forms an opinion vocabulary. While opinion words can be analyzed in their base form (describe and convey the opinion directly) and comparative form (convey the opinion indirectly, by comparison with other entities), this research focuses only on base type opinion words.
In the context of the ever expanding world of social media and user generated content, instant access, world-wide coverage and diversity of perspective are the norm of the information flow. As an application of our approach, we propose to study the movie domain. There is a strong user interest in watching, tracking and discussing movies, generating highly diverse opinion content. Movies are subject to a variety of classifications, expanding the field of analysis. Moreover, the lifespan of a movie topic is longer than for usual topics, thus introducing a temporal dimension that can be further explored. Nowadays, accessing and assessing the public opinion has taken on a new form. Social networking encourages the exchange of information and sharing of opinions between individuals, friends and communities. Therefore, in our case study we directly address movie comments, as posted on Twitter, a popular social networking and microblogging website, and aim at identifying the diversity of opinions expressed in tweets related to movies. We determine a variety of polarized opinion words about a certain movie, and use these word frequency counts to obtain an overall aggregated opinion about the movie. Moreover, we can observe variations in opinions over time, related to a certain movie, by comparing the word frequency counts obtained from tweets belonging to a time interval (e.g. an hour, day, week).
The paper is structured as follows: in Section 2 we describe our algorithm for constructing a domain-driven opinion vocabulary, while Section 3 presents the Twitter movie comments use-case. The last section of the paper is dedicated to conclusions and future work.
We start from the idea that expressing opinions is dependent on
the topic’s context and we focus on the role of adjectives as
opinion indicators; in the future we plan to broaden this line of
work by including verbs and adverbs. The starting point is
represented by a domain-specific corpus, from which we
determine a small number of seed opinion words that we further
extend, thus forming a domain-driven opinion vocabulary.
There are three main approaches to constructing an opinion
vocabulary: manual, dictionary based and corpus based. The
manual approach is not really in line with our work, as we are
considering automatic, scalable approaches. The dictionary based
approach provides a simple and efficient way of obtaining a good
vocabulary. SentiWordNet [
V. Hatzivassiloglou et al [6] showed the relevance of using connectives in gathering information about the orientation of conjoined adjectives. They emphasized that conjoined adjectives are of the same orientation, for most connectives, but reversing the relationship. The connectives are conjunctions used to join one or more adjectives together. In our algorithm we used a subset of the possible conjunctions (and, or, nor, but, yet), that cover many common syntactic patterns and are easier to correlate with the adjectives that they connect.
Other lines of research, like S.-M. Kim and E. Hovy [7] try to identify opinion expressions together with their opinion holder starting from a word seed list and use the WordNet synsets to determine the strength of the opinion orientation for the identified opinion words. M Gamon and A. Aue [5] extend the Turney-style [9] approach of assigning opinion orientation to the determined candidate words, working under the assumptions that in the opinion domain, opinion terms with similar orientation tend to cooccur, while terms with opposite orientation do not tend to cooccur at sentence level.
V Jijkoun et al [10] propose a different style of approach, by starting from an existing lexicon (clues) and focusing it. They perform a dependency parsing on a set of relevant documents, resulting in triplets (clue word, syntactic context, target of sentiment) that represent the domain specific lexicon. H. Kanayama and T. Nasukawa [11] apply the idea of context coherency (same polarity tend to appear successively) to the Japanese language. Starting from a list of polar atoms (minimum syntactic structure specifying polarity in a predicative expression), they determine a list of domain specific words using the overall density and precision of coherency in the corpus. Sinno Jialin Pan et al [12] propose a cross-domain classification method. Starting from a set of labeled data in a source domain and determining domain-independent words (features) that occur both in the source and the target domain, they construct a feature bipartite graph that models the relationship between domain-specific words and independent words. To obtain the domain specific words they use an adapted spectral clustering algorithm on the feature graph Based on these premises, we propose a method to construct an opinion vocabulary by expanding a small set of initial (seed) words with the aid of connectives. The method consists of four steps, as follows: 1. Given a positive word seed list and a negative word seed list and making use of WordNet’s synsets, we expand the initial seed lists based on the synonymity / antonymy relations.
The initial words will be assigned a score of 1 for positive words and -1 for negative words, respectively. We compute the orientation score for each newly found word by recursively processing the synsets for each seed word. A word can be found in synsets corresponding to different seed words, either in a synonymity or antonymy relations. Another factor we take into account is the distance between the seed word and the currently processed word, as provided by the WordNet hierarchy. From these two considerations, a more formal way to compute the score of a word (sw) to be added to the seed list is: ( ) ( ( )) where
{ and o is a seed word, while f is a parameter for which we empirically assigned values between 0 and 1 (in our current implementation f = 0.9); in our future work we plan to determine its value by optimization.
The result of this step is an expanded seed word list together with their orientation score. 2. From a corpus of documents, we parse and extract all adjectives and conjunctions, constructing a set of relationships between the determined words. There can be two types of relationships, indicating if two or more words have the same context orientation (words connected by and, or, nor) or opposite orientation (words connected by but, yet). We will refer to them in the following algorithms as ContextSame and ContextOpposite relations, respectively.
1. G = ({}, {}) 2. foreach document d in corpus 3. foreach sentence s in d 4. parseTree = GetParseTree(s) 5. {w,c} = RetrieveWordsAndConjunctions(parseTree) 6. ConstructRelationGraph(G, {w, c}) 7. HandleNegation(G, s)
Based on the determined relations, we can then construct a relationship graph G(W, E), where W={set of determined adjectives} and E={wiwj, where wi, wj from W if there is a determined relationship between wi and wj, each edge having a positive weight for the ContextSame relationship and a negative weight for the ContextOpposite relationship}.
In what follows, we describe the algorithm for building the relationship graph G (see Figure 1).
We used a maximum entropy parser1 to retrieve a sentence’s parse
tree that we then analyze in the RetrieveWordsAndConjunctions
procedure. We construct an adjective stack w and a conjunction
stack c by extracting the relevant nodes according to their
part-ofspeech tags and group them together based on the common parent
node between the adjective nodes and the conjunctions nodes. In
the ConstructRelationGraph, we will add the nodes for each
newly found adjective and add new edges to the relationship
1 http://sharpnlp.codeplex.com/
graph G according to each conjunction’s behavior. Each edge has
an associated weight with values between 0 and 1, determined by
optimization. We handle the presence of negation in the sentence
by reversing the type of the relation, if a negation is detected. For
example, considering the sentence “Some of the characters are
fictitious, but not grotesque”, the initial relation between fictitious
and grotesque would be a ContextOpposite relationship, but the
presence of the negation is converting it to a ContextSame
relationship. We depict another example visually, in Figure 2.
3. The third step implies cleaning the resulting set of words and
relationship graph by removing stop words and self-reference
relations. Consider the example “The movie has a good casting
and a good plot”. The algorithm detects a ContextSame
relationship between the adjective good and itself. Since there is
no useful context information we can use, we do not want them to
influence the results of the scoring done in the next step.
4. In the fourth step, we determine the orientation of the words
extracted from the corpus by applying an algorithm on the
relationship graph obtained in the previous steps, which was
inspired by the well-known PageRank algorithm [
1. InitializeScoreVectors(sPos(W), sNeg(W)) 2. do { 3. foreach word wi in W 4. wi
foreach relation relij in relationship graph G that contains if relij is a ContextSame relation sPos(wi) += weigth(relij) * prevSPos(wj) sNeg (wi) += weigth(relij) * prevSNeg(wj) else if relij is a ContextOpposite relation sPos(wi) += weigth(relij) * prevSNeg(wj) sNeg (wi) += weigth(relij) * prevSPos(wj)
NormalizeScores(sPos(wi), sNeg(wi)) 12. } while more than 1% of the words wi in W change orientation
We initialize the score vectors based on the orientation scores of the expanded seed word list (see step 1). We will assign the corresponding positivity or negativity score swj for each adjective w found in the seed list. For the opposite score we assign a very small value (ε), in order to allow for meaningful values when computing the score for ContextOpposite relations.
A ContextSame relation enforces the existing positive and negative scoring of wi proportionally with the scoring of wj. A ContextOpposite enforces the negativity score of wj with respect to the positivity of wi, and the positivity score of wj with respect to the negativity score of wi.
Concerning the movie domain, research was done in classifying
movie reviews by overall document sentiment [8], but there are
few lines of research connecting the movie domain with social
media. Sitaram Asur and Bernardo A. Huberman [
Our aim is to see how well a domain specific vocabulary constructed from movie reviews performs when applied to analyzing tweets. We used a document corpus of 27,886 IMDb (Internet Movie Database) movie reviews 3 and constructed a movie domain specific vocabulary according to the approach presented in Section 2. We retrieved 9,318 words, from which 4,925 have a negative orientation and 4393 have a positive orientation. Table 1 shows a few examples of positive and negative adjectives extracted from the movie review corpus. For our tests, we crawled 220,387 tweets, using the Twitter Search API6, over a two month interval, keyed on 84 movies, spanning different genres and release dates. As search keywords we used the movie name and the movie tag, in order to increase the relevance of the results. We used a simple tokenizer to split the text of the retrieved tweets and kept the tokens that had a dictionary entry as adjectives. We then matched the tweet adjectives to our domain specific vocabulary. For all subsequent analysis we only considered adjectives that were used in tweets and also appeared in our vocabulary, since we were intersected to see the relevance of our vocabulary in terms of actual usage and frequency over time. Without actually classifying each tweet, we counted the frequency of positive and negative opinion words that we identified in the collection of tweets. An example of top opinion words that we identified for the highest and lowest 3 http://www.cs.cornell.edu/people/pabo/movie-review-data/ ranking movies are shown in Table 2. Table 3 presents a sample of the movies that we analyzed, showing for each movie the genre, number of tweets, our score obtained by counting the positive opinion words and the IMDb score. In Figure 4 we represent graphically the positive and negative opinion word counts for the movie Inception.
Movie mystery, sci-fi,
thriller animation, comedy, family drama, thriller drama, music,
romance comedy, war comedy, musical action, sci-fi drama, sci-fi, thriller sci-fi
Our score 66.52 67.71 63.67 70.78 40.67 45.88 56.65 56.65 66.23
IMDb score 8.9 7.3 7 6.2 2.5 4.5 8.7 8.3 8.4
Tweets 19,256 8,109 15,349 1,244 44 79 1,947 407 419
In the cases presented in Table 3, there is a relationship between the number of positive opinion words and the rating from IMDb. One thing to notice is that in IMDb the movie ratings can be roughly grouped in three categories: ratings between seven and ten points accounting for good and very good movies, between five and seven points for average movies, and below five points for poor quality movies. Our positive opinion word count has a maximum of approximately 70 (or seven on a scale from zero to ten). In our future work we plan to conduct a series of experiments in order to determine if there exists a correlation between the two numbers: the IMDb rating and the number of positive opinion words. This involves collecting a higher number of movie related tweets (in the order of hundreds) in order to be able to report significant results.
In this paper, we presented an approach to identifying opinion diversity expressed within text, with the aid of a domain-specific vocabulary. As a use case, we processed a corpus of IMDb movie reviews, extracted a set of adjectives together with their opinion orientation and used the generated opinion lexicon to analyze a different opinion source corpus, i.e. a tweet collection. For future work, we plan to further extend our algorithm to include opinion words expressed by verbs and adverbs, as well as more complex expressions. A second item point is carrying out a set of experiments in order to determine the correlation between positive opinion words for a given movie and the IMDb movie rating. Thirdly, from the lessons learned, we would look into applications in other domains like product reviews.
The research leading to these results has received funding from the Slovenian Research Agency and the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n°257790.