Sentiment analysis is becoming one of the most active area in Natural Language Processing nowadays. Its importance coincides with the growth of social media and the open space they create for expressing opinions and emotions via reviews, forum discussions, microblogs, Twitter and social networks. Most of the existing approaches on sentiment analysis rely mainly on the presence of a ect words that explicitly reect sentiment. However, these approaches are semantically weak, that is, they do not take into account the semantics of words when detecting their sentiment in text. Only recently a few approaches (e.g. sentic computing) started investigating towards this direction. Following this trend, this paper investigates the role of semantics in sentiment analysis of movie reviews. To this end, frame semantics and lexical resources such as BabelNet are employed to extract semantic features from movie reviews that lead to more accurate sentiment analysis models. Experiments are conducted with di erent types of semantic information by assessing their impact in movie reviews dataset. A 10-fold cross-validation shows that F1 measure increases slightly when using semantics in sentiment analysis in social media. Results show that the proposed approach considering word's semantics for sentiment analysis is a promising direction.
In the movie domain, there has been a huge number of review sites giving opinions about the performance of the movie. While the stars rating to a movie tell us about the success or failure of a movie quantitatively, reviews give us a deeper qualitative insight on di erent aspects of the movie.
Sentiment analysis of movie reviews aims to automatically infer the opinion of the reviewer with respect to various topics or the overall polarity of the review. Most of existing work on sentiment analysis of movie reviews use statistical methods to extract features from the review (bag of words, n-grams, word2vec,etc.) [10]. However, statistical methods su er from the lack of handling meanings and semantics which are crucial for understanding reviewer's sentiment. Therefore, there is the need to shift from a word-level to a conceptuallevel analysis of sentiments. This intuition has been the basis of a novel, multidisciplinary approach to sentiment analysis, called sentic computing1, which aims to include semantic features into sentiment analysis.
Following this trend, some works emerged and explored a new type of features called semantic features aiming to handle meaning and semantics. While they intend to extract semantics, the proposed approaches su er from the lack of handling anaphora resolution and word sense disambiguation.
To overcome this problem, we propose, in this paper, a supervised approach
for sentiment analysis using frame semantics and lexical resources such as
BabelNet2 [
Several approaches have been proposed to solve polarity detection problem
within movie reviews ranging from supervised approaches [10], unsupervised
approaches [
The supervised approaches use a wide range of features and labeled data for
training sentiment classi ers. For instance, Pouransari and Ghili [10] applied
the bag of by supervised ones. Hence, few hybrid approaches, that combine both
supervised and unsupervised methods, emerged. For instance, Maas et al. [
All works mentioned above concentrate on the use of two types of features;
n-grams features and lexicon-based features, for sentiment analysis. However, it
has been argued that sentiment in text is not always associated with individual
words, but instead, through relations and dependencies between words, which
often formulate sentiment [13]. Therefore, a new type of features for sentiment
analysis has been explored called semantic features aiming to handle meanings
and semantics which are crucial for understanding sentiment. In this direction,
some works start to emerge. Maas et al. [
While they intend to extract semantics, all the proposed approaches su er
from the lack of handling anaphora resolution and word sense disambiguation.
Usage of simple lexicon at the nal stage for polarity detection also decreases
its accuracy. To overcome this problem, Recupero et al. [
3.1
Data Description
We use a long text corpora refers to movie reviews. The movie reviews dataset
is a subset of the IMDB movie reviews corpus [
Data Preprocessing For our dataset, we performed a preprocessing step to clean up the data. Data preprocessing consists of three steps: 1. tokenization; we segment text by splitting it by spaces and punctuation marks, and form a bag of words. 2. removal of unnecessary punctuation like ' !', ' ?', etc. as they do not provide any substantial information; 3. removal of HTML tags like '<br>' by using simple regular expressions matching to remove these HTML tags from the text. 4. removing stop words from the text using Stanford NLP stop word list4. 3 https://www.kaggle.com/deepmatrix/imdb-5000-movie-dataset 4 https://github.com/stanfordnlp/CoreNLP/blob/master/data/edu/stanford/ nlp/patterns/surface/stopwords.txt 3.3
Feature Extraction
We use a variety of features for our classi cation experiments. For the baseline,
we use n-grams features. However, in our approach, we focus on identifying new
sets of features to be added to the trained model for sentiment classi cation.
Therefore, we investigate a novel set of features derived from word's semantics,
expressed in term of semantic frames and lexical resources such as BabelNet. To
this end, we leveraged Framester5 a wide coverage hub of linguistic linked data
standardized using frame semantics [
TF.IDF features While n-gram model concentrates more on higher frequency parts of the review, it completely ignores the portions which might be less frequent but have more signi cance for the overall polarity of the review. To overcome this shortcoming of n-gram model, we feature representation of words using TF.IDF. The feature representation for this model is similar to the unigram model except that we use TF.IDF values for each word instead of their frequency counts.
Semantic features The semantic features that we extracted correspond to the semantic frames and the BabelNet synsets returned by Framester for each movie review.
{ BabelNet synsets are sets of synonyms in di erent languages grouped by
BabelNet which is an encyclopedic dictionary that provides concepts and named
entities lexicalized in many languages and connected with large amounts of
semantic relations, automatically created by linking Wikipedia6 to
WordNet7 [
Binary Classi cation Polarity The overall task in this paper is for binary classi cation polarity of movie reviews as negative or positive. Therefore, for this task we build a sentiment classi er using Naive Bayes (NB) method.
To implement this BN classi er, we use the following standard bag-of-features model. Let ff1; : : : ; fkg be a prede ned set of k features that can appear in a movie review. Each feature fi could be expressed in term of frequency or TF.IDF. Let wi(r) be the function representing how fi occurs in the movie review r. Then, each review is represented by the following review vector: !r := (w1(r); w2(r); : : : ; wk(r)).
Our task of polarity detection is to assign to a given review m the sentiment s = argmaxsP (s n r). We derive NB classi er by rst observing that by Bayes' rule:
P (s n r) =
P (s)P (r n s)
P (r) where P (r) plays no role in selecting s . To estimate the term P (s n r), Naive Bayes decomposes it by assuming the fi's are conditionally independent given r's class:
PNB(s n r) =
P (s)(Qik=1 P (fi n s)ni(r))
P (r) We implement our BN classi er in JAVA using an open source code8. 4
In this section, we evaluate the integration of semantic features on binary polarity detection and present the obtained results on our movie reviews dataset and their derivations that led to 7 datasets in total. We then compare these results with those obtained using unigram features (the baseline features).
Our goal for these experiments is then two-fold. First, we aim to evaluate whether our training data with labels derived from frame semantics and lexical resource BabelNet is useful for training sentiment classi ers for social media. 8 https://github.com/ptnplanet/Java-Naive-Bayes-Classifier/tree/master/ src/main/java/de/daslaboratorium/machinelearning/classifier Second, we want to evaluate the e ectiveness of the semantic features for sentiment analysis in user-generated data. How useful are the semantic features on movie reviews texts? How much gain do we get from these features?
For our set of experiments, we use NB trained from word unigrams as a baseline model, for each derived dataset. Then, we incorporate the semantic features into NB by either replacing the original bag-f-words feature space or augmenting into it. For each dataset, we train two Bayes classi ers, which use di erent features: unigram features and TF.IDF features.
For each dataset, we perform a 10-cross validation and report results averaged over 10 runs using F 1 measure.
Table 1 shows F 1 measures with bag-of-word (BOW) model (unigram model) and TF.IDF model, of our sentiment classi cation using unigrams as baseline and the other combinations of semantic features with replacement (i) BabelNet synsets (BNS), (ii) semantic frames (SF), (iii) BabelNet synsets and semantic frames (BNS+SF) and augmentation (i) unigrams and BabelNet synsets, ii) unigrams and semantic frames, iii) unigrams, BabelNet synsets and semantic frames).
Features unigrams BabelNet synsets Semantic Frames
BNS+SF unigrams+BNS unigrams+SF unigrams+BNS+SF
TF.IDF BOW 79:56 79:97 79:27 80.64 78:97 78:30 79:48 79:81 79:57 79:69 79:57 79:79 79.65 79:65
According to the results shown in Table 1, the incorporation of semantic features slightly outperforms the unigrams baselines in the two training model. The gain is small and that something we expected as this dataset is of a speci c domain and the extracted semantic features are not able to well represent each object. 5
In this paper, we presented a supervised approach for sentiment polarity detection in movie reviews using semantic features. We de ned semantic features in term of semantic frames and BabelNet synsets extracted with the linguistic linked data Framester. We explored two di erent approaches for incorporating them into sentiment analysis; with replacement and augmentation. Then, we trained a sentiment classi er that is able to determine positive and negative sentiments of reviews. The classi er is based on the multinomial Naive Bayes classi er that uses n-grams and TF.IDF models.
As future work, we plan to model the problem of sentiment analysis in social media as multi-class classi cation problem where we classify the sentiment in more than binary classes like "Happy", "Bored", "Afraid", etc. Furthermore, we plan to remodel this problem as a regression problem where we can predict the degree of a nity for the review instead of a simple negative/positive class. As well as, we plan to perform more experiments on di erent type of user-generated data other than movie reviews, namely tweets, and blogs.
This work has been supported by Sardinia Regional Government (P.O.R. Sardegna F.S.E. Operational Programme of the Autonomous Region of Sardinia, European Social Fund 2014-2020 - Axis IV Human Resources, Objective l.3, Line of Activity l.3.1.). 10. Pouransari, H. and Ghili, S.: Deep Learning for Sentiment Analysis of Movie Reviews. From https://cs224d.stanford.edu/reports/PouransariHadi.pdf, 2014. 11. Recupero, D.R., Presutti, V., Consoli, S., Gangemi, A., Giovanni, A. and Nuzzolese, G.: Sentilo: Frame-Based Sentiment Analysis. Journal of Cognitive Computation, Volume 7, number 2, pp. 211-225, 2015. 12. Rothfels,J. and Tibshirani, J.: Unsupervised sentiment classi cation of English movie reviews using automatic selection of positive and negative sentiment items.
Technical report, Stanford University, 2010. 13. Saif, H., M., He, Fernandez, Y. and Alani, H.: Contextual semantics for sentiment analysis of Twitter. In proceedings of Journal of Information Processing and Management, volume 52(1), pp.5-19, 2016. 14. Zagibalov, T. and Carroll, J.: Automatic seed word selection for unsupervised sentiment classi cation of Chinese text. In Proceedings of the 22nd International Conference on Computational Linguistics, Volume 1, pp. 1073-1080. Association for Computational Linguistics, 2008.