In this paper we present a supervised hybrid approach for Sentiment Analysis in Real-time Applications. The main goal of this work is to design an approach which employs very few resources but obtains near state-of-the-art results.
Recent years have seen the birth of Social Networks and Web 2.0. They have facilitated people to share aspects and opinions about their everyday life. This subjective information can be interesting for general users, brands and organisations. However, the vast amount of information (for example, over 500 million messages per day in Twitter1) complicates traditional sentiment analysis systems to process this subjective information in realtime. The performance of sentiment analysis tools has become increasingly critical.
The main goal of our work is to design a sentiment analysis approach oriented to realtime applications. An approach that balances e ciency and quality. It must employ very few resources, in order to be able to process as many texts as possible. This will also make sentiment analysis more accessible for
This research work has been partially funded by the University of Alicante, Generalitat Valenciana, Spanish Government, Ministerio de Educacion, Cultura y Deporte and Ayudas Fundacion BBVA a equipos de investigacion cient ca 2016 through the projects TIN2015-65100-R, TIN201565136-C2-2-R, PROMETEOII/2014/001, GRE1601: Plataforma inteligente para recuperacion, analisis y representacion de la informacion generada por usuarios en Internet and Analisis de Sentimientos Aplicado a la Prevencion del Suicidio en las Redes Sociales (ASAP).
1www.internetlivestats.com/twitter-statistics everybody. In addition, the quality of the approach should be near the state-of-the-art results. In the following sections we explain our approach in detail. Section 2 brie y describes the related work in the eld and introduce our work. In Section 3 we detail the approach we propose. Finally, Section 4 concludes the paper, and outlines the future work. 2
Two main approaches can be followed:
machine learning and lexicon-based
Most of the current sentiment analysis
approaches employ words, n-grams and phrases
as information units for their models, either
as features for machine learning approaches,
or as dictionary entries in the lexicon-based
approaches. However, words and n-grams
have some problems to represent the
exibility and sequentiality of human language.
This is the reason why we decided to use
skipgrams. The use of skipgrams is a
technique whereby n-grams are formed (bigrams,
trigrams, etc.), but in addition to using
adjacent sequences of words, it also allows some
words to be skipped
Our contribution consists in a hybrid approach which creates a lexicon from a labelled dataset and builds a polarity classi er from the dataset and the generated lexicon with machine learning techniques. Its architecture can be seen in Figure 1. In the following subsections we explain the di erent parts of our approach in detail.
Dataset
Tokenisation Lexicon Generation
Supervised Learning
Lexicon
Classifier
We tried to employ the minimum number of
external linguistic tools, to minimise the
possible propagation of external errors, in
addition to the extra time they can consume. The
tokenisation process starts obtaining all the
words in the text. We only extract words
containing alphabetic characters. Numbers,
punctuation symbols, or emoticons, are not
considered at this moment, but we are
studying the best way to include them in the
future. The only external resource we employ
for the tokenisation process is a stemmer to
obtain the most general form of the words
we extracted. We preferred a stemmer over
a lemmatiser because they are much faster
Once we have the words in the text, we combine them using the skipgram modelling to obtain multiword terms. We will use two variables in this work: n will be the maximum number of words when building a new term with the skipgram modelling, and k will be the maximum number of skips. Note that n = 3 includes all the terms with 1, 2 and 3 words, and k = 3 includes 1, 2 and 3 skips. 3.2
Lexicon generation
In summary, our sentiment lexicon consists
of a list of terms for each polarity, assigning
a score indicating how strongly that term is
2snowball.tartarus.org
related to that polarity. To build this lexicon,
we need a polarity labelled dataset, which
will provide both the terms in the lexicon and
their scores. There exist many term scoring
techniques
First, we will describe our counting formulas. In general, when we want to count the number of documents the term t occurs, we usually loop over the dataset and add 1 each time we nd that term in a document. Instead, we add a value that is inversely proportional to the number of skips. This is what formulas in Equations 1 and 2 do, where D is the labelled dataset; jDj is the number of documents in D, d is a document in D, Dp is the subset of documents in D labelled with polarity p, jtj is the number of words in term t, and (t; d) is the number of skips of term t in document d.
C(t) = C(t; p) =
X [t 2 d] d2D X [t 2 d] d2Dp
jtj jtj + (t; d)
jtj jtj + (t; d)
With this counting formulas, the number of skips is taken into account, and we can build our nal scoring formula shown in Equation 3, where s(t; p) is the score of term t for the polarity p, and is a factor that gives more relevance to terms that appear a largest number of times. This factor depends on the size and the domain of the dataset. s(t; p) =
C(t; p)
C(t)
C(t; p) C(t; p) +
At the end of this process we have a list of
skipgrams with a score for each polarity: our
sentiment lexicon. Table 1 shows an example
of a dictionary built using the Movie Reviews
dataset
this mess worst movie is terrible ludicrous waste
outstanding is terri c
nest breathtaking is excellent
.862 .826 .823 .803 .795 We use machine learning techniques to create a model able to classify the polarity of new texts. The documents in the dataset are employed as training instances, and the labelled polarities are used as categories. However, in contrast with text classi cation approaches, we do not create one feature per term, we create a feature per polarity. In other words, we have the same number of features and categories. Our hypothesis is that this number of features is enough to obtain a decent system quality with a low latency. The weight of each feature is calculated as speci ed in Equation 4, where w(d; p) is the weight of the feature for polarity p in document d. w(d; p) = X s(t; p) t2d
jtj jtj + (t; d) (4)
Table 2 shows an example of feature weighting for the text \worst movie ever" using again the scores of a dictionary built using the Movie Reviews dataset, with n = 2 and k = 10. The nal weights (positive = 1:48, negative = 3:40) will be employed as feature weights for the machine learning process.
To build our model we employed
Support Vector Machines (SVM), as it has been
proved to be e ective on text categorisation
tasks
3www.cs.waikato.ac.nz/ml/weka worst movie ever worst movie worst ever movie ever weight(w) In this paper we presented a supervised hybrid approach for Sentiment Analysis in Twitter. We built a sentiment lexicon from a polarity dataset using statistical measures. We employed skipgrams as information units, to enrich the sentiment lexicon with combinations of words that do not appear explicitly in the text. The lexicon created was used in conjunction with machine learning techniques to create a polarity classi er.
Preliminary performance experiments have shown an acceptable speed to be employed in real-time applications4. Processing speeds go from 1; 000 documents per second in the worst cases (long texts, great values for n and k) to 10; 000 in the best cases (short texts, low values for n and k). These numbers are good enough to work with extensively used platforms like Twitter, where users generate over 500 million tweets per day (this is almost 6,000 tweets per second)5.
Moreover, experiments with di erent
datasets have also obtained promising
results
As future work, we plan to study new methods to calculate and combine the weight 4Using a Macbook Pro 2.4 GHz i5 with 8GB RAM 5www.internetlivestats.com/twitter-statistics of the skipgrams. We also want to add more features to the machine learning algorithm, but always trying to maintain a small number of them, in order to avoid increasing the latency. In addition, we want to include external resources and tools, such as knowledge from existing sentiment lexicons, but always focused in real-time applications. We will also extend our study to di erent corpora and domains, to con rm the robustness of the approach. Tweets. En Proceedings of the International Workshop on Semantic Evaluation (SemEval-2013).