This paper describes the participation of the GTI research group of AtlantTIC, University of Vigo, and Gradiant (Galician Research and Development Centre in Advanced Telecommunications), in the tass 2015 workshop. Both groups have worked together in the development of a hybrid approach for sentiment analysis, at a global level, of Twitter, proposed in task 1 of tass. A system based on classi ers and unsupervised approaches, built with polarity lexicons and syntactic structures, is presented here. The combination of both approaches has provided highly competitive results over the given datasets.
In recent years, research on the eld of Sentiment Analysis (sa) has increased considerably, due to the growth of user content generated in social networks, blogs and other platforms on the Internet. These are considered valuable information for companies, which seek to know or even predict the acceptance of their products, to design their marketing campaigns more e ciently. One of these sources of information is Twitter, where users are allowed to write about any topic, using colloquial and compact language. As a consecuence, SA in Twitter is specially challenging, as opinions are expressed in one or two short sentences. Moreover, they include special elements such as hashtags or mentions. Henceforth, additional treatments must be applied when analyzing a tweet.
Numerous contributions on this subject
can be found in the literature. Most of them
are supervised machine learning approaches,
although unsupervised semantic can also be
found in this eld. The rst ones are
usually classi ers built from features of a \bag
of words" representation
The tass evaluation workshop aims at
providing a benchmark forum for comparing
the latest approaches in this eld. In this
way, our team only took part in task 1 related
to sa in Twitter. This task encompasses
four experiments. The rst consists of
evaluating tweet polarities over a big dataset of
tweets, with only 4 tags, positive (p), negative
(n), neutral (neu) or no opinion (none)
expressed. In the second experiment, the same
evaluation is requested over a smaller
selection of tweets. The third and fourth
experiments propose the same two datasets,
respectively, but with 6 di erent possible tags,
including strong positive (p+) and strong
negative (n+). In addition, a training set has
been provided, in order to build the
models
The rest of this article is structured as follows: Section 2 presents in detail the system proposed. Section 3 describes the results obtained and some experiments performed over the target datasets. Finally, Section 4 summarizes the main ndings and conclusions. 2
Our system is a combination of two di erent
approaches. The rst approach is an
unsupervised approach, based on sentiment
dictionaries, which are automatically generated
from the set of tweets to analyze (a set of
positive and negative seeds, created manually,
are necessary to start the process). The
second is a supervised approach, which employs
Conditional Random Fields (crfs)
In this way, both approaches have been
previously adapted to the English language
and submitted to the SemEval-2015
sentiment analysis task, achieving good rankings
and results separately
The rst treatments to be applied over the set of tweets rely on natural language processing (nlp) and are common to both approaches. They include preprocessing, lexical and syntactic analysis and generation of sentiment lexicons.
The language used on Twitter contains words that are not found in any dictionary, because of orthographic modi cations. The aim here is to normalize the texts to get closer to formal language. The actions executed in this stage are the substitution of emoticons, which are divided in several categories, by equivalent Spanish words, for example, :) is replaced by e feliz ; the substitution of frequent abbreviations; the removal of repeated characters and the replacement of speci c Twitter words such as hashtags, as well as mentions or urls, by hashtag, mencion and url tags, respectively.
After the preproccessing, the input text is
morphologically tagged to obtain the
part-ofspeech (PoS) associated with a word, as
adjectives, adverbs, nouns and verbs. Finally,
a dependency tree is created with the
syntactic functions annotated. These steps are
performed with the Freeling tool
Sentiment lexicons have been used in many
supervised and unsupervised approaches for
sentiment detection. They are not so
common in Spanish as in English,
although there are some available, such as
socal
However, these dictionaries are not
contextualized, so we generate additional ones
automatically from the words in the
syntactic dependencies of each tweet, considering
verbs, nouns and adjectives. Then, we
apply a polarity expansion algorithm based on
graphs
As explained in the next sections, the dictionaries obtained must be adapted for using them in the supervised approach. In this case, only a list of positive and negative words is required, with no associated values. 2.2
This subsection presents the supervised approach for the tagging of Spanish tweets. After the previous steps, lexical, PoS and crf labels are jointly combined to build the nal features that de ne the input to a logistic regression classi er.
The system works in two phases. First, a learning phase is applied in which the system learns the parameters of the supervised model using manually tagged data. Second, the supervised model is only trained with the training vector provided by the organization.
This strategy uses several dictionaries as an
input for di erent steps of the feature
extraction process. Hence, a polarity dictionary,
previously created and adapted, containing
positive and negative words, is provided as
input in this step. Certain polarity shifters
play an important role in the detection of the
polarity of a sentence. Previous attempts in
the academic literature followed di erent
approaches, like hand-crafted rules
Additional dictionaries are used in the system (e.g. adversative particles or superlatives) but their main purpose is to give support of the learning steps with the polarity
Polarity shifters are speci c particles (e.g. no (no)), words (e.g. evitar (avoid)), or constructions (e.g. fuera de (out of)) that modify the polarity of the words under their inuence. Detecting these scopes of in uence closely related to the syntactic graphs is difcult due to the unreliability of dependency and syntactic parsers on Twitter. To solve this problem we trained sequential crfs for each problem we wanted to solve. crfs are supervised techniques that assign a label to each component (in our case the words of a sentence).
Our system follows a similar approach to
Lapponi, Read and Ovrelid (2012) but it
has been enhanced to track intensi cation,
comparisons within a sentence, and the e ect
of adversative clauses (e.g. sentences with
pero (but) particles). We refer the reader to
All the characteristics from previous steps are included as input features of a statistical classi er. The lexical features (word, stem, word and stem bigrams and ags extracted from the polar dictionaries) are included with PoS and the labels from the crfs. The learning algorithm employed was a logistic regressor. Due to the size of the feature space and its sparsity, l1 (0.000001) and l2 (0.00005) regularization was applied to learn the most important features and discard the least relevant for the task. 2.3
The unsupervised approach is based on
generated polarity lexicons applied to the
syntactic structures previously obtained. The nal
sentiment result of each tweet is expressed as
a real number, calculated as follows: rst, the
words in the dependency tree are assigned a
polarity from the sentiment dictionary;
second, a polarity value propagation based on
Usually, adverbs act as intensi ers or
diminishers of the word that follows them.
For example, there is a di erence between
bonito (beautiful) and muy bonito (very
beautiful). The rst one has a positive
connotation, whose polarity is increased by the
adverb muy (very). So, its semantic
orientation is altered. Therefore, the intensi cation
is achieved by assigning a positive or
negative percentage in the intensi ers and
diminishers
If words that imply denial appear in the
text, such as no (no), nunca (never) or ni
(neither)
Sometimes, two words appearing together
express opposite sentiments. The aim
here is to detect these cases, known as
polarity con icts
There is a point in common between
adversative and concessive sentences. In both
cases, one part of the sentence is in
contrast with the other. While the former
express an objection in compliance with what
is said in the main clause, the latter express
a di culty in ful lling the main clause. We
can assume that both constructions will
restrict, exclude, amplify or diminish the
sentiment re ected in them. Some
adversative nexus can be pero (but) or sin
embargo (however)
In order to decide the nal polarity of each tweet, we combine both approaches as follows: applying the supervised approach, 15 di erent outputs are generated, randomizing the training vector and selecting a subset of them for training (leaving out 1500 records in each iteration). Then, another 15 outputs are generated applying the unsupervised approach, using 15 di erent lexicons, created by combining each general lexicon (SDAL, SOCAL, eSOL) with the automatically generated one, and also combining 3 or 4 of them. During this process, when a word appears in several dictionaries, we apply a weighted average, varying the relevance assigned to each dictionary, thus providing more output combinations. Afterwards, we apply a majority voting method among the 30 outputs obtained to decide the nal tweet polarity. This strategy has shown better performance than only one of the approaches by itself, making the combination of both a good choice for the experiments, as explained in the next section. 3
The performance in task 1 was measured by means of the accuracy (correct tweets according to the gold standard). Table 1 shows the results, where accuracy is represented for each experiment, as well as the results of the top ranking systems, out of 16 participants for the 6-tag subtasks, and 15 participants for the 4-tag subtasks.
It can be noticed that the results for 6 tags are considerably worse than those for 4 tags. It appears that it becomes more di cult for our system, and for any system in general, to detect positive or negative intensities, rather than just distinguishing positive from negative. Furthermore, we can also observe in the results for the smaller dataset that accuracy diminishes notably for both experiments.
As previously said, in order to obtain our results, we combined both approaches, by means of a majority voting method. On the
LIF GTI-GRAD
ELIRF
GSI LYS
6 67:22 59:25 67:31 61:83 56:86
Accuracy 6 (1k) 4 51:61 50:92 48:83 48:74 43:45 72:61 69:53 72:52 69:04 66:45 4 (1k) 69:21 67:42 64:55 65:83 63:49 one hand, the outputs resulting from the supervised approach were generated by applying classi ers, with di erent training records. On the other hand, the unsupervised approach requires the use of several dictionaries, getting a real number polarity for each tweet, and then applying an interval to determine when a tweet carries an opinion or not. This interval is xed to [-1, 1] for no opinion. In addition, the number of words containing a polarity is taken into account to decide the neutrality of a tweet. That is, if it contains polar words but the total result lies in [-1, 1], this means that there is a contraposition of opinions, so the tweet is tagged as neutral. However, our combined system seemed to work not so well for neutral texts, specially in the bigger datasets. This may be due to the small proportion of neutral tweets through out the whole dataset, as they only represent a 2.15% of the total number of tweets, rising to 6.3% for the small datasets.
For the 6-tag experiments, p+ and n+ tags were determined with the supervised approach. This decision was taken because the unsupervised approach was not able to discriminate e ciently between p and p+ or between n and n+.
Table 2 shows several experiments with the supervised and unsupervised models, as well as with the combined one, so we can appreciate the improvement in the last case. These results were obtained by applying a majority voting method to each approach separately, with 15 outputs, and then to 30 outputs of the combined result. 4
This paper describes the participation of the GTI Research Group (AtlantTIC, Univer
Supervised Unsupervised
Combined
6 sity of Vigo) and Gradiant (Galician Research and Development Centre in Advanced Telecommunications) in tass 2015 Task 1: Sentiment Analysis at global level. We have presented a hybrid system, combining supervised and unsupervised approaches, which has obtained competitive results and a good position in the nal ranking.
The unsupervised approach consists of sentiment propagation rules on dependencies, whilst the supervised one is based on classiers. This combination seems to work considerably well in this task.
There is still margin for improvement, mostly in neutral tweets detection and more re ned distinction of degrees of positivity and negativity. sion of feature-level opinion lexicons. In Proc. of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, pages 125{131, Stroudsburg, PA, USA. ACL.