=Paper=
{{Paper
|id=Vol-1702/tass2016_proceedings_v216
|storemode=property
|title=GTI at TASS 2016: Supervised Approach for Aspect Based Sentiment Analysis in Twitter
|pdfUrl=https://ceur-ws.org/Vol-1702/tass2016_proceedings_v216.pdf
|volume=Vol-1702
|authors=Tamara Álvarez-López,Milagros Fernández-Gavilanes,Silvia García-Méndez,Jonathan Juncal-Martínez,Francisco Javier González-Castaño
|dblpUrl=https://dblp.org/rec/conf/sepln/Alvarez-LopezGG16
}}
==GTI at TASS 2016: Supervised Approach for Aspect Based Sentiment Analysis in Twitter==
https://ceur-ws.org/Vol-1702/tass2016_proceedings_v216.pdf
TASS 2016: Workshop on Sentiment Analysis at SEPLN, septiembre 2016, pág. 53-57
GTI at TASS 2016: Supervised Approach for Aspect Based
Sentiment Analysis in Twitter∗
GTI en TASS 2016: Una aproximación supervisada para el análisis de
sentimiento basado en aspectos en Twitter
Tamara Álvarez-López, Milagros Fernández-Gavilanes, Silvia Garcı́a-Méndez,
Jonathan Juncal-Martı́nez, Francisco Javier González-Castaño
GTI Research Group, AtlantTIC
University of Vigo, 36310 Vigo, Spain
{talvarez,mfgavilanes,sgarcia,jonijm}@gti.uvigo.es, javier@det.uvigo.es
Resumen: Este artı́culo describe la participación del grupo de investigación GTI,
del centro AtlantTIC, perteneciente a la Universidad de Vigo, en el tass 2016. Este
taller es un evento enmarcado dentro de la XXXII edición del Congreso Anual de
la Sociedad Española para el Procesamiento del Lenguaje Natural. En este trabajo
se propone una aproximación supervisada, basada en clasificadores, para la tarea de
análisis de sentimiento basado en aspectos. Mediante esta técnica hemos conseguido
mejorar las prestaciones de ediciones anteriores, obteniendo una solución acorde con
el estado del arte actual.
Palabras clave: Análisis de sentimiento, aspectos, SVM, aprendizaje automático,
Twitter
Abstract: This paper describes the participation of the GTI research group of
AtlantTIC, University of Vigo, in tass 2016. This workshop is framed within the
XXXII edition of the Annual Congress of the Spanish Society for Natural Language
Processing event. In this work we propose a supervised approach based on classifiers,
for the aspect based sentiment analysis task. Using this technique we managed to
improve the performance of previous years, obtaining a solution reflecting the actual
state-of-the-art.
Keywords: Sentiment analysis, aspects, SVM, machine learning, Twitter
1 Introduction mum length of the post. However, tweets
have other elements we have to consider,
The social media activity is being profused
like hashtags, mentions and retweets. More
in the recent years, users post opinions and
concretely, aspect-based sentiment analysis
comments in Twitter and in other social plat-
(absa) consists of extracting opinions, i.e.
forms. Due to this, there is a huge amount
determining the sentiment polarity, from spe-
of information available that could be use-
cific entities in the text (Liu, 2012). There-
ful for business, in order to design marketing
fore, this task becomes a challenge on the
campaigns or to apply any kind of business
field of nlp.
analysis.
As a consequence, the research on text The tass Workshop (Garcı́a-Cumbreras
mining and also on the field of Sentiment et al., 2016) and the sepln conference of-
Analysis (sa) has grown considerably these fer an opportunity for participants to know
days. sa is the part of Natural Language Pro- about the latest advances on the field of nlp
cessing (nlp) responsible for determining the for Spanish language.
polarity of a text or a whole sentence. The Many approaches applied to sa can be
sa applied to Twitter has to be conducted found in the literature, where it is possi-
in a restricted scenario due to the maxi- ble to distinguish between knowledge based
∗
approaches (Brooke, Tofiloski, and Taboada,
This work was partially supported by the Minis-
terio de Economı́a y Competitividad under project
2009; Fernández-Gavilanes et al., 2016), us-
COINS (TEC2013-47016-C2-1-R) and by Xunta de ing grammars and thesaurus and others
Galicia (GRC2014/046). based on machine learning approaches (Mo-
ISSN 1613-0073
T. Álvarez-López, M. Fernández-Gavilanes, S. García-Méndez, J. Juncal-Martínez, F. J. González-Castaño
hammad, Kiritchenko, and Zhu, 2013). In plying sa to Twitter has been fully ad-
the last years we can also find deep learning dressed (Pak and Paroubek, 2010; Han and
approaches (Bengio, 2009), applied to this Baldwin, 2011). Within the chosen solu-
task. tions, we highlight the text normalization
We present our supervised machine learn- approach (Fabo, Cuadros, and Etchegoyhen,
ing (ml) system which consists of a Support 2013) and the use of key elements in classifi-
Vector Machine (svm) classifier. Our objec- cation approach (Wang et al., 2011). Others
tive is to conduct the sa process at an aspect hold the advantages of using deep learning
level, task 2, determining the polarity of a techniques in this task (dos Santos and Gatti,
specific given part of a sentence. 2014).
The article is structured as follows. Sec- According to the purpose of the developed
tion 2 is a review of the research involving sa systems, it is possible to find applications
in the Twitter domain. Then, the Section 3 like classification of product reviews and po-
describes the applied approach and the im- litical sentiment and election results pre-
plemented system. In Section 4, we show the diction (Bermingham and Smeaton, 2011),
experimental results of our system. Finally, among others.
in Section 5 we present the conclusions and
future works. 3 System Overview
In this section we make a brief description
2 Related work of the system submitted for Task 2: Aspect-
A large amount of literature related to Opin- based sentiment analysis. We developed a
ion Mining (om) and sa can be found (Pang supervised system, based on a svm classifier
and Lee, 2008; Martı́nez-Cámara et al., using different features. In the next subsec-
2016). Most of the systems are applied to tions we explain the different steps required.
Twitter. However others are applied to social
media platforms within the micro-blog con- 3.1 Preprocessing
text. Due to this, the approaches are varied Before applying any supervised approach to
technically and in connection with the pur- our corpus, some preprocessing is needed.
pose. First of all, we have to normalize the text,
Two main approaches exist in sa: super- since in Twitter language we can find abbre-
vised and unsupervised learning ones. Super- viations, mentions, hashtags, URLs or mis-
vised systems implement classification meth- spellings. In order to do that, we replace the
ods like svm, Logistic Regression (lr), Con- URLs with the “URL” tag and we replace the
ditional Random Fields (crf), K-Nearest abbreviations or misspellings with the correct
Neighbors (knn), etc. Cui, Mittal, and Datar entire word. For mentions and hashtags, we
(2006) affirmed that svm are more appro- keep them unchanged but deleting the “@”
priate for sentiment classification than gen- or “#” symbols. Moreover, when a hashtag
erative models, due to their capability for is composed of several words, we split and
working with ambiguity, that is, dealing with treat them as different tokens.
mixed feelings. Supervised algorithms are After this, a lexical analysis is carried out.
used when the number of classes, as well as It consists of lemmatization and POS tag-
the representative members of each class, are ging, which are performed by means of Freel-
known. ing tool (Atserias et al., 2006).
Unsupervised systems are based on lin- Once we have analysed lexically the texts,
guistic knowledge like lexicons, and syntactic we decided to separate the sentences by the
features in order to infer the polarity (Pal- different aspects. For doing that, the scope
toglou and Thelwall, 2012). These last tech- of each aspect is determined, applying the
niques represent a more effective approach in following rules, which are adapted from our
the cross-domain context and for multilingual English aspect based sentiment anaylisis sys-
applications. The unsupervised classification tem (Alvarez-López et al., 2016)
algorithms do not work with a training set,
in contrast, some of them use clustering algo- • If there is only one aspect in the sen-
rithms in order to distinguish groups (Li and tence, we keep the sentence unchanged,
Liu, 2010). and introduce it entirely as input for the
As noted earlier, the special case of ap- next step.
54
GTI en TASS 2016: Una aproximación supervisada para el análisis de sentimiento basado en aspectos en Twitter
• If there are multiple aspects, we separate a number of political issues, such as health
the sentences by punctuation marks, or economy, among others. These issues are
conjunctions or other aspects found. framed in the political campaign of Andalu-
sian elections in 2015, where each aspect re-
• If there are several aspects with no words
lates to one or several entities that corre-
between them, we consider that they be-
spond to one of the main political parties
long to the same context, and assign the
in Spain (PP, PSOE, IU, UPyD, Cs and
same polarity to all of them.
Podemos). The corpus is composed by 1,284
tweets, and has been divided into a training
3.2 SVM classifier set (784 tweets) and a set of evaluation (500
In this section we describe the strategy fol- tweets).
lowed to determine the sentiment (positive, In order to evaluate the performance of
negative or neutral) for each aspect prede- the various features for polarity classification
fined in corpus. at an aspect-based level, we perform a se-
We develop a svm classifier, using the lib- ries of ablation experiments as shown in Ta-
svm library (Chang and Lin, 2011). The in- ble 1. We start with the word token base-
puts for the svm will be the sentences sep- line classifier, and then add all four sets of
arated by contexts, as explained in the pre- features that help to increase performance as
vious subsection. The features extracted are measured by accuracy. As we might expect,
the following: including the aspect feature has the most
marked effect on the performance of polarity
• Word tokens of nouns, adjectives and classification, although all the features con-
verbs in the sentence. tributed to improving overall performance on
• Lemmas of verbs, nouns and adjectives stompol corpus.
that appear in each sentence.
Type Accuracy Improvement
• POS tags of nouns, adjectives and verbs.
Word token 56.12
• N-grams of different length, grouping the +Lemmas 57.64 +1.52%
words in each sentence. +pos tags 58.26 +0.62%
• Aspects appearing in the sentence. We +Aspects 59.94 +1.68%
join “aspect”-“entity”, defined in each +Negations 60.60 +0.66%
target as a feature.
• Negations. We create a negation dic- Table 1: Results for polarity feature ablation
tionary, which contains several parti- experiments on stompol corpus
cles indicating negation, such as “no”,
“nunca”, etc. Due to the low participation of research
teams in task 2 this year, we decided to com-
The previous features are all binary ones, pare our proposal to the systems presented
assigning the value 1 if the current feature is this year and also to that ones of last year,
present in the tweet and the value 0, if not. because of the use of the same dataset.
For this reason, Table 2 compares results
4 Experimental Results for our approach with different official ones
The Task 2: Sentiment Analysis at the as- submitted in 2015 and 2016 tass editions.
pect level consists of assigning a polarity label In this way, we compared our results for a
to each aspect, which were initially marked ml approach based on well-known squared-
in the stompol corpus (Martı́nez-Cámara et regularised logistic regression with a snippet
al., 2016) raised by the tass organization. In of length 4 (Lys-2) described in Vilares et
this way, this corpus provides both polarity al. (2015), a clustering method focused on
labels and the identification of the aspects grouping authors with similar sociolinguis-
that appear in each tweet. The aim is to be tic insights (TID-spark) described in Park
able to correctly assign to each aspect a pos- (2015), a recurrent neural network composed
itive, negative or neutral polarity. of a single long short term memory and a
In this regard, the stompol corpus con- logistic function (Lys-1) described in Vilares
sists of a set of Spanish tweets related to et al. (2015), a ml approach based on a
55
T. Álvarez-López, M. Fernández-Gavilanes, S. García-Méndez, J. Juncal-Martínez, F. J. González-Castaño
svm with a snipped of length 5,7 and 10 In Proceedings of LREC, volume 6, pages
(ELiRF) described in Hurtado, Plà, and Bus- 48–55.
caldi (2015), and the best performing run of
Bengio, Y. 2009. Learning deep architec-
the actual task 2 tass edition (ELiRF-UPV).
tures for AI. Found. Trends Mach. Learn.,
2(1):1–127, January.
Experiment Task edition Accuracy
Bermingham, A. and A. F. Smeaton. 2011.
ELiRF-UPV 2016 63.3
On using Twitter to monitor political sen-
ELiRF 2015 63.3
timent and predict election results.
GTI 2016 60.6
LyS-1 2015 59.9 Brooke, J., M. Tofiloski, and M. Taboada.
TID-spark 2015 55.7 2009. Cross-linguistic sentiment analysis:
Lys-2 2015 54.0 From english to spanish. In G. Angelova,
K. Bontcheva, R. Mitkov, N. Nicolov, and
N. Nikolov, editors, RANLP, pages 50–
Table 2: Results of different approaches in 54. RANLP 2009 Organising Committee
2015/2016 tass editions on stompol corpus / ACL.
Comparing the results, the performance of Chang, C.-C. and C.-J. Lin. 2011. Libsvm: a
our current model is close from the top rank- library for support vector machines. ACM
ing systems of this and last year. Transactions on Intelligent Systems and
Technology (TIST), 2(3):27.
5 Conclusions and future works
Cui, H., V. Mittal, and M. Datar. 2006.
This paper describes the participation of the
Comparative experiments on sentiment
GTI group in the tass 2016, Task 2: Aspect-
classification for online product reviews.
Based Sentiment Analysis. We developed a
In Proceedings of the 21st National Con-
supervised system based on a svm classifier
ference on Artificial Intelligence - Vol-
for the aspect-based sentiment analysis. The
ume 2, AAAI’06, pages 1265–1270. AAAI
performance of our approach has been com-
Press.
pared to that ones submitted this year but
also to that ones submitted last year. Exper- dos Santos, C. N. and M. Gatti. 2014. Deep
imental results suggest that we need to in- convolutional neural networks for senti-
clude explore new features, such as word em- ment analysis of short texts. In COLING,
bedding representations or paraphrase (Zhao pages 69–78.
and Lan, 2015), in order to improve the per-
Fabo, P. R., M. Cuadros, and T. Etchegoy-
formance.
hen. 2013. Lexical normalization of
As future work we plan to include new fea-
spanish tweets with preprocessing rules,
tures explained before and to develop a new
domain-specific edit distances, and lan-
system which combines different ml classifi-
guage models. In Proceedings of the Tweet
cation methods. We are also interested in
Normalization Workshop co-located with
considering different paradigms of heteroge-
29th Conference of the Spanish Society
neous classification, such as deep learning to
for Natural Language Processing (SEPLN
increase the performance.
2013), Madrid, Spain, September 20th,
References 2013., pages 59–63.
Alvarez-López, T., J. Juncal-Martınez, Fernández-Gavilanes, M., T. Álvarez-López,
M. Fernández-Gavilanes, E. Costa- J. Juncal-Martı́nez, E. Costa-Montenegro,
Montenegro, and F. J. González-Castano. and F. J. González-Castaño. 2016. Unsu-
2016. Gti at semeval-2016 task 5: Svm pervised method for sentiment analysis in
and crf for aspect detection and unsu- online texts. Expert Systems with Appli-
pervised aspect-based sentiment analysis. cations, 58:57–75.
Proceedings of SemEval, pages 306–311. Garcı́a-Cumbreras, M. A., J. Villena-Román,
Atserias, J., B. Casas, E. Comelles, E. Martı́nez-Cámara, M. C. Dı́az-Galiano,
M. González, L. Padró, and M. Padró. M. T. Martı́n-Valdivia, and L. A. Ureña-
2006. Freeling 1.3: Syntactic and seman- López. 2016. Overview of tass 2016. In
tic services in an open-source NLP library. Proceedings of TASS 2016: Workshop on
56
GTI en TASS 2016: Una aproximación supervisada para el análisis de sentimiento basado en aspectos en Twitter
Sentiment Analysis at SEPLN co-located (LREC’10), Valletta, Malta, may. Eu-
with the 32nd SEPLN Conference (SE- ropean Language Resources Association
PLN 2016), Salamanca, Spain, Septem- (ELRA).
ber.
Paltoglou, G. and M. Thelwall. 2012. Twit-
Han, B. and T. Baldwin. 2011. Lexi- ter, myspace, digg: Unsupervised sen-
cal normalisation of short text messages: timent analysis in social media. ACM
Makn sens a #twitter. In Proceedings of Transactions on Intelligent Systems and
the 49th Annual Meeting of the Associa- Technology (TIST), 3(4):66.
tion for Computational Linguistics: Hu- Pang, B. and L. Lee. 2008. Opinion min-
man Language Technologies - Volume 1, ing and sentiment analysis. Found. Trends
HLT ’11, pages 368–378, Stroudsburg, PA, Inf. Retr., 2(1-2):1–135, January.
USA. Association for Computational Lin-
guistics. Park, S. 2015. Sentiment classification us-
ing sociolinguistic clusters. In Proceedings
Hurtado, L. F., F. Plà, and D. Bus- of TASS 2015: Workshop on Sentiment
caldi. 2015. ELiRF-UPV en TASS Analysis at SEPLN co-located with 31st
2015: Análisis de sentimientos en Twit- SEPLN Conference (SEPLN 2015), Ali-
ter. In Proceedings of TASS 2015: Work- cante, Spain, September 15, 2015., pages
shop on Sentiment Analysis at SEPLN co- 99–104.
located with 31st SEPLN Conference (SE-
PLN 2015), Alicante, Spain, September Vilares, D., Y. Doval, M. A. Alonso, and
15, 2015., pages 75–79. C. Gómez-Rodrı́guez. 2015. Lys at
TASS 2015: Deep learning experiments
Li, G. and F. Liu. 2010. A clustering-based for sentiment analysis on spanish tweets.
approach on sentiment analysis. In Intel- In Proceedings of TASS 2015: Work-
ligent Systems and Knowledge Engineer- shop on Sentiment Analysis at SEPLN co-
ing (ISKE), 2010 International Confer- located with 31st SEPLN Conference (SE-
ence on, pages 331–337. IEEE. PLN 2015), Alicante, Spain, September
15, 2015., pages 47–52.
Liu, B. 2012. Sentiment Analysis and Opin-
ion Mining. Synthesis Lectures on Human Wang, X., F. Wei, X. Liu, M. Zhou, and
Language Technologies. Morgan & Clay- M. Zhang. 2011. Topic sentiment anal-
pool Publishers. ysis in Twitter: A graph-based hashtag
sentiment classification approach. In Pro-
Martı́nez-Cámara, E., M. A. Garcı́a- ceedings of the 20th ACM International
Cumbreras, J. Villena-Román, and Conference on Information and Knowl-
J. Garcı́a-Morera. 2016. Tass 2015 - the edge Management, CIKM ’11, pages 1031–
evolution of the spanish opinion mining 1040, New York, NY, USA. ACM.
systems. Procesamiento del Lenguaje
Natural, 56:33–40. Zhao, J. and M. Lan. 2015. Ecnu: Lever-
aging word embeddings to boost perfor-
Mohammad, S. M., S. Kiritchenko, and mance for paraphrase in Twitter. In Pro-
X. Zhu. 2013. Nrc-canada: Building the ceedings of the 9th International Work-
state-of-the-art in sentiment analysis of shop on Semantic Evaluation (SemEval
tweets. In Proceedings of the seventh in- 2015), pages 34–39, Denver, Colorado,
ternational workshop on Semantic Evalu- June. Association for Computational Lin-
ation Exercises (SemEval-2013), Atlanta, guistics.
Georgia, USA, June.
Pak, A. and P. Paroubek. 2010. Twit-
ter as a corpus for sentiment analy-
sis and opinion mining. In N. C. C.
Chair), K. Choukri, B. Maegaard, J. Mar-
iani, J. Odijk, S. Piperidis, M. Ros-
ner, and D. Tapias, editors, Proceedings
of the Seventh International Conference
on Language Resources and Evaluation
57