<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <issn pub-type="ppub">1613-0073</issn>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Overview of TASS 2017</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Eugenio Mart nez-Camara</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Manuel C. D az-Galiano</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>M. Angel Garc a-Cumbreras</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Manuel Garc a-Vega</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Julio Villena-Roman</string-name>
          <email>3julio.villena@sngular.team</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ubiquitous Knowledge Processing Lab (UKP-TUDA)</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science, Technische Universitat Darmstadt</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Grupo de Investigacion SINAI Universidad de Jaen</institution>
          ,
          <addr-line>Jaen</addr-line>
          ,
          <country country="ES">Spain</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2017</year>
      </pub-date>
      <fpage>13</fpage>
      <lpage>21</lpage>
      <abstract>
        <p>This paper describes TASS 2017, the sixth edition of the Workshop on Semantic Analysis at SEPLN 2017. The main aim is to encourage the research and development of new resources, algorithms and techniques for di erent tasks of semantic analysis in Spanish. In this paper, we present the proposed tasks, the generated datasets, and a summary of the submitted systems.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Since some years ago, Natural Language
Processing (NLP) researchers have been working
on the discovery of the meaning of utterances
from di erent perspectives. One of those
perspectives is the understanding of the
subjective information or rather opinion
information. The task of Sentiment Analysis (SA)
is the result of this study, and it is de ned
as the computational treatment of opinion,
sentiment and subjectivity in text
        <xref ref-type="bibr" rid="ref13">(Pang and
Lee, 2008)</xref>
        .
      </p>
      <p>However, the potential semantic
information encoded in an utterance is so rich and
broad that di erent new NLP tasks have
arosen, such as argumentation mining, stance
classi cation, irony detection or the
considered tasks in the di erent editions of our
sibling workshop SemEval.1</p>
      <p>
        The Spanish language is the second native
language in the world and the second
language in number of speakers. Nevertheless,
the progress of the NLP research in
Spanish is far away to the advance of other
languages like English. Consequently, TASS2
(Taller de Analisis de Sentimientos en la
SEPLN / Workshop on Sentiment Analysis at
SEPLN) was born in 2012 with the aim of
fostering the development of speci c NLP
techniques for the computational treatment
of opinions of text written in Spanish. The
previous editions in 2016
        <xref ref-type="bibr" rid="ref4 ref7">(Garc a-Cumbreras
et al., 2016)</xref>
        , 2015
        <xref ref-type="bibr" rid="ref18">(Villena-Roman et al.,
2015)</xref>
        , 2014
        <xref ref-type="bibr" rid="ref18">(Villena Roman et al., 2015)</xref>
        ,
2013
        <xref ref-type="bibr" rid="ref19">(Villena-Roman et al., 2014)</xref>
        and 2012
        <xref ref-type="bibr" rid="ref19 ref20">(Villena-Roman et al., 2013)</xref>
        have yielded
outstanding linguistic resources such as the
General Corpus of TASS and some datasets
for the task of polarity classi cation at
aspect level, used by a great number of research
groups and companies as reference for
Spanish. Additionally, a research community has
been created around TASS that usually
participate in the workshop and contribute with
vivid discussions about the state-of-the-art
and the next challenges in SA in Spanish.
2http://www.sepln.org/workshops/tass
      </p>
      <p>Copyright © 2017 by the paper's authors. Copying permitted for private and academic purposes.
The organization committee of the
workshop has updated its name in the edition
of 2017 because of the need of widening
the gamut of semantic tasks in TASS. The
new name of TASS is Workshop on
Semantic Analysis at SEPLN (Taller de Analisis
Semantico en la SEPLN ), which allows to
keep the acronym TASS. The change of the
name is a call to researchers on other
semantic tasks (argumentation mining, irony
detection, stance classi cation...) to organize
a shared-task for the treatment of semantic
information in Spanish for the next edition.</p>
      <p>The edition of 2017 proposes two subtasks,
polarity classi cation at document (tweet)
level (Task 1) and aspect level polarity
classi cation (Task 2). Apart from reusing
several datasets of previous editions, a new
dataset was speci cally generated for this
edition. The new dataset is called
InterTASS, which is composed of more than 2,000
tweets annotated at four opinion intensity
level (positive, neutral, negative and
none). Further details about the tasks and
the datasets in Sections 2 and 3 respectively.</p>
      <p>The edition of 2017 has attracted the
participation of 11 teams, mainly from Spain
and America. Most of the systems follow the
state-of-the-art of SA, which is the use of a
deep learning architecture. Most of the teams
participated in Task 1, and a few of them in
Task 2, which is an indication that polarity
classi cation at aspect level is a tough task.</p>
      <p>The rest of this paper is organized as
follows. Section 2 presents in more details
the two subtasks of TASS 2017. Section 3
describes the datasets and how we created
them. Section 4 presents the submitted
systems and the results reached by them.
Finally, Section 5 concludes and points the
future work in TASS.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Tasks</title>
      <p>TASS 2017 has proposed two tasks
addressing the challenging task of SA in Twitter in
Spanish.
2.1</p>
      <sec id="sec-2-1">
        <title>Task 1. Sentiment Analysis at</title>
      </sec>
      <sec id="sec-2-2">
        <title>Tweet level</title>
        <p>This main task focused on the evaluation of
polarity classi cation systems at tweet level
in Spanish. Systems were evaluated on three
di erent datasets: the two test sets of the</p>
        <p>General Corpus of TASS3 and a new corpus,
InterTASS, which was speci cally developed
in 2017 for the task (see Section 3).</p>
        <p>Datasets were annotated with 4 di erent
polarity labels positive, negative,
neutral and none), and systems had to
identify the intensity of the opinion expressed in
each tweet in any of those 4 intensity
levels. For the two sets of the General Corpus
of TASS, which was annotated in 6 polarity
tags, a direct translation from P+ into P and
N+ into N was performed so that the
evaluation is consistent with InterTASS and based
on 4 levels of intensity of polarity.</p>
        <p>All datasets were divided into training,
development and test datasets, which were
provided to participants in order to train and
evaluate their systems. Systems were allowed
to use any set of data as training dataset, i.e.
the training set of InterTASS, other
training sets from the previous editions of TASS
or other sets of tweets. However, using the
test set of InterTASS and the test set of the
datasets of previous editions as training data
was obviously forbidden. Apart from that,
participants could use any kind of linguistic
resource for the development of their classi
cation model.</p>
        <p>Participants were expected to submit 3
experiments per each evaluation set, so each
participant team could submit a maximum of
9 les of results. Results must be submitted
in a plain text le with the following format:
t w e e t i d n t p o l a r i t y</p>
        <p>Allowed values for polarity were P, NEU,
N and NONE.</p>
        <p>Accuracy and the macro-averaged
versions of Precision, Recall and F1 were used
as evaluation measures. Systems were be
ranked by the Macro-F1 and Accuracy
measures.
2.2</p>
      </sec>
      <sec id="sec-2-3">
        <title>Task 2. Aspect-based</title>
      </sec>
      <sec id="sec-2-4">
        <title>Sentiment Analysis</title>
        <p>This second task proposed the development
of aspect-based polarity classi cation
systems. Two datasets from previous editions
were used to evaluate the systems:
SocialTV and STOMPOL (see Section 3). The two
datasets were annotated for aspect, the main
3The entire test set annotated with 4 classes, the
1k test set also annotated with 4 classes.
category of aspect, and the polarity of the
opinion about the aspect. Systems had to
classify the opinion about the given aspect in
3 di erent polarity labels (positive,
negative, neutral).</p>
        <p>Participants were expected to submit up
to 3 experiments for each corpus, each in a
plain text le with the following format:
t w e e t i d n t a s p e c t n t p o l a r i t y
Allowed polarity values were p, neu and n.</p>
        <p>For evaluation, exact match with a single
label combining \aspect-polarity" was used.
Similarly to Task 1, the macro-averaged
version of Precision, Recall and F1, and
Accuracy were the evaluation measures, and
Macro-F1 were used for ranking the systems.
3</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Datasets</title>
      <p>TASS 2017 provides four datasets to the
participants for the evaluation of their systems.
Three of the datasets were used in previous
editions, and a new dataset was created for
TASS 2017, namely InterTass.</p>
      <p>The datasets will be made freely available
to the community after the workshop.4
3.1</p>
      <sec id="sec-3-1">
        <title>InterTASS</title>
        <p>International TASS Corpus (InterTASS ) is
a new corpus released this year for general
task (Task 1). The goal of the
organization of TASS is the creation of a corpus
of tweets written in the Spanish language
spoken in Spain and in di erent
HispanoAmerican countries. We release the rst
version of InterTASS in TASS 2017, which is
only composed of tweets posted in Spain and
written in the Spanish language spoken in
Spain.</p>
        <p>More than 500,000 tweets were collected,
from July 2016 to January 2017, using some
keywords. The downloaded set of tweets was
ltered out in order to meet the following
requirements:</p>
        <p>The language of the tweets must be
Spanish5,
each tweet must contain at least one
adjective,
the minimum length of each tweet must
be four words.</p>
        <p>4Further information for requesting the datasets
in: http://www.sepln.org/workshops/tass/.
5We used the python library langdetect.</p>
        <p>Then, the general sentiment of a random
selection of tweets was manually annotated
by ve annotators. We used a scale of 4
levels of polarity: positive (p), neutral (neu),
negative (n) and no sentiment tag (none).
Each tweet was nally annotated at least by
three annotators. When a tweet has the same
tag by two of more annotators, the process
end. If not, each annotator revised the tweet
again, until it has the same tag by two of
more annotators. The annotation resulted in
a corpus of 3,413 tweets, which was split into
three datasets: training, development and
test. Table 1 shows the size of each dataset
of InterTASS corpus.</p>
        <sec id="sec-3-1-1">
          <title>Corpus</title>
          <p>Training
Developement
Test
Total
#Tweets
1,008</p>
          <p>506
1,899
3,413</p>
          <p>Each tweet includes its ID (tweetid), the
creation date (date) and the user ID (user).
Due to restrictions in the Twitter API Terms
of Service,6 it is forbidden to redistribute a
corpus that includes text contents or
information about users. However, it is valid if those
elds are removed and instead IDs
(including Tweet IDs and user IDs) are provided.
The actual message content can be easily
obtained by making queries to the Twitter API
using the tweetid.</p>
          <p>The training set was released, so the
participants could train and validate their
models. The test corpus was provided without
any annotation and has been used to
evaluate the results. The InterTass statistics are
in Table 2.</p>
          <p>P
NEU
N
NONE
Total
&lt;tweet&gt;
&lt;tweetid&gt;768212591105703936&lt;/</p>
          <p>
            tweetid&gt;
&lt;user&gt;martitarey13&lt;/user&gt;
&lt;content&gt;@estherct209 jajajaja la
tuya y la d mucha gente seguro
!! Pero yo no puedo sin mi
melena me muero &lt;/content&gt;
&lt;date&gt;2016-08-23 22:25:29&lt;/date&gt;
&lt;lang&gt;es&lt;/lang&gt;
&lt;sentiment&gt;
&lt;polarity&gt;
&lt;value&gt;N&lt;/value&gt;
&lt;type&gt;AGREEMENT&lt;/type&gt;
&lt;/polarity&gt;
&lt;/sentiment&gt;
&lt;/tweet&gt;
The General Corpus of TASS has 68,000
tweets, written in Spanish by about 150
wellknown personalities and celebrities of the
world of politics, economy, communication,
mass media and culture, between November
2011 and March 2012. The details of the
corpus are described in
            <xref ref-type="bibr" rid="ref18 ref4 ref7">(Villena-Roman et al.,
2015; Garc a-Cumbreras et al., 2016)</xref>
            .
Figure 2 shows a tweet from the General Corpus
of TASS.
3.3
          </p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>Social-TV Corpus</title>
        <p>
          The Social-TV corpus was collected during
the 2014 Final of Copa del Rey
championship in Spain between Real Madrid and
F.C. Barcelona, played on 16 April 2014 at
Mestalla Stadium in Valencia. After
ltering out useless information a subset of 2.773
tweets was selected. The details of the corpus
are described in
          <xref ref-type="bibr" rid="ref18 ref4 ref7">(Villena-Roman et al., 2015;
Garc a-Cumbreras et al., 2016)</xref>
          .
        </p>
        <p>All tweets were manually annotated with
31 di erent aspects and its sentiment
polarity. It was randomly divided into training
set (1.773 tweets) and test set (1.000 tweets),
with a similar distribution of both aspects
and sentiments.</p>
        <p>Figure 3 shows a tweet from the Social-TV
corpus.
&lt;tweet id="456544894501146625"&gt;
Para mi, &lt;sentiment
aspect="JugadorIsco" polarity="P"&gt;ISCO&lt;/
sentiment&gt;
ha hecho un &lt;sentiment aspect="
Partido" polarity="P"&gt;partidazo&lt;/
sentiment&gt;.
&lt;sentiment aspect="Partido" polarity=
"P"&gt;El mejor partido&lt;/sentiment&gt;
desde que llego al
&lt;sentiment aspect="Equipo-Real_Madrid
" polarity="NEU"&gt;Real Madrid&lt;/
sentiment&gt;.
&lt;/tweet&gt;
STOMPOL (corpus of Spanish Tweets for
Opinion Mining at aspect level about
POLitics) is a corpus of Spanish tweets developed
for the research in opinion mining at aspect
level. Each tweet in the corpus has been
manually annotated by two annotators, and
a third one in case of disagreement, with the
sentiment polarity at aspect level.</p>
        <p>
          The corpus is composed of 1,284 tweets,
and has been divided into training set (784
tweets), which is provided for building and
validating the systems, and test set (500
tweets) that will be used for evaluation.
The details of the corpus are described
in
          <xref ref-type="bibr" rid="ref18 ref4">(Villena-Roman et al., 2015; Garc
aCumbreras et al., 2016)</xref>
          . Figure 4 shows a
tweet from the STOMPOL corpus.
&lt;tweet id="591172256971280385"&gt;
@rosadiezupyd lamenta que el #
&lt;sentiment aspect="Economia" entity="
Union_Progreso_y_Democracia"
polarity="N"&gt;empleo&lt;/sentiment&gt;
no termine de estabilizarse y
dice que el
&lt;sentiment aspect="Economia" entity="
Union_Progreso_y_Democracia"
polarity="N"&gt;#paro&lt;/sentiment&gt; "
sigue siendo dramA~ tico" http://t
.co/1xdS3UjJWk #EPA
&lt;/tweet&gt;
Most of the systems submitted in TASS 2017
are based on the use of deep learning
techniques as the state-of-the-art in SA in
Twitter. However, some of the systems are based
on traditional machine learning methods and
others are meta-classi ers whose inputs are
the output of deep learning systems and
traditional machine learning algorithms. We
depict the main features of the systems
submitted in the subsequent paragraphs.
        </p>
        <p>Table 3, Table 4 and Table 5 show the
results reached by the submitted systems in
Task 1, using the test sets of InterTASS
corpus and the General Corpus (full test and 1k
test). Table 6 and Table 7 shows the results
reached by the submitted systems in Task 2,
using the test sets of Social-TV corpus and
STOMPOL corpus respectively.</p>
        <p>
          <xref ref-type="bibr" rid="ref6">Hurtado, Pla, and Gonzalez (2017</xref>
          )
participated in the two tasks. They submitted
the same system for both tasks, and the only
di erence between the tasks lies in the
characteristics of the input. The input of the
rst task is the entire tweet, meanwhile the
input in the second task is the context of
the aspects, which is previously determined.
The authors created a set of domain-speci c
word embeddings following the approach of
          <xref ref-type="bibr" rid="ref16">Tang (2015)</xref>
          . The former word embeddings
set is jointly used with a general-domain set
of embeddings to represent the tokens of the
tweets. The authors evaluated three di erent
neural networks architectures, the rst one is
a multilinear perceptron (MLP), the second
encodes the tweets with a convolutional
recurrent neural network (CNN) and the third
one with a long-short term memory (LSTM)
recurrent neural network (RNN). The
performance of each con guration depends on the
training set used.
        </p>
        <p>
          <xref ref-type="bibr" rid="ref3">Ceron-Guzman (2017)</xref>
          presented an
ensemble classi er system for the rst task.
The author generated quantitative features
from the tweets, such as the number of
words in upper case and the number of words
with repeated letters. Moreover, the system
used lists of opinion bearing words like iSOL
          <xref ref-type="bibr" rid="ref9">(Molina-Gonzalez et al., 2013)</xref>
          , as well as the
inversion of the polarity of words following a
window shifting approach for negation
handling. The base classi ers of the ensemble
system were Logistic Regression and SVM.
The system followed two ensemble strategies,
namely stacking and maximum classi cation
con dence. The maximum con dence
strategy outperformed the stacking strategy and
it reached the highest accuracy value with the
test set of the InterTASS dataset.
        </p>
        <p>
          <xref ref-type="bibr" rid="ref10">Montan~es Salas et al. (2017</xref>
          ) used the
classi er FastText
          <xref ref-type="bibr" rid="ref7">(Joulin et al., 2016)</xref>
          for
only classifying the test set of the InterTASS
dataset. The authors performed a traditional
pre-processing to the input tweets, however
the substitution of words with a emotional
meaning by their synonyms from a list of
words with a emotional meaning
          <xref ref-type="bibr" rid="ref2">(Bradley
and Lang, 1999)</xref>
          stands out.
        </p>
        <p>
          <xref ref-type="bibr" rid="ref15">Rosa et al. (2017)</xref>
          participated in the two
tasks. Concerning the rst task, the authors
submitted three systems: 1) a SVM classi er
with word embeddings and quantitative
linguistic properties as features; 2) a deep
neural network grounded on the use of a CNN for
encoding the input tweets; and 3) the
combination of the two previous classi ers by the
selection of the output class with a higher
probability mean from the two previous
classi ers. The third strategy outperformed the
other ones in two test sets of Task 1.
Regarding the Task 2, the authors submitted
two SVM classi ers mainly grounded on the
use word embeddings.
        </p>
        <p>
          <xref ref-type="bibr" rid="ref5">Garc a-Vega et al. (2017</xref>
          ) submitted four
systems for the classi cation of the test set
of the InterTASS dataset. The rst two
systems are a SVM classi er that uses
wordembeddings as features. The di erence
between these two systems lies in the use of
additional tweets from the users of the training
set. The intention of the authors was the
in
        </p>
        <sec id="sec-3-2-1">
          <title>System</title>
          <p>ELiRF-UPV-run1
RETUYT-svm cnn
ELiRF-UPV-run3
ITAINNOVA-model4
jacerong-run-2
jacerong-run-1
INGEOTECevodag 001
RETUYT-svm
tecnolengua-sent only
ELiRF-UPV-run2
ITAINNOVA-model3
RETUYT-cnn3
SINAI-w2v-nouser
tecnolengua-run3
tecnolenguasent only xed
ITAINNOVA-model2
LexFAR-run3
LexFAR-run1
jacerong-run-3
SINAI-w2v-user
INGEOTECevodag 002
OEG-victor2
OEG-victor0
OEG-laOEG
LexFAR-run2
GSI-sent64-189
SINAI-embed-rnn2
GSI-sent64-149-ant-2
GSI-sent64-149-ant
troduction of the use of language of each user
in the classi cation. The two last systems are
deep neural networks grounded on the use of
LSTM RNN for the encoding of the
meaning of the input tweets. The rst neural
architecture uses word embeddings as features,
and the second one the TF-IDF value of each
word of the tweets.</p>
          <p>
            <xref ref-type="bibr" rid="ref8">Moctezuma et al. (2017)</xref>
            participation
was based on an ensemble of SVM
classiers combined into a non-linear model
created with genetic programming to tackle
the task of global polarity classi cation at
tweet level. They used B4MSA algorithm,
a proposed entropy-based term weighting
scheme, which is a baseline supervised
learning system based on the SVM classi er, an
entropy-based term-weighting scheme.
Additionally they used EvoDAG, a GP system
that combines all decision values predicted by
B4MSA systems. They also used two
external datasets to train the B4MSA algorithm.
          </p>
          <p>
            <xref ref-type="bibr" rid="ref12">Navas-Loro and Rodr guez-Doncel (2017</xref>
            )
participated only on Task 1. They
experimented with two classi er algorithms,
Multinominal Nave Bayes and Sequential Minimal
Optimization for SVM. Furthermore they
used morphosyntactic analyses for negation
detection, along with the use of lexicons and
dedicated preprocessing techniques for
detecting and correcting frequent errors and
expressions in tweets.
          </p>
          <p>
            <xref ref-type="bibr" rid="ref1">Araque et al. (2017)</xref>
            have proposed, for
Task 1, a RNN architecture composed of
LSTM cells followed by a feed-forward
network. The architecture makes use of two
di erent types of features: word embeddings
and sentiment lexicon values. The recurrent
architecture allows them to process text
sequences of di erent lengths, while the lexicon
inserts directly into the system sentiment
information. Two variations of this
architecture were used: a LSTM that iterates over
the input word vectors, and on the other
M-F1
0.562
0.557
0.555
0.526
lexicons are represented by the bag-of-word
model and they are weighted using Term
Frequency measure at tweet level.
          </p>
          <p>
            <xref ref-type="bibr" rid="ref11">Moreno-Ortiz and Perez Hernandez
(2017</xref>
            ) have proposed, for Task 1, a
classi cation model based on the Lingmotif
Spanish lexicon, and combined this with a
number of formal text features, both general
and CMC-speci c, as well as single-word
keywords and n-gram keywords. They use
logistic regression classi er trained with the
optimal set of features, SVM classi er on
the same features set. Sentiment features
are obtained with Lingmotif SA engine
(sentiment feature set, text feature set and
keywords feature set).
5
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Conclusion and Future work</title>
      <p>TASS was the rst workshop about sentiment
analysis focused on the processing of texts
written in Spanish. In this edition, 11 teams
participated with a total of 123 runs, most of
them in the InterTASS task.</p>
      <p>Anyway, the released corpora and the
reports from participants will for sure be
helpful for other research groups approaching
these tasks.</p>
      <p>The future work will mainly go in two
directions. On the one hand, the
organization of one o more shared-tasks for the
treatment of semantic information in Spanish like
those mentioned above (argumentation
mining, irony detection and stance classi cation).
On the other hand, the extension and
improvement of the InterTASS corpus. This
corpus has been received with great
interest, almost 90% of the experiments have been
developed in the rst task, so an exhaustive
analysis of the behavior of the corpus in this
task will shows the right way for a new
version of the corpus.</p>
      <p>TASS corpora will be released after the
workshop for free use by the research
comhand a combination of the input word vectors
and polarity values from a sentiment lexicon.</p>
      <p>Tume Fiestas and Sobrevilla Cabezudo
(2017) have proposed, for Task 2, an
approach based on word embeddings for
polarity classi cation at aspect-level. They used
word embeddings to get the similarity
between words selected from a training set and
make a model to classify each polarity of each
aspect for each tweet. Their results show that
the more tweets are used, the better accuracy
is obtained.</p>
      <p>
        <xref ref-type="bibr" rid="ref14">Reyes-Ortiz et al. (2017)</xref>
        have proposed,
for Task 1, a system that uses machine
learning, vector support machines algorithm and
lexicons of semantic polarities at the level of
lemma for Spanish. Features extracted from
munity.
      </p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgement</title>
      <p>
        This research work is partially supported by
the project REDES
        <xref ref-type="bibr" rid="ref16">(TIN2015-65136-C2-1-R)</xref>
        and a grant from the Fondo Europeo de
Desarrollo Regional (FEDER).
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Araque</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Barbado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. F.</given-names>
            <surname>Sanchez-Rada</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C. A.</given-names>
            <surname>Iglesias</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Applying recurrent neural networks to sentiment analysis of spanish tweets</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEUR-WS.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Bradley</surname>
            ,
            <given-names>M. M.</given-names>
          </string-name>
          and
          <string-name>
            <given-names>P. J.</given-names>
            <surname>Lang</surname>
          </string-name>
          .
          <year>1999</year>
          .
          <article-title>Affective norms for english words (anew): Stimuli, instruction manual, and a ective ratings</article-title>
          .
          <source>Technical report</source>
          , Center for Research in Psychophysiology, University of Florida.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Ceron-Guzman</surname>
            ,
            <given-names>J. A.</given-names>
          </string-name>
          <year>2017</year>
          .
          <article-title>Classier ensembles that push the state-of-the-art in sentiment analysis of spanish tweets</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Garc</surname>
            a-Cumbreras,
            <given-names>M. A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Villena-Roman</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.</surname>
          </string-name>
          <article-title>Mart nez-</article-title>
          <string-name>
            <surname>Camara</surname>
            ,
            <given-names>M. C.</given-names>
          </string-name>
          <article-title>D az-</article-title>
          <string-name>
            <surname>Galiano</surname>
            ,
            <given-names>M. T.</given-names>
          </string-name>
          <string-name>
            <surname>Mart</surname>
            n-Valdivia, and
            <given-names>L. A.</given-names>
          </string-name>
          <string-name>
            <surname>Uren</surname>
          </string-name>
          ~a Lopez.
          <year>2016</year>
          .
          <article-title>Overview of tass 2016</article-title>
          .
          <source>In TASS 2016: Workshop on Sentiment Analysis at SEPLN</source>
          , pages
          <volume>13</volume>
          {
          <fpage>21</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Garc</surname>
            a-Vega,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Montejo-Raez</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. C.</surname>
          </string-name>
          <article-title>D az-</article-title>
          <string-name>
            <surname>Galiano</surname>
            , and
            <given-names>S. M.</given-names>
          </string-name>
          <string-name>
            <surname>Jimenez-Zafra</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Sinai en tass 2017: Clasi cacion de la polaridad de tweets integrando informacion de usuario</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Hurtado</surname>
            , L.-
            <given-names>F.</given-names>
            , F.
          </string-name>
          <string-name>
            <surname>Pla</surname>
            , and
            <given-names>J.-A.</given-names>
          </string-name>
          <string-name>
            <surname>Gonzalez</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Elirf-upv en tass 2017: Analisis de sentimientos en twitter basado en aprendizaje profundo</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Joulin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Grave</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Bojanowski</surname>
          </string-name>
          , and
          <string-name>
            <given-names>T.</given-names>
            <surname>Mikolov</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Bag of tricks for efcient text classi cation</article-title>
          .
          <source>arXiv preprint arXiv:1607</source>
          .
          <fpage>01759</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Moctezuma</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Gra</surname>
          </string-name>
          , S. MirandaJimenez,
          <string-name>
            <given-names>E. S.</given-names>
            <surname>Tellez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Coronado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. N.</given-names>
            <surname>Sanchez</surname>
          </string-name>
          , and J.
          <string-name>
            <surname>Ortiz-Bejar</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>A genetic programming approach to sentiment analysis for twitter: Tass'17</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEURWS.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Molina-Gonzalez</surname>
            ,
            <given-names>M. D.</given-names>
          </string-name>
          , E. Mart nezCamara,
          <string-name>
            <surname>M.-T.</surname>
            Mart -Valdivia, and
            <given-names>J. M.</given-names>
          </string-name>
          <string-name>
            <surname>Perea-Ortega</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Semantic orientation for polarity classi cation in spanish reviews</article-title>
          .
          <source>Expert Systems with Applications</source>
          ,
          <volume>40</volume>
          (
          <issue>18</issue>
          ):
          <volume>7250</volume>
          {
          <fpage>7257</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Montan~es Salas</surname>
            ,
            <given-names>R. M.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>R. del Hoyo</given-names>
            <surname>Alonso</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Vea-Murgu a</surname>
          </string-name>
          <string-name>
            <surname>Merck</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. Aznar</given-names>
            <surname>Gimeno</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F. J.</given-names>
            <surname>Lacueva-Perez</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>FastText como alternativa a la utilizacion de deep learning en corpus pequen~os</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Moreno-Ortiz</surname>
            ,
            <given-names>A</given-names>
          </string-name>
          . and
          <string-name>
            <given-names>C. Perez</given-names>
            <surname>Hernandez</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Tecnolengua lingmotif at tass 2017: Spanish twitter dataset classi cation combining wide-coverage lexical resources and text features</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEUR-WS.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Navas-Loro</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <given-names>V.</given-names>
            <surname>Rodr</surname>
          </string-name>
          guez-Doncel.
          <year>2017</year>
          . Oeg at tass 2017:
          <article-title>Spanish sentiment analysis of tweets at document level</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEUR-WS.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Pang</surname>
            ,
            <given-names>B. and L.</given-names>
          </string-name>
          <string-name>
            <surname>Lee</surname>
          </string-name>
          .
          <year>2008</year>
          .
          <article-title>Opinion mining and sentiment analysis</article-title>
          .
          <source>Foundations and Trends in Information Retrieval</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          -2):1{
          <fpage>135</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Reyes-Ortiz</surname>
            ,
            <given-names>J. A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Paniagua-Reyes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Priego-Sanchez</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Tovar</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Lexfar en la competencia tass 2017: Analisis de sentimientos en twitter basado en lexicones</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEUR-WS.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Rosa</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Chiruzzo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Etcheverry</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Castro</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Retuyt en tass 2017: Analisis de sentimientos de tweets en espan~ol utilizando svm y cnn</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Tang</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          <year>2015</year>
          .
          <article-title>Sentiment-speci c representation learning for document-level sentiment analysis</article-title>
          .
          <source>In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM '15</source>
          , pages
          <fpage>447</fpage>
          {
          <fpage>452</fpage>
          , New York, NY, USA. ACM.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <given-names>Tume</given-names>
            <surname>Fiestas</surname>
          </string-name>
          ,
          <string-name>
            <surname>F.</surname>
          </string-name>
          and
          <string-name>
            <given-names>M. A. Sobrevilla</given-names>
            <surname>Cabezudo</surname>
          </string-name>
          .
          <year>2017</year>
          . C100tpucp at tass 2017:
          <article-title>Word embedding experiments for aspect-based sentiment analysis in spanish tweets</article-title>
          .
          <source>In Proceedings of TASS 2017: Workshop on Sentiment Analysis at SEPLN co-located with 33nd SEPLN Conference (SEPLN</source>
          <year>2017</year>
          ), volume
          <volume>1896</volume>
          <source>of CEUR Workshop Proceedings</source>
          , Murcia, Spain, September. CEUR-WS.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Villena-Roman</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Garc</surname>
          </string-name>
          a-Morera,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Garc</surname>
          </string-name>
          a-Cumbreras,
          <string-name>
            <surname>E.</surname>
          </string-name>
          <article-title>Mart nez-</article-title>
          <string-name>
            <surname>Camara</surname>
            ,
            <given-names>M. T.</given-names>
          </string-name>
          <string-name>
            <surname>Mart</surname>
            n-Valdivia, and
            <given-names>L. A.</given-names>
          </string-name>
          <string-name>
            <surname>Uren</surname>
          </string-name>
          ~a Lopez.
          <year>2015</year>
          .
          <article-title>Overview of tass 2015</article-title>
          .
          <source>In TASS 2015: Workshop on Sentiment Analysis at SEPLN</source>
          , pages
          <volume>13</volume>
          {
          <fpage>21</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Villena-Roman</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Garc</surname>
          </string-name>
          a-Morera,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lana-Serrano</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>J. C.</given-names>
            <surname>GonzalezCristobal</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Tass 2013 - a second step in reputation analysis in spanish</article-title>
          .
          <source>Procesamiento del Lenguaje Natural</source>
          ,
          <volume>52</volume>
          (
          <issue>0</issue>
          ):
          <volume>37</volume>
          {
          <fpage>44</fpage>
          ,
          <string-name>
            <surname>March</surname>
          </string-name>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Villena-Roman</surname>
            , J.,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Lana-Serrano</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <article-title>Mart nez-Camara, and</article-title>
          <string-name>
            <given-names>J. C.</given-names>
            <surname>GonzalezCristobal</surname>
          </string-name>
          .
          <year>2013</year>
          . Tass - workshop
          <source>on sentiment analysis at sepln. Procesamiento del Lenguaje Natural</source>
          ,
          <volume>50</volume>
          :
          <fpage>37</fpage>
          {
          <fpage>44</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>Jimenez</given-names>
            <surname>Zafra</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Tass 2014 - the challenge of aspect-based sentiment analysis</article-title>
          .
          <source>Procesamiento del Lenguaje Natural</source>
          ,
          <volume>54</volume>
          (
          <issue>0</issue>
          ):
          <volume>61</volume>
          {
          <fpage>68</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>