This paper describes the participation of LaSTUS/TALN team in the shared task Sentiment Analysis at SEPLN (TASS) organized in the context of IberLEF 2019. TASS focuses on the classi cation of tweets written in the Spanish language (from Spain, Peru, Costa Rica, Uruguay and Mexico) with respect to their polarity or sentiment. This year TASS proposes two sub-tasks: monolingual and cross-lingual sentiment analysis. This paper presents a deep learning approach based on bidirectional LSTM (biLSTM) models to face both sub-tasks. The paper reports and discusses the o cial results achieved by our team.
Sentiment analysis is the process of detecting subjective information of a given
text such as whether the text expresses a positive, negative or neutral opinion.
Sentiment analysis is widely used in several application areas. For instance,
private companies or political organizations are interested in knowing what their
clients think about their product or services [
TASS at IberLEF 20191 focuses on the evaluation of polarity classi cation
systems of tweets written in the Spanish language spoken in Spain, Peru, Costa
Rica, Uruguay and Mexico [
This paper describes a neural network for sentiment analysis of Tweets in Spanish. The rest of the paper is organized as follows: In section 2, we present an overview of the related work for sentiment analysis, speci cally on Spanish. In Section 3, we describe our model. In Section 4, we provide the results and discuss the performance of the system. Lastly, in Section 5, we give the conclusions. 2
Previous research on Twitter sentiment analysis can be considered in two
categories: supervised approaches and lexicon-based approaches. Where supervised
methods are of concerned, the algorithms used are based on classi ers such as
Random Forest, Support Vector Machine, Naive Bayes with diverse features such
as Part-Of-Speech (POS) tags, N-grams, hashtags, retweets, emoticons [
In the previous edition of TASS (in 2018) [
The participants were provided with a training and a development corpora and several test corpora. All the corpora are annotated with 4 di erent levels of opinion intensity as positive, negative, neutral or none (P, N, NEU, NONE).
We address the problem with a neural network based on two bidirectional LSTM (biLSTM) models with two dense layers at the end. In Figure 1 a simplied schema of our shared model can be seen.
First, the tweets were preprocessed removing punctuation marks and keeping emojis and full hashtags since they can contribute to de ne the meaning of a tweet, and then, the tweets were tokenized.
Second, the embedding layer transforms each element in the tokenized tweet into a low-dimension vector. The embedding layer was randomly initialized from a uniform distribution (between -0.8 and 0.8 values and with 100 dimensions). In addition, the initialized embedding layer was updated with the corresponding word vectors related to Spanish variant to predict, which were updated during the training. These word vectors are included in a pre-trained model from Regional Embeddings 2, which provides FastText word embeddings for Spanish language variations.
Then, two subsequent biLSTM layers get high-level features from previous
embeddings with 128 and 64 units, respectively. A disadvantage of LSTM models
is that they compress all information into a xed-length vector, causing the
incapability of remembering long tweets. To overcome the limitation of xed-length
vector keeping relevant information from long tweet sequences, after biLSTMs,
we added an attention layer producing a weight vector and merge word-level
features from each time step into a tweet-level feature vector, by multiplying the
weight vector [
Moreover, to be able to mitigate over tting problem we applied dropout regularization. Dropout operation sets randomly to zero a proportion of the hidden units during forward propagation, creating more generalizable representations of data. In the model, we employ dropout on the embeddings and biLSTM layers. The dropout rate was set to 0.5 in all cases. Finally, the model was compiled using the Adam optimizer and the categorical cross-entropy as loss function. 4
In the Subtask 1 (monolingual sentiment analysis), we used the training and test dataset for each language (ES-Spain, PE-Peru, CR-Costa Rica, UR-Uruguay and MX-Mexico). For this Subtask, our results have been ranked between third and fth positions depending on the Spanish variant (see Table 1).
On the other hand, in the Subtask 2 (cross-lingual sentiment analysis), we trained our model using all datasets other than the test dataset. For example, to predict results in Spanish (ES), we trained with the data for the following Spanish variants: PE-Peru, CR-Costa Rica, UR-Uruguay and MX-Mexico. In this case, we have achieved better results, between the second and third positions depending on the Spanish variant (see Table 2). 5
In this paper, we presented our results for the participation to TASS task of IberLEF 2019. We described and evaluated our system which is based on two 2 https://github.com/INGEOTEC/ biLSTM models with an Attention layer, to classify the tweet in 4 di erent levels of opinion intensity (P, N, NEU, NONE). Regarding the results of the TASS task, we have achieved better results in the cross-lingual sub-task, although the model has been trained with di erent Spanish variants, there was more data to learn the classi cation than the monolingual task. In Table 1 and Table 2, we can also observe the best system of the task. Our results are usually close to the winning system, indicating the di culty of the task. Due to time constraints, we were not able to perform an error analysis, for that reason, in future work, we will work in a detailed error analysis in order to understand the limitations of our approach. Furthermore, more detailed analyses on integration of linguistic annotations into neural network and other models (such as convolution) can be considered in order to improve the performance of the model.
Our work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM2015-0502). We thank to reviewers for their constructive comments.