This paper presents three approaches for classifying the sentiment of tweets for di erent Spanish variants in the TASS 2019 challenge. The classi ers are based on Multilayer Perceptron (MLP), Long Short Term Memory networks (LSTM), and transfer learning using BERT. Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). IberLEF 2019, 24 September 2019, Bilbao, Spain.
Sentiment analysis in tweets is an interesting task due to the large volume of
information generated every day, the subjective nature of most messages, and the
easy access to this material for analysis and processing. Speci c tasks related to
this eld have been organized for several years now: the International Workshop
on Semantic Evaluation (SemEval) includes a task on Tweets Sentiment Analysis
since 2013 [
As in many NLP areas, in the last years most of the works on sentiment
analysis have incorporated techniques based on Deep Learning and Word
Embeddings, in search of improving results. In recent editions of the TASS shared
tasks (2017 and 2018), the majority of participating systems rely on di erent
neural network models and on the use of word embeddings [
In TASS 2018 (task 1) the best results were obtained by systems which used
deep learning [
In TASS 2017 [
SemEval 2018 [
In this paper we describe three di erent approaches for Spanish tweet
classication presented by the RETUYT-InCo team for the TASS 2019 [
In this approach we considered multiple variants to perform the classi cation of the sentence word vectors mean. This kind of sentence representation is not aware of the sentence order but it can give surprisingly good results. We performed the classi cation through layered fully connected neural networks and support vector machines.
Model Features We attempted to improve the classi cation performance including the following additional features.
{ Sentiment Lexicon: We considered a sentiment lexicon constituted by two sets of words: positive and negative. We added two dimensions to the input with the amount of words in the tweet of each set. { Tweet split: We split the input tweets into phrases and obtained the mean vector through the mean vector of each phrase. { Dimensionality Reduction: We reduced the input vector dimensionality using Principal Component Analysis (PCA). { Crosslingual exclusion: In the crosslingual task we excluded an additional country from the training data. Experiments Performed We trained multiple classi ers to achieve a good model setting observing an overall better performance with MLP than SVM. Regarding dimensionality reduction we used PCA trying many numbers of principal components, however, we could not obtain any considerable performance improvement with this approach.
In the crosslingual experiments, we trained models excluding each country and we observed a performance improvement when excluding the Peru corpus from the training data (for all the Spanish variants). Note also that the results obtained for the Peru corpora are lower than the ones obtained for the remaining variants.
Regarding the MLP we considered batch sizes of 100 and 200 and di erent con guration of hidden layer sizes. 2.2
This approach relies on the transfer learning from a pretrained Spanish BERT [
User. { URLs: Each url was replaced by the token URL. { Hashtags: The hash symbols (#) were deleted. { Laugh normalization: The laugh tokens (e.g. ja, jajj, jajjajaj, etc.) were replaced by jaja. { Numbers: The numbers (e.g. 1,2,3,...) were replaced by its word names (uno, dos, tres, ...) Fine Tuning In the experiments we performed four ne tuning strategies: only the encoder, only the classi er, both sequentially ( rst encoder and then classi er) and both jointly. The best results were obtained when the encoder and classi er are ne tuned jointly.
The best performances were obtained when the model was trained for 5 epochs balancing the training corpus according to its polarity. When the training corpus is not balanced, the model tends to over t and give the majority class.
This approach performed better in the crosslingual than in the monolingual tasks. Probably, this was because of the reduced training data for the monolingual task. 2.3
The third approach is based on the use of fastText embeddings [
The tweets were normalized, as in [
FastText was used to train 300 dimension vectors, using the training corpus of each variant for monolingual experiments, and the whole training corpus for crosslingual experiments.
The best results on the development corpora were obtained by an LSTM network with two LSTM layers, and the following con guration: { drop out: 0.5 { number of words in input: 20 { number of neurons: 256 { batch size: 256 { embeddings adjust: TRUE { using the early stopping and model checkpoint techniques 3
Five di erent corpora considering ve Spanish variants were used for this task: Spain (ES), Costa Rica (CR), Peru (PE), Uruguay (UY), and Mexico (MX). Furthermore, the systems could be trained with training data for the corresponding Spanish variant (monolingual case), or they could be trained using data from other variants (crosslingual case). We decided to submit the best results for each approach on each of the variants and training combinations. Despite we performed some experiments on the Uruguayan dataset, we decided not to send our results on this corpus because we participated in the corpus annotation process. The results we obtained on the test corpora are shown in table 1.
For the monolingual datasets, the best results were achieved by the rst approach, based on MLP. On the other hand, for crosslingual datasets, the system based on the BERT model performed better (except for the case of Costa Rica).
Comparing to the systems developed by the other teams participating in the TASS share task, we obtained some interesting results. Our MLP approach was ranked rst on the corpus from Costa Rica for the monolingual task, and our BERT based approach was ranked rst on the corpus from Spain for the crosslingual task.
Concerning the Uruguayan datasets, we had a good performance evaluating on the development corpus, reaching a Macro-F of 0.50 for the monolingual task (BERT based approach) and 0.48 for the crosslingual task (MLP based approach). 4
We presented three approaches for TASS 2019 about classifying the sentiment of tweets in di erent Spanish variants. The approaches we used are: MLP using word embedding centroids and manually crafted features, transfer learning based on the BERT model, and LSTM using fastText word embeddings.
Our MLP based approach achieved good results in monolingual experiments while the BERT based system performed better in the crosslingual task. Probably this approach needs a bigger dataset for training.