The Irony Detection in Spanish Variants (IroSvA'19) shared task aims at investigating the recognition of irony in Twitter messages in three di erent Spanish variants: from Spain , Mexico, and Cuba. This paper describes our approach to the shared task: a Neural Network system based on a simple bidirectional LSTM (biLSTM) model. Since we have developed the system in the context of other related IberLEF 2019 shared tasks, we train multiple models simultaneously sharing some of the layers of the neural architecture between them. Furthermore, we also have applied a method for data augmentation given the scarcity of resources which were available.
Figurative language is an important device for communication in social media allowing people to express themselves in unexpected ways. Due to its signi cance, investigating gurative language such as irony has gathered considerable attention from various disciplines.
The main problem with automatic detection of irony is that ironic messages aim to convey the opposite meaning of what is literally said or written. This e ect is sometimes achieved using humor as a key device. For various natural language processing (NLP) applications, irony detection has a great potential, especially where semantic analysis is of concern.
IroSvA focuses on the detection of irony in short messages (tweets and news
comments) written in Spanish and with respect to a given context [
Therefore, in addition to detect if a text is ironic or not depending on the context, the problem is also related to the way irony is expressed in distinct Spanish variants. This paper describes a neural network for irony detection for di erent dialects. The rest of the paper is organized as follows: In section 2, we present an overview of the related work for irony detection. In Section 3, we describe our model and the di erences between di erent runs for each sub-task. In Section 4, we provide the results and discuss the performance of the system. Lastly, in Section 5, we give the conclusions. 2
Previous research on automatic irony detection follows two main approaches:
(i) rule-based or (ii) machine-learning based methods [
Apart from rule based systems, there are machine learning approaches with
handcrafted feature engineering and deep-learning systems. Most of the machine
learning approaches start by calculating the features with various methods to
then use a machine learning algorithm to classify the text as ironic or not.
Gonzalez-Ibanez et. al investigated the contribution of linguistic and pragmatic
features and found that pragmatic features such as positive emoticons (e.g.
smileys), negative emoticons (e.g. frowning faces) and user mentions { indicating
that a message is addressed to some speci c entity { are distinctive features
[
More recently, among the systems that participated in a shared task on irony
detection in Italian, EVALITA 2018, innovative deep learning approaches showed
high performance, with the best performing system based on a deep learning
approach [
On the other hand, some previous works focusing only on detecting the
language variety of the tweets have highlighted the challenges researchers face [
The corpus that was provided by the shared task organizers consist of 9,000 short messages about di erent topics written in Spanish as 3,000 from Cuba, 3,000 from Mexico and 3,000 from Spain. The messages are annotated according to being ironic or not. Roughly, 80% of the corpus is given for training purposes and the remaining 20% is given for testing purpose.
In this work, we presented a neural network based on a simple bidirectional
LSTM (biLSTM) model with two dense layers at the end to detect ironic
messages based on our previous work [
In this scenario, we de ned an Embedding layer for each Spanish variant. Classi cation tasks with the same Spanish variant used the same Embedding layer during the training process. For instance, the embedding layer related to the Spanish from Mexico was used by the MEX-A3T task, the Mexican part of the TASS 2019 task and the tweets written in the Spanish language spoken in Mexico from IroSvA. Furthermore, all task shared the biLSTM layer during training.
In Figure 1 a simpli ed schema of our shared model can be seen. In the following we explain how the model works in one speci c classi cation task. In order to train all task at the same time, we have divided each data set into the same number of batches. Then, during the training, a batch of data is randomly selected and it is used to train its speci c model (sharing the embedding and BiLSTM layers with other models). In this sense, we consider one epoch when all batches from all task were trained.
First, the text of the tweets were tokenized, removing punctuation marks, and keeping emojis and full hashtags since they can contribute to de ne the meaning of a tweet.
Second, the embedding layer transforms each element in the tokenized tweet into a low-dimension vector. The embedding layer, composed of the vocabulary of the task, was randomly initialized from a uniform distribution (between -0.8 and 0.8 values and with 100 dimensions). The initialized embedding layer was updated with the word vectors included in a pre-trained model from Regional Embeddings, which provides FastText word embeddings for Spanish language variations.
Then, a biLSTM layer gets high-level features from previous embeddings. A disadvantage of seq2seq models (such as LSTM) is that they compress all information into a xed-length vector, causing the incapability of remembering long tweets. To overcome the limitation of xed-length vector keeping relevant information from long tweet sequences, we added an attention layer producing a weight vector and merge word-level features from each time step into a tweetlevel feature vector, by multiplying the weight vector. Finally, the tweet-level feature vector produced by the previous layers is used for classi cation task by two fully-connected (dense) layers.
Moreover, to be able to mitigate over tting problem we applied dropout regularization. Dropout operation sets randomly to zero a proportion of the hidden units during forward propagation, creating more generalizable representations of data. The dropout rate was set to 0.5 in all cases. We generated 2 submissions per Spanish variant in this task. In the rst submission, we trained 4 models at the same time: for IroSvA from Cuba, Spain, Mexico and for the MEX-A3T task (Figure 1). In the second one, we trained 10 models at the same time: three Spanish variants from the Irosva task, one from the MEX-A3T and one HAHA tasks, and ve from the TASS task.
Results of our submissions for each task are given in the Table 1 together with the baselines proposed by the organizers. In this paper, we have presented our results from the participation in the IroSvA task from the IberLEF 2019. We have investigated multi-task learning on neural networks with di erent tasks. Our results were close to the baselines presented by the organizers. As commented before, each Spanish variation data set from IroSva only includes 2,400 tweet for training, in comparison with other tasks (MEX-A3T or HAHA with 7,700 and 30,000 tweets, respectively). We divided each data set into the same number of batches, then, batches related to the IroSva task contain less information to train than batches from MEX-A3T or HAHA. In this case, the updates in the models produced by the IroSva batches could be diminished by other batches. In any case, our model related to each Spanish variant from IroSva and mostly trained with other tasks achieved results compared with the baselines, opening new research lines with the purpose of improving the relevance of the IroSva task during the multi-task training (e.g. data augmentation in the IroSva task). In addition, we want to test di erent types of neural networks (e.g. convolutions or combinations of convolutions and LSTM layers) and share more layers between task. Finally, we also consider that the integration of linguistic features (e.g. word frequency, POS tags and word shape) and metadata (e.g. whether a tweet is a response to another tweet) can represent useful contextual information to improve our performance.