=Paper=
{{Paper
|id=Vol-2421/IroSvA_paper_7
|storemode=property
|title=Computational Models for Irony Detection in Three Spanish Variants
|pdfUrl=https://ceur-ws.org/Vol-2421/IroSvA_paper_7.pdf
|volume=Vol-2421
|authors=Simona Frenda,Viviana Patti
|dblpUrl=https://dblp.org/rec/conf/sepln/FrendaP19
}}
==Computational Models for Irony Detection in Three Spanish Variants==
https://ceur-ws.org/Vol-2421/IroSvA_paper_7.pdf
Computational Models for Irony Detection in
Three Spanish Variants
Simona Frenda1,2 and Viviana Patti1
1
Dipartimento di Informatica, Università degli Studi di Torino, Italy
2
PRHLT Research Center, Universitat Politècnica de València, Spain
{frenda, patti}@di.unito.it
Abstract. The lack of understanding of figurative language online, such
as ironic messages, is a common cause of error for systems that analyze
automatically the users’ opinions online detecting sentiment, emotions
or stance. In order to deal with this problem of automatic processing of
natural language, IroSvA shared task at IberLef 2019 asks participants to
detect, for the first time, irony in short texts written in Spanish language,
considering the three linguistic variants from Spain, Mexico and Cuba.
Another novelty of this task is the presence of labels specifying the con-
text of the utterance, such as current political or social issues discussed
online. In the context of this shared task, we approached irony detection
in Spanish short texts trying to exploit the provided topic information.
In addition, we investigated the usefulness of stylistic, lexical and affec-
tive features during the development of the irony detection models for
the three Spanish variants. Experimental results and further analyses al-
low to shed some light on the analogies and differences in the expression
of irony in the three variants, and suggest new research directions, in a
perspective of comparison with other languages.
Keywords: Irony Detection · Spanish · Mexican Spanish · Cuban Span-
ish · Linguistic Analysis · Affective features
1 Introduction
Irony is a special figurative device frequently used in speech and everyday life
to communicate something that is different or contrary to what is literally said
[29]. Irony has different functions in communications, which have been studied
so far in the context of different disciplines such as linguistics, psychology and
philosophy. Some of these studies highlight that one of the principal trait of the
use of irony consists in emphasizing unexpected occurrences especially amusing
the readers [1,44].
Copyright c 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0). IberLEF 2019, 24 Septem-
ber 2019, Bilbao, Spain.
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
Other studies focus on a very common type of verbal irony, sarcastic irony,
which is typically featured by a sharper tone of critique and by the speaker’s
intent to convey scorn or insult [14]. The lack of understanding of irony could
deceive the reader into thinking that the transmitted message is true. Especially
in texts produced by users online, the lack of context and the shortness of the
message often make the correct interpretation more difficult for humans and
especially for the machine.
In the Big Data era, the automatic understanding of contents online is one
of the current purposes of the majority of companies or political organizations
that want to analyze the opinions of the users about products, subjects and
individuals. The presence of irony, as well as sarcasm, affects the interpretation
of the real opinions of the users, and impedes the correct functioning of systems
of sentiment analysis, stance or emotion detection.
For this reason, especially in the campaigns of evaluations of automatic sys-
tems of Natural Language Processing (NLP), various shared tasks on irony de-
tection have been proposed. The majority of these competitions provide corpora
in English [18,45], but recently the analysis of irony is extended to other lan-
guages such as Italian [5,3,12] and French [6]. In the framework of IberLef 2019,
for the first time IroSvA shared task organizers [32] ask participants to classify
ironic and non-ironic texts in three variants of Spanish: Castilian, Mexican and
Cuban. The corpus provided for each linguistic variant is a collection of short
texts annotated as ironic and non-ironic. These corpora contain also labels of
specific topics referred to current political or social issues discussed online in the
chosen geographical areas of Spain, Mexico and Cuba.
Considering the importance of the context in the process of recognition of
irony [46], for each variant we analyzed the impact of topic information and
semantic context. Moreover, inspired by previous work on computational mod-
els for irony detection [21,43,33] we explored the role of features related to the
affective information present in the tweets and the psychological response stim-
ulated by the message. On the basis of the analyses in [16], we took into account
also the presence of abusive language. Finally, considering the previous studies
about irony detection in other languages, such as English in [8], we examined
also the role of lexical and stylistic features. Comparing our results with the four
challenging baselines provided by the organizers, only the system proposed for
the Cuban variant overcomes all the baselines. The other two systems overcome
only the baseline calculated on majority voting.
A preliminary analysis of errors show some important analogies among the
three variants which are in line with the observations emerged in other languages
[26,11]. Although from the first experiments in this work some slight differences
emerged among these variants, these analogies suggest a new challenge in a
multilingual direction.
The paper is organized as follows. The next section summarizes the related
work. Section 3 describes the IroSvA dataset and the used approaches focusing
on the feature engineering and performed experiments. Section 4 reports the
298
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
obtained results in the competition. Finally, Section 5 and 6 discuss the results
and draw some conclusions, proposing a plan for future analyses.
2 Related Work
In recent years, the importance to recognize the figurative language to under-
stand better the opinion of users have encouraged various researchers to explore
texts such as commentaries [47], reviews [38] and tweets [39]. Most of the studies
are focused on English language [23], but recently the need to develop linguistic
and computational resources also for other languages incited the NLP commu-
nity to focus on Italian [5,3,12], French [6], Dutch language [30] and Spanish.
The literature about figurative language detection in the Spanish language
is actually limited. The authors of [10] organized the HAHA shared task in the
context of IberEval 2018 about identification of humor in Spanish tweets. About
satire detection, the authors of [13] proposed a psychological based approach
exploiting news satirical sources on Twitter for Mexican and Castilian variants of
Spanish, while the authors of [4] employed linguistic and semantic features (such
as ambiguity and synonyms), sentiment analysis and slang words in a similar
collection of tweets from Spain. The studies about figures of speech of irony and
sarcasm are really few for the Spanish language. To the best of our knowledge,
the study proposed in [22] is the first to explore the irony in Spanish tweets
considering the sarcasm as a subclass of irony. In particular, they explored word
and character level of the texts employing n-grams of words and characters and
word embedding, using Support Vector Machines (SVM) and Random Forests as
classifiers. Recently, the authors of [25] explored deeply the function of sarcasm
in Spanish dialogues online creating a corpus annotated taking into account the
presence of sarcasm and the tone of nastiness.
Since the literature about the identification of ironic texts in Spanish is poor,
the studies that inspired this work examine the characteristics of irony based on
corpora developed on other languages. The larger part of works on this field ex-
ploits various combinations of features and computational techniques. Especially
with classic machine learning techniques, some researchers examined the impact
of stylistic features [8], pragmatic symbols (such as hashtags, mentions and emo-
jis in tweets) [28,19], syntactic patterns [20,41], sentiment and emotional lexica
[21], semantic context and users information [2,24]. Recently, technique of deep
learning have been exploited also for irony detection [50,49].
On the basis of these previous studies, we carried out a deep analysis of
the three corpora examining the impact of emotions, psychological reactions of
readers, topics, semantic contexts, lexical, abusive speech and stylistic features.
In the next sections, we describe our approach and the performed analyses that
show some analogies and differences among the ironic characteristic of the texts
in the three variants of Spanish.
299
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
3 Datasets and Approach Description
The IroSvA shared task is separated on three subtasks:
– Subtask A: Irony detection in Spanish tweets from Spain
– Subtask B: Irony detection in Spanish tweets from Mexico
– Subtask C: Irony detection in Spanish news comments from Cuba
For each subtask the organizers provided a collection of short texts about spe-
cific social and politic issues. For the Subtask A and B the datasets contain
tweets about the administration of Mexico city, the theory of the flat Earth, the
exhumation of the dictator Francisco Franco or the actions of some politicians.
For the Subtask C, the organizers provided a collection of news comments espe-
cially about Internet services and economical problems in Cuba extracted from
the news papers journals Cubadebate (http://www.cubadebate.cu/), Granma
(http://www.granma.cu/) and OnCubaNews (https://oncubanews.com/).
Each short text (tweet and news comment) is annotated as ironic and non-
ironic and contains the label of the referred topic. Below, Table 1 describes the
composition of the released datasets.
Table 1. Composition of the datasets including information about data: the number
of topics (N topics) and genre of texts.
Training set Test set N topics Genre of texts
ironic non-ironic ironic non-ironic
Subtask A 800 1,600 200 400 10 Tweets
Subtask B 800 1,600 200 400 10 Tweets
Subtask C 800 1,600 200 400 9 News comments
Total 7,200 1,800
For the classification of ironic and non-ironic texts, we employed for each vari-
ant a classical supervised machine learning approach exploiting a combination
of stylistic, semantic, affective and lexical features, named SCoMoDI (Spanish
Computational Models to Detect Irony). In particular, we used a simple SVM
classifier with radial basis function kernel using the following parameters: C = 5
and γ = 0.01. The kernel and the parameters of the SVM classifier have been
set on the basis of various experiments. Considering the imbalanced collection
of data, we used the function to balance the weights of the classes provided by
Scikit-learn library [34] for Python.
3.1 Features Engineering
In this section, we describe the features used to build the models of irony detec-
tion in the three variants of Spanish.
300
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
Lexical Features As lexical features we used unigrams of words weighted with
TF-IDF (Term Frequency-Inverse Document Frequency) measure. To extract
the unigrams, we pre-processed the texts deleting all symbols and numerical
characters and selecting words using a tokenizer able to take into account the
compound nouns. Finally, in order to weight the words without considering their
inflectional morphology, we used the SnowballStemmer for the Spanish language
provided by NLTK (Natural Language Toolkit) [7].
Stylistic Features Taking into account the corpora-based analyses carried out
in [27] for English, French and Italian, we examined the impact of features such
as hyperbole expressed by exclamation marks (!, ¡), ellipsis expressed by dots
(...), questions denoted by question marks (?, ¿) and quotes expressed by in-
verted commas (“”, ‘’). Considering the fact that some ironic texts could be
characterized by a sarcastic tone against someone, we took into account also the
typical symbol of mention in Twitter (@). In the features vector, these features
are represented by a simple count of the number of times each item appears in
the text.
Semantic Similarity Features In this group we gather semantic contexts and
topic information. The semantic contexts of each text are computed calculating
the cosine of similarity between the vocabulary of the text and the vocabularies
extracted from each group of ironic texts labeled with the same topic. The cosine
of similarity is calculated on the basis of pre-trained word embedding of the
Spanish Billion Words Corpus (available at http://crscardellino.github.
io/SBWCE/) provided by the authors of [9]. To lead the classifier to capture
similarities between texts belonging to the same topic, we extracted the topic
distribution of the text considering the number of topics of each subtask (see
Table 1). To this purpose, we created the Latent Dirichlet Allocation (LDA)
models on the provided training sets using Gensim library [37] for Python, taking
into account also bigrams and trigrams of words. The idea is to gather the texts
that talk about the same topic in a similar manner in the same class.
Affective Features We considered affective features exploiting different types
of lexical resources for capturing different facets of affect.
Emotional Categories To identify the emotions involved in each text, we counted
the number of words that belong to emotional lexica, such as the multilingual
EmoLex provided by the authors of [31] and the Spanish Emotion Lexicon (SEL)
provided by the authors of [42] and [36]. We considered for each variant only the
emotions that are relevant for the classification. It is surprising that, for all the
variants, the most significant emotions are negative, such as anger, fear, disgust
and sadness.
Dimensional Models of Emotions In order to understand the mental responses
to stimuli of ironic texts, we investigated the impact of psychological dimensions
such as imagery, activation and pleasantness. Inspired by [40,21], we use an
301
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
automatic translated Spanish version of the Dictionary of Affect in Language
(DAL) [48]. From our analysis the dimensions that turned out useful for the
classification in all the three variants are pleasantness and imagery.
Abusive Language Features Inspired by the work of [25] and considered the
relevance of negative emotions, we analyzed also the impact of abusive language
counting the words included in the Spanish lexica of derogatory expressions and
profanities created by the authors of [17]. These lists of words prove to be sig-
nificant for the classification especially in Mexican and Castilian tweets.
During the experimental phase, we used 5-fold cross validation on the training
sets tuning the system on the metric used for the competition: the average of
F1-scores of the classes. To study the impact of the described features, we carried
out also the ablation feature test and, on the basis of these analyses, we created
the models for each variant. The highest F1-scores values obtained with the
relevant features are reported in Table 2.
Table 2. Experimental results on training data.
Subtask A Subtask B Subtask C Subtask A Subtask B Subtask C
Lexical Features v v v v
Stylistic Features
hyperbole v v v v
ellipsis v v v v
question v v v v
quotation v v v v
mention v v v
Semantic Similarity Features
semantic context v v v v v v
topic information v v v v v
Affective Features
Emotional Categories
anger v v v v
fear v v v v
disgust v v v v
sadness v v v v
Dimensional Models of Emotions
imagery v v v v v
activation v v v
pleasantness v v v v v
Abusive Language Features
derogatory expressions v v v v
profanities v v v v
F1-scores 45.57 47.69 50.12 54.86 55.34 52.26
302
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
4 Evaluation and Results
The organizers of IroSvA shared task provided four baselines calculated consid-
ering different representation of the texts. They used: n-grams of words (Word
nGrams), word embeddings (W2V) and low dimensionality statistical embed-
ding (LDSE) [35]. They used also the majority voting (Majority) technique as
additional baseline.
As evaluation measure for the ranking the average of F1-score (avg) of all
three variants is used. In Table 3 we report the results obtained in the competi-
tion compared with the provided baselines.
Table 3. Results obtained in the competition
Subtask A Subtask B Subtask C avg
Baselines
LDSE 67.95 66.08 63.35 65.79
W2V 68.23 62.71 60.33 63.76
Word nGrams 66.96 61.96 56.84 61.92
Majority 40.00 40.00 40.00 40.00
Our approach
SCoMoDI 66.52 55.74 63.38 61.88
As we can see, only the model built for the Cuban variant (Subtask C) over-
comes slightly all the provided baselines, while the other models overcome only
the Majority baseline. This difference could be due to the textual genre of news
comments which do not contain Twitter mentions (@USER), hashtags or emojis.
Another influential factor could be the use of a set of features able to capture
characteristics such as unigrams that in general help the text classification.
5 Error Analysis and Discussion
Although the different genres of texts, analyzing the misclassified texts, we no-
ticed that in all the three variants of Spanish the irony is expressed similarly.
Actually, we individuated various figures of speech involved in the expression of
irony. With the proposed models we aimed at capturing some of them by exploit-
ing textual marks, but the error analysis highlights that it was not sufficient.
In particular, we found that hyperbole (examples 1) and ellipsis (examples 2)
are expressed more at a semantic level. See, for instance, the following examples
from the IroSvA test set where irony was not recognized:
(1) Felicidades director muy buena tarifa ası́ se hace.
Congratulations director it is a very good rate, this is how it should be
done.
(2) Cuando yo sea grande quiero ser como los inventores del paquete.
When I grow up, I want to be like the inventors of this offer.
303
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
As defined in [29], hyperbole is expressed by “exaggerated or extravagant
terms used for emphasis and not intended to be understood literally”. In fact,
the example 1 (from Subtask C) is a clear example of hyperbole that aims to
exaggerate positively the actions of someone who is doing bad his work. The im-
portance of hyperbole have been already underlined by [15] in irony and sarcasm
detection.
We found this same phenomenon in the misclassified tweets of Subtask A
(example 3) and Subtask B (example 4):
(3) @Maras70 @okdiario Te falta Pisarello, Echenique y nuestro gran
concejal de tráfico el señor Grezzi.
@Maras70 @okdiario You miss Pisarello, Echenique and our fantastic
city councillor of the traffic Mister Grezzi.
(4) El pueblo sabio y bueno salio a expresar su voz @lopezobrador
The wise and good people came out to express their voice.
In the example 2 (from Subtask C), the irony is expressed by ellipsis. In [29]
ellipsis is defined as “omission of a word easily supplied”. In this news comment,
the author wanted to subtract, on purpose, some words containing information
that could complete the meaning of the sentences. This subtraction is possible
because of the presence of context that give us some intuition about the real
meaning of the message. This same phenomenon is found especially in Subtask
B (example 5):
(5) Será que le da clases particulares el @brozoxmiswebs
It is possible that he teaches him private lessons.
Unfortunately, the simple syntactic features that we used especially for Sub-
task C are not sufficient to capture these more complex puns based on semantic
incongruity.
Another common linguistic phenomenon found during the error analysis in
Mexican and Cuban variants is the use of apostrophe to stimulate the ironic
interpretation of the message. In [29] the apostrophe is defined as action of
“breaking off a discourse to address some person or personified thing either
present or absent”, as we can see in the following examples extracted from the
misclassified texts in Subtask B (example 6) and C (example 7):
(6) Con todo respeto señor presidente, le solicito atentamente que haga
una auditorı́a al @ColegioNal mx cuyos miembros se rayan y donde la
mafia de Octavio Paz se ha instalado.
With all due respect, Mr. President, I kindly ask you to do an audit in the
@ColegioNal mx whose members benefit and where the mafia of Octavio
Paz has settled.
(7) Y ahora es que usted se entera que la honestidad pasó de moda?
And only now you realize that honesty went out of fashion?
304
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
Moreover, the rhetorical questions seem to be one of the most used device
to express irony in all the variants of Spanish. In [29] the rhetorical question is
defined as question “which implies an answer but does not give or lead us to
expect one”. We noticed that although for Subtask B we took into account the
presence of question marks, this expedient is not enough to classify correctly
irony. Observe the following texts:
(8) Cuando pedirán perdón Alemania e Italia a los Valencianos , por
mandarnos a la Oltra y Grezzi ? https://t.co/MdFup1pbvu
When Germany and Italy will apologize to Valencians, for sending us
Oltra and Grezzi? https://t.co/MdFup1pbvu
(9) Disculpa, sabes si para trabajar en el @Conacyt MX ¿Debo llevar mi
curriculum impreso o depilado?
Excuse me, do you know if to work at the @Conacyt MX Should I bring
my curriculum printed or shaved?
(10) Otra interrogante, por qué nadie fuera de Cuba ha denunciado que
los cubanos violamos abiertamente los derechos de los productores de
esas programaciones?? O será que el paquete ha venido a ser el primer
“embajador” en el restablecimiento de las relaciones??
Another question, why anyone outside of Cuba have not declared that
Cubans openly violated the rights of the producers of these programs?
Or has the package become the first “ambassador” in the restoration of
relations ??
In the examples 8 (from Subtask A), 9 (from Subtask B) and 10 (from Subtask
C) we can see that rhetorical questions involve also other figures of speech such
as apostrophe (example 8 and 9) and metaphor (example 9 and 10) which fill
the messages with various allusions, making its interpretation more difficult.
These observations suggest that there are some analogies on how Spanish
speakers prefer to express irony. Moreover, some of these features have been
already explored also in English, French and Italian ironic tweets [27]. Therefore,
it seems that such kind of puns are in general characterizing the expressions of
irony independently from the language and genre of the text.
6 Conclusions
In this paper we describe our participation at IroSvA shared task presenting an
initial study on irony detection in the Spanish language. In fact, on the basis of
the previous works in other languages, we analyzed the impact of various fea-
tures on the classification of Spanish ironic and non-ironic texts. Moreover, the
carried out analyses highlight some analogies and slight differences on the way
users express irony in the three proposed variants. We observed that Spanish
irony seems to be characterized by specific puns and especially by negative emo-
tions. However, only Mexican and Castilian ironic tweets seem contain abusive
language.
305
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
As future work, we planned to examine deeply the role that the topic and
the context play in Spanish irony detection, proposing a comparison with a
topic-independent approach. Moreover, considering our intuition about the use
of similar figures of speech to express irony in different languages, we would like
to compare multilingual data exploring their real impact in irony detection.
References
1. Attardo, S.: Irony as relevant inappropriateness. Journal of pragmatics 32(6), 793–
826 (2000)
2. Bamman, D., Smith, N.A.: Contextualized sarcasm detection on twitter. In: Ninth
International AAAI Conference on Web and Social Media (2015)
3. Barbieri, F., Basile, V., Croce, D., Nissim, M., Novielli, N., Patti, V.: Overview of
the evalita 2016 sentiment polarity classification task. In: Proceedings of EVALITA
2016 (2016)
4. Barbieri, F., Ronzano, F., Saggion, H.: Is this tweet satirical? a computational
approach for satire detection in spanish. Procesamiento del Lenguaje Natural 55,
135–142 (2015)
5. Basile, V., Bolioli, A., Nissim, M., Patti, V., Rosso, P.: Overview of the evalita 2014
sentiment polarity classification task. In: Proceedings of EVALITA 2014 (2014)
6. Benamara, F., Grouin, C., Karoui, J., Moriceau, V., Robba, I.: Analyse d’opinion et
langage figuratif dans des tweets: présentation et résultats du défi fouille de textes
deft2017. In: Atelier TALN 2017: Défi Fouille de Textes (DEFT 2017) (2017)
7. Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly
Media (2009)
8. Buschmeier, K., Cimiano, P., Klinger, R.: An impact analysis of features in a
classification approach to irony detection in product reviews. In: Proceedings of
the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and
Social Media Analysis. pp. 42–49 (2014)
9. Cardellino, C.: Spanish Billion Words Corpus and Embeddings (March 2016)
10. Castro, S., Chiruzzo, L., Rosá, A.: Overview of the haha task: Humor analysis
based on human annotation at ibereval 2018. In: Proceedings of IberEval 2018.
vol. 2150. CEUR-WS (2018)
11. Cignarella, A.T., Bosco, C., Patti, V., Lai, M.: Application and analysis of a multi-
layered scheme for irony on the italian twitter corpus twittiro. In: Proceedings
of the Eleventh International Conference on Language Resources and Evaluation
(LREC-2018) (2018)
12. Cignarella, A.T., Frenda, S., Basile, V., Bosco, C., Patti, V., Rosso, P.: Overview of
the evalita 2018 task on irony detection in italian tweets (ironita). In: Proceedings
of EVALITA 2018. vol. 2263, pp. 1–6. CEUR-WS (2018)
13. Del Pilar Salas-Zárate, M., Paredes-Valverde, M.A., Rodriguez-Garcia, M.A.,
Valencia-Garcı́a, R., Alor-Hernández, G.: Automatic detection of satire in twitter:
A psycholinguistic-based approach. Knowledge-Based Systems 128, 20–33 (2017)
14. Du Marsais, C.C.: Traité des tropes: Pour servir d’introduction a la rhetorique et
a la logique. Chez la veuve Gaspard Fritsch (1757)
15. Filatova, E.: Irony and sarcasm: Corpus generation and analysis using crowdsourc-
ing. In: Lrec. pp. 392–398. Citeseer (2012)
16. Frenda, S.: The role of sarcasm in hate speech: A multilingual perspective. In:
Proceedings of Doctoral Symposium at SEPLN 2018. CEUR-WS (2018)
306
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
17. Frenda, S., Somnath, B.: Deep analysis in aggressive mexican tweets. In: Proceed-
ings of IberEval 2018. vol. 2150, pp. 108–113. CEUR-WS (2018)
18. Ghosh, A., Li, G., Veale, T., Rosso, P., Shutova, E., Barnden, J., Reyes, A.:
Semeval-2015 task 11: Sentiment analysis of figurative language in twitter. In:
Proceedings of SemEval 2015. pp. 470–478 (2015)
19. González-Ibánez, R., Muresan, S., Wacholder, N.: Identifying sarcasm in twitter:
a closer look. In: Proceedings of the 49th Annual Meeting of the Association for
Computational Linguistics: Human Language Technologies: Short Papers-Volume
2. pp. 581–586. Association for Computational Linguistics (2011)
20. Hao, Y., Veale, T.: An ironic fist in a velvet glove: Creative mis-representation in
the construction of ironic similes. Minds and Machines 20(4), 635–650 (2010)
21. Hernández Farı́as, D.I., Patti, V., Rosso, P.: Irony detection in twitter: The role
of affective content. ACM Transactions on Internet Technology (TOIT) 16(3), 19
(2016)
22. Jasso, G., Meza, I.: Character and word baselines for irony detection in spanish
short texts. Procesamiento del Lenguaje Natural 56, 41–48 (2016)
23. Joshi, A., Bhattacharyya, P., Carman, M.J.: Automatic sarcasm detection: A sur-
vey. ACM Computing Surveys (CSUR) 50(5) (2017)
24. Joshi, A., Sharma, V., Bhattacharyya, P.: Harnessing context incongruity for sar-
casm detection. In: Proceedings of the 53rd Annual Meeting of the Association for
Computational Linguistics and the 7th International Joint Conference on Natural
Language Processing (Volume 2: Short Papers). vol. 2, pp. 757–762 (2015)
25. Justo, R., Alcaide, J.M., Torres, M.I., Walker, M.: Detection of sarcasm and nasti-
ness: New resources for spanish language. Cognitive Computation 10(6), 1135–1151
(2018)
26. Karoui, J., Benamara, F., Moriceau, V., Aussenac-Gilles, N., Hadrich Belguith, L.:
Towards a contextual pragmatic model to detect irony in tweets. In: Proceedings of
the 53rd Annual Meeting of the Association for Computational Linguisticsand the
7th International Joint Conference on Natural Language Processing. pp. 644–650.
Association for Computational Linguistics (ACL) (2015)
27. Karoui, J., Benamara, F., Moriceau, V., Patti, V., Bosco, C., Aussenac-Gilles, N.:
Exploring the impact of pragmatic phenomena on irony detection in tweets: A
multilingual corpus study. In: Proceedings of the 15th Conference of the European
Chapter of the Association for Computational Linguistics. vol. 1, pp. 262–272
(2017)
28. Kunneman, F., Liebrecht, C., Van Mulken, M., Van den Bosch, A.: Signaling sar-
casm: From hyperbole to hashtag. Information Processing & Management 51(4),
500–509 (2015)
29. Lanham, R.A.: A Hypertext Handlist of Rhetorical Terms: For Macintosh Com-
puters. University of California Press, Berkeley, CA, USA (1996)
30. Liebrecht, C., Kunneman, F., van Den Bosch, A.: The perfect solution for detecting
sarcasm in tweets# not. In: Proceedings of the 4th Workshop on Computational
Approaches to Subjectivity, Sentiment and Social Media Analysis. pp. 29–37. ACL
(2013)
31. Mohammad, S.M., Turney, P.D.: Crowdsourcing a word–emotion association lexi-
con. Computational Intelligence 29(3), 436–465 (2013)
32. Ortega-Bueno, R., Rangel, F., Hernández Farı́as, D.I., Rosso, P., Montes-y-Gómez,
M., Medina, J.E.: Overview of the task on irony detection in spanish variants.
In: Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019), co-
located with 34th Conference of the Spanish Society for Natural Language Pro-
cessing (SEPLN 2019). CEUR-WS.org (2019)
307
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
33. Pamungkas, E.W., Patti, V.: #NonDicevoSulSerio at SemEval-2018 task 3: Ex-
ploiting emojis and affective content for irony detection in English tweets. In: Pro-
ceedings of The 12th International Workshop on Semantic Evaluation. pp. 649–654.
Association for Computational Linguistics (2018)
34. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O.,
Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A.,
Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine
learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011)
35. Rangel, F., Rosso, P., Franco-Salvador, M.: A low dimensionality representation
for language variety identification. In: 17th International Conference on Intelli-
gent Text Processing and Computational Linguistics, CICLing’16. Springer-Verlag,
LNCS(9624), pp. 156-169 (2018)
36. Rangel, I.D., Guerra, S.S., Sidorov, G.: Creación y evaluación de un diccionario
marcado con emociones y ponderado para el español. Onomázein: Revista de
lingüı́stica, filologı́a y traducción de la Pontificia Universidad Católica de Chile
29, 31–46 (2014)
37. Řehůřek, R., Sojka, P.: Software Framework for Topic Modelling with Large Cor-
pora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP
Frameworks. pp. 45–50. ELRA, Valletta, Malta (May 2010)
38. Reyes, A., Rosso, P.: Making objective decisions from subjective data: Detecting
irony in customer reviews. Decision support systems 53(4), 754–760 (2012)
39. Reyes, A., Rosso, P., Buscaldi, D.: From humor recognition to irony detection:
The figurative language of social media. Data & Knowledge Engineering 74, 1–12
(2012)
40. Reyes, A., Rosso, P., Veale, T.: A multidimensional approach for detecting irony
in twitter. Language resources and evaluation 47(1), 239–268 (2013)
41. Riloff, E., Qadir, A., Surve, P., De Silva, L., Gilbert, N., Huang, R.: Sarcasm as
contrast between a positive sentiment and negative situation. In: Proceedings of
the 2013 Conference on Empirical Methods in Natural Language Processing. pp.
704–714 (2013)
42. Sidorov, G., Miranda-Jiménez, S., Viveros-Jiménez, F., Gelbukh, A., Castro-
Sánchez, N., Velásquez, F., Dıaz-Rangel, I., Suárez-Guerra, S., Trevino, A., Gor-
don, J.: Empirical study of opinion mining in Spanish tweets. MICAI 2012. LNCS
7629, 1–14 (2012)
43. Sulis, E., Hernández Farı́as, D.I., Rosso, P., Patti, V., Ruffo, G.: Figurative mes-
sages and affect in Twitter: Differences between #irony, #sarcasm and #not.
Knowledge-Based Systems 108, 132–143 (2016)
44. Utsumi, A.: A unified theory of irony and its computational formalization. In:
Proceedings of the 16th conference on Computational linguistics. vol. 2, pp. 962–
967. Association for Computational Linguistics (1996)
45. Van Hee, C., Lefever, E., Hoste, V.: Semeval-2018 task 3: Irony detection in en-
glish tweets. In: Proceedings of The 12th International Workshop on Semantic
Evaluation. pp. 39–50 (2018)
46. Wallace, B.C.: Computational irony: A survey and new perspectives. Artificial
Intelligence Review 43(4), 467–483 (2015)
47. Wallace, B.C., Charniak, E., et al.: Sparse, contextually informed models for irony
detection: Exploiting user communities, entities and sentiment. In: Proceedings of
the 53rd Annual Meeting of the Association for Computational Linguistics and
the 7th International Joint Conference on Natural Language Processing (Volume
1: Long Papers). vol. 1, pp. 1035–1044 (2015)
308
Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)
48. Whissell, C.M.: The dictionary of affect in language. In: The measurement of emo-
tions, pp. 113–131. Elsevier (1989)
49. Zhang, M., Zhang, Y., Fu, G.: Tweet sarcasm detection using deep neural net-
work. In: Proceedings of COLING 2016, The 26th International Conference on
Computational Linguistics: Technical Papers. pp. 2449–2460 (2016)
50. Zhang, S., Zhang, X., Chan, J., Rosso, P.: Irony detection via sentiment-based
transfer learning. Information Processing & Management 56(5), 1633–1644 (2019)
309