Irony in text allows expressing implicit negative opinions, using a gurative and humorous language. IroSvA is a challenging task proposed this year that allows the evaluation of Irony Detection algorithms in the analysis of short texts for three Spanish language variants (Cuban, Spain and Mexican). Our proposal focuses on the study of three di erent representations of textual information and similarity measures using a weighted combination of these representations. We use an adaptation of the impostors method for classifying texts in Ironic or Non ironic. We consider the non-ironic texts of the training dataset as the list of impostors. The results achieved are encouraging and the best were obtained for the cuban variant.
Irony is a fundamental rhetorical device. It is a uniquely human mode of
communication, curious in that the speaker says something other than what he or
she intends [
As such, modeling irony has a large potential for applications in various research
areas, including text mining, author pro ling, detecting online harassment and,
perhaps one of the most investigated applications at present, automatic
sentiment analysis. [
Therefore, detecting irony involves the most challenging areas of natural
language processing. Research in user-generated content is a very challenging task.
Despite the social media texts convey an invaluable source of information, they
are di cult to process because they are noisy, informal, with little context, and
plenty of grammatical mistakes [
In the task of detecting irony, although in its beginnings, the English language
[
IroSvA (Irony Detection in Spanish Variants) is the rst shared task fully
dedicated to identify the presence of irony in short messages (tweets and news
comments) written in Spanish. [
The task will be structured into three subtasks, each one of these, for predicting whether messages are ironic or not, in one of the three Spanish variants. The three subtasks aim to the same goal: participants should determine whether a message is ironic or not according to speci ed context.
Recent advances in detection of irony have shown that the supervised classi
cation methodology with a great extent of feature engineering produces satisfactory
indicators for irony or sarcasm. This methodology has been tested in short texts,
such as product reviews, news commentaries and tweets [
Our approach is based on the representation of the text using three di erent
vectors of features. For classifying a new text in the Ironic or Non-ironic class,
we used the proposed General Impostor Method (GIM) of [
A text (document) is modeled as an object with three vector representations, but using some features in common. The idea behind this type of representation is to adjust what representation could impact for di erent genres of text. One representation, and denoted by R1, is based on the classical Bag of Word (BoW) vector and we build this with the tokens extracted from a Natural Language tokenizer or with lemmas extracted using a Natural Language lemmatizer. The second representation, R2, is based only on the frequency of Punctuation Sign, considering all punctuation extracted by a Part of Speech Tagger. For the third representation, R3, are computed di erent stylistic features and the frequency of them. Some of the stylistic features are: entire capitalized words (QUE BUENO), character ooding (oooooohhhhhh), repetition of closed exclamation sign (!!!!!!!), etc. 2.2
The similarity measures to compare two documents needs to consider all the representations proposed, but also to be exible, so that we can use only some of them. For that reason we use the following measure.
(D1; D2) = 1(D1R1 ; D2R1 ) + 2(D1R2 ; D2R2 ) + 3(D1R3 ; D2R3 ) (1) For the similarity function we implemented di erent similarities proposed in the literature, for example, cosene, jaccard, dice, tanimoto or distance like euclidean or minmax. In the evaluation phase we tested with all of them and used the one that allows us to obtain the best result for each representation. The parameters ; ; , let us give importance to the representation, and if one of the parameters is 0, then, not considering that representation. + + = 1. 2.3
We used the proposed General Impostor Method (GIM), but not considering in the IM the step of randomly choose a subset of features from the representation of text.
The set of impostors S corresponds to the set of non-ironic documents provided in the training dataset. D1 is the document to be classi ed and D2 is an ironic document. is a parameter that needs to be adjusted using the training dataset and allows us to determine that D1 is ironic if it has more similarity with ironic D2 than a percentage of non-ironic impostors.
Input: (D1; D2) A pair of documents. S: a set of impostors documents Output: (ironic) or (non-ironic) 1. Score = 0 2. Repeat k times a. Randomly choose n impostors from S : I1; :::; In b. Score+ = k1 if (D1; D2) (D2; D1) > (D1; Ii) (D2; Ii); for each i 2 f1:::ng 3. return (ironic) if Score > ; else (non-ironic)
The set of Non-ironic texts are considered the impostors texts and for the classi cation of the evaluation dataset we used all the Non-ironic texts provided in the training dataset. All the parameters were optimized and the best value used for the evaluation phase and resumed in the Evaluation section. Next we present the pseudo-code of IM and GIM.
Input: (D) A document to be classi ed. Y : (D1; :::; Dn) ironic documents Output: (ironic) or (non-ironic) 1. For each pair of documents (D; Di)
a. Run original IM to obtain a similarity binary score S(D; Di) 2. Score = Average over similarity scores [S(D; D1); :::; S(D; Dn)] 3. return (ironic) if Score > ; else (non-ironic) 3
The evaluation was executed for the three variants of Spanish language presented by the task, and for each of them we needed to optimize the parameters of the GIM and the weighted similarity measure. For that purpose we run a 10 cross fold validation over the training dataset provided. The range of the parameters evaluated was varied and the best parameter chosen is illustrated in the Table 1(parameters of UO-run2).
The structure and data distribution for the training and test datasets were
presented by [
In the Table 2, we present in row 1 and row 2 the results achieved by the two
runs sent to the task, and all the baselines exposed by the organizers, and the
highest value obtained by[
Our best result was achieved for UO-run2, and for the cuban variant, also similar to those of two of the baselines. It is important to notice that with lemma representation the results were always better than the representation based only on lexical tokens, because in the rst one we reduced the lexical variety of words referring to the same lemma. 4
In general, based on the results achieved in the cross validation from the training dataset of the Spain and Mexican dataset (spanish tweets), the R3 representation gets the worse results and this is based on the stylistic variety between ironic tweets, and also the similarity in stylistic features between ironic and non-ironic tweets.
As future directions, we will introduce representations based on Word nGrams and feature selection methods based on the importance of features by class.