In the present paper we describe the participation of the ATC team at the IroSvA 2019 shared task at IberLEF 2019, which is focused on Irony Detection in Spanish Variants, addressing its identi cation as a classical binary classi cation task. The approach is mainly oriented in performing a preliminary test of the importance of morphosyntactic information in the task of irony detection. For this reason, we exploited a straightforward methodology: a Support Vector Classi er with a linear kernel, combined with shallow features based on morphology and dependency syntax. For the representation of such kind of knowledge we relied on the application on the data of the well-known Universal Dependencies format.
The recognition of irony and the identi cation of pragmatic and linguistic devices
that activate it are known as very challenging tasks to be performed by both
humans or automatic tools [
The growing interest in this task is attested by the proposal of shared tasks
focusing on irony detection and its impact on sentiment analysis in social media
[
In this paper we describe our submission at the IroSvA 2019 shared task
[
The task is structured into three subtasks, each one for predicting whether messages are ironic or not in three di erent variants of the Spanish language, i.e. those spoken respectively in Spain, Mexico and Cuba. The aim is that of investigating whether a short message, written in the Spanish language, is ironic or not with respect to a given context. The organizers provided a di erent training and test set for each Spanish variant where the items are distributed as shown in Table 1. The Spanish and Mexican sets are composed by tweets, while the set from Cuba included news comments. As far as the distribution of irony, each training set contained 800 ironic and 1,600 not ironic texts. The same proportion of 33%-66% has been also maintained in the test set, counting respectively 200 ironic and 400 not-ironic texts in each subset. 3
Because of our interest in features related to syntax and their contribution in gurative language detection, we didn't take into account lexical features also considering that they can be too much in uenced by the involved Spanish varieties. Therefore, we trained our automatic system on the three datasets altogether (7,200 texts) and tested the same model on the three di erent test sets, regardless of the three variants of Spanish. 3.1
We applied two preprocessing steps: the stripping of URLs from texts all normalized to lowercase letters, and the morpho-syntactic analysis. We trained indeed the UDPipe (which includes tokenization, Part of Speech tagging and parsing) on the UD Spanish-GSD corpus for generating for each item of the dataset a CoNLL-U tree in Universal Dependencies (UD) format, the de facto standard for dependency-based syntactic representations. 3.2
After having performed experiments with Random Forest, Decision Trees and Support Vector Machine, we nally implemented our system with a linear kernel with the last classi er, which resulted the best performing one. We propose a straightforward approach with two di erent types of features: a) common \baseline" features, widely explored in sentiment analysis tasks and in irony detection tasks too; and b) new syntactic \dependency-based" features. Our novel contribution mainly consists in the exploitation of these latter features to create vectorial representations of texts; all them (listed in point b) have been made available thanks to the application of the UDpipeline and the subsequent generation of the dependency trees corresponding to the items of the datasets. Figure 1 shows a tweet where the UD format has been applied (Translation: I have launched a reporter into the air and they do not y... ).
Bag of Words (BoW): each tweet was pre-processed to convert it to lowercase letters. Then we extracted unigrams, bigrams and trigrams to create a binary representation
Bag of Char-grams (BoC): we considered the sequence of char-grams in a range from 2 to 5 characters.
Bag of Dependency Relations (BoDeprel): following the approach described by
Ghanem et al. [
aux root obj det
conj nmod case det punct cc advmod punct AUX VERB DET NOUN ADP DET NOUN CCONJ ADV VERB PUNCT SYM He lanzado un relator a el aire y no vuelan ...
Bag of SyntaxPath's Word Forms (Path Form): starting from the intuition
of Sidorov et al. [
Bag of SyntaxPath's Deprels (Path Deprel): we created a Bag of Deprels, collecting the dependency relations occurring in the structure of the syntactic tree, i.e. following the syntactic paths, thus creating a vectorial space based on bigrams, combining dependency relations in pairs. For instance, the Path Deprel corresponding to the sentence in Figure 1 includes ['root', 'aux'], ['root', 'punct'], ['root', 'conj'], ['conj', 'cc'], ['conj', 'advmod'], ['conj', 'punct'], ['root', 'obj'], ['obj', 'det'], ['obj', 'nmod'], ['nmod', 'case'], ['nmod', 'det'].
The system is available at: https://github.com/AleT-Cig/ATC_IroSvA_2019. 4
In Table 2 we show our o cial results in comparison with the four baselines
proposed by the organizers. We can observe how, on average, our dependency-based
approach performs better than shallow lexical techniques, such as word n-grams,
but is not stronger than more re ned approaches, such as those implemented in
the neural-network based approach of word2vec and LDSE [
In this paper we presented an overview of the ATC submission for the IroSvA 2019 shared task on Irony Detection in Spanish Variants at IberLEF 2019. We participated to the shared task by submitting one single system for the detection of irony in Spanish, Mexican and Cuban texts with a view to testing the suitability of the features we engineered across di erent datasets of Spanish variants based on syntactic knowledge.
Our approach, chie y based on morphological and dependency-based
syntactic features, proved to perform better than straightforward baselines, such as
word n-grams and majority voting, and also to be able to provide in-line results
with respect to stronger baselines based on word2vec semantic representation
and the LDSE system [
The results show that focusing on syntactic features, namely Bag of SyntaxPath's Word Forms and Bag of SyntaxPath's Deprels (i.e. the novel contribution of our work), produced a good contribution to the Irony Detection task in Spanish Variants. Considering that the results seem quite promising, and that the dependency-based features deserves a ner-grained study, in future work we will further investigate them observing their behavior in other tasks related to sentiment analysis. In particular, thanks to the great adaptability of the UD format across di erent languages, we plan to test these new features in a multilingual scenario too.
The work of Cristina Bosco was partially funded by Progetto di Ateneo/CSP 2016 (Immigrants, Hate and Prejudice in Social Media, S1618L2BOSC01).