We present the first model for argumentation mining for Italian short argumentative texts. We adapted to Italian the software developed by (Peldszus and Stede, 2015) and built a suitable corpus of Italian "microtexts" by semi-automatically translating the original English corpus. Our results are comparable to those of (Peldszus and Stede, 2015), which proves that their model is applicable successfully to languages other than English and German.1 This task can be decomposed into several subtasks: segmentation of the text in elementary discourse units (EDUs), identification of argumentative discourse units (ADUs), classification of argumentative discourse units, identification of the relations between argumentative discourse units and classification of these relations. The argumentation structure of a text can be presented as a tree structure, with a node for each argumentative discourse unit and different edges between nodes representing the different types of relations. There are many simple models that recognize automatically the argumentation structure of a micro-text.
In recent years, argumentation mining
Our results are slightly lower than the ones for German and English, but they demonstrate that the model can be considered valid also for Italian. Besides, a major contribution of this paper is the free availability of the annotated Italian corpus.2 2 https://github.com/PietroTotis/evidencegraph
One of a few similar approaches is that of (Stab and Gurevych, 2017), who introduced a corpus of persuasive essays annotated with argumentation structures related to the argmicrotexts and presented a similar approach for parsing argumentation structures.
An example of argumentation mining for
Italian is presented in
The interest in argumentation-oriented corpora of monologue text is rising, but most of the present data are not suitable for these operations. For this reason it is necessary to have wellformed and controlled corpora of short argumentative texts.
In order to provide a corpus of Italian short
argumentative texts we translated to Italian the
arg-microtexts corpus, a freely available3 parallel
3 https://github.com/peldszus/arg-microtexts
corpus of 113 short texts and a total of 576
ADUs
The texts are short but at the same time “complete” and the underlying argumentation structure is relatively clear. The probands were asked to first gather a list with the pros and cons of the trigger question, then take stance for one side and argue for it in a short argumentative text, which had to be at least five segments long with each segment argumentatively relevant, had to contain at least one objection and finally had to be understandable without having its trigger question as a headline. All of the microtexts were originally written in German and have been successively professionally translated in English.
The annotation scheme we used for our
corpus is the same used for the original corpus,
developed by Peldszus and Stede on the basis of
different ideas from literature about
argumentation structures
The annotation scheme used for the
argmicrotexts corpus is based mainly on Freeman’s
theories, which integrate Toulmin’s ideas into
the argument diagraming techniques of the
informal logic tradition
The choice of translating into Italian the argmicrotexts corpus, likewise it was previously done for English, is motivated by the controlled setting of the experiment. The translation process had two phases. In the first phase we automatically translated the entire corpus using DeepL Translator4, a free and multilingual translation service. In the second phase, all the translations have been manually checked and, if needed, post-edited.
Some corrections were necessary in almost every microtext: from a syntactic point of view the translator respected most of the dependencies, losing however accuracy with increasingly complex syntactic structures. As 4 https://www.deepl.com/translator foreseeable, a lot of words were translated with the most common Italian translation, but not the most appropriate. All the microtexts have been thereby post edited in order to look as they were generated directly in Italian. Connectives have a fundamental role in the identification of function, role and attachments of a sentence. We therefore dedicated special attention to this aspect; in the automatic translation, many different original forms converged to the most common connective in the target language. For example, almost all the connectives expressing similarity were translated with “e” (“and”) and most of the connectives expressing contrast were translated with “ma” (“but”). In order to have a more realistic corpus we tried to use a more various set of connectives, comparable to the set used in the original corpus.
The annotated graph structures are stored in XML format. The main advantage of translating the arg-microtexts corpus was that it was not necessary to make the annotations from scratch. As expected, there was a one by one correspondence between original sentences in German and the translations in Italian. In order to have Italian annotated graph structures it was only necessary to automatically substitute every German sentence in the XML file with the corresponding Italian sentence. In case a sentence contained more ADUs, it has been divided manually.
The code for computing the tree predictions
have been taken over from the work of Peldszus
and Stede
In order to recognize the argumentation structure, the model considers not only the probability of attachment of each segment pair, but also the probabilities of role, function and of being the central claim. In order to do so it is necessary to predict probabilities for each argumentative unit on different levels: attachment, central claim, role (proponent or opponent) and function (supporting or attacking).
The first step is to build a fully connected
multigraph that connects every segment pair
with as many edges as the function types. In
order to get central claim, role, function and
attachment probabilities, the model uses
different classifiers and then jointly combines
these probabilities in a single edge score, defined
as the weighted sum of the level specific edge
scores, on which it is possible to apply a MST
(minimum spanning tree) algorithm
The result represents the best global
attachment structure for the text. This model
outperformed other baseline and simpler models
when tested on the German and English parallel
corpus
In order to run the original experiments on the Italian corpus, we adapted the sections of the code related to the corpus and the NLP tools. The latter represents the major divergence from the original setting, since it entailed upgrading the spaCy package, along with its language models. This also involved upgrading other packages and porting the whole project to Python 3.x, but these were minor modifications that should not have a meaningful impact on the performances.
A language-specific set of connectives is
essential for the classification of the relations
between ADUs. For this purpose, we used LiCo5,
a lexicon of Italian connectives
- Syntactic type (preposition, adverb, coordinating conjunction, subordinating conjunction). - Relation type (as cause, concession, contrast, purpose).
5 http://connective-lex.info/ 6.
The metrics to evaluate our adaptation are Macro F1 and Micro F1 for each sub-task: central claim, role, function and attachment detection. The results are reported in Table 1.
Compared to the results obtained in the
experiment with the English and the German
corpus
cc Macro F1 0.813 Micro F1 0.883 Table 1: Results for Italian ro 0.724 0.811 cc ro Macro F1 0.825 0.765 Micro F1 0.888 0.841 Table 2: Results for English
We investigated the reason for the lower performances in the task of function classification: Figure 2 and 3 show an example of misclassification. The prediction for the microtext mistakenly detects an attacking and an undercutting relation in place of two supporting relations. Wrong function classification of some argumentative unit can be found in most of the outputs of the corpus.
Another common error is the wrong
attachment: Figure 3 and 4 present an interesting
error for this task. In place of an “attach to first”
structure, which is typical of the English style of
essay writing and can be used as baseline, our
model has attached all the argumentative units to
the preceding segment, which is also a typical
baseline in discourse parsing
We investigated the role of connectives in the attachment prediction and ran the same experiment on a less specific list of connectives, i.e. with more general relation types. With this simplified version of the connectives, the classifier achieved lower results in all the tasks. This suggests that specificity is not the reason behind these errors and at the same time proves the central role of the connectives in the recognition of an argumentation structure.
We presented, to our knowledge, the first model
that transfers on an Italian microtexts corpus the
approach developed by