English. This paper presents some issues about a computational lexicon employed in a generation system for Italian (Mazzei et al., 2016). The paper has three goals: (i) to describe the SQL resources produced during the construction of the lexicon; (ii) to describe the algorithm for building the lexicon; (iii) to present an ongoing work for enhancing the lexicon by using the syntactic information extracted from a treebank.
A number of free large multilingual resources
covering Italian have been released, e.g.
MultiWordnet, UniversalWordnet, BabelNet
A notable exception is the
PAROLE-SIMPLECLIPS lexicon, that is a four-level (i.e.
phonological, morphological, syntactical, semantic) general
purpose lexicon composed by 53; 044 lemmata
Rule-based natural language realization
engines, that are systems performing
linearisation and morphological inflections of a
protosyntactic input tree
In order to build this specific lexicon, that we
have called SimpleLEX-IT, we have decided to
merge three free resources for Italian, namely
Morph-it!
This paper is organized as follows: in Section 2 we describe the conversion of the three lexical resources used into a relational database; in Section 3 we provide some details about the algorithm used to build SimpleLEX-IT; in Section 4 we describe a work in progress to enrich the lexicon by using the syntactic information extracted from a treebank; finally, Section 5 closes the paper with conclusions. 2
In order to merge different lexical resources we needed to convert them in a common computational representation. We used a relational database2 (SQL henceforth) since all the three resources are originally provided as text files, organized as tables or simple list.
The first resource that we exploited for
populating SimpleLEX-IT is Morph-it!
corsi correre-VER:ind past+1+s where the features are the part of speech (PoS, VERb), the mood of the verb (indicative), the tense (past), the person (1), and the number (singular). The last released version of Morphit! (v.48, 2009-02-23) contains 505; 074 different forms corresponding to 35; 056 lemmas. It has been realized starting from a large newspaper corpus, nevertheless it is not balanced and a 2We used the PostgreSQL database . small number of also very common Italian words are not included in the lexicon, e.g. sposa (bride), ovest (west) or aceto (vinegar). Morph-it! represents extensionally the Italian language by listing all the morphological inflections, i.e. adjective, verbs, nouns inflections are represented as a list rather than by using morphological rules. We converted Morph-it! in SQL by exploiting its original feature structure: we used one single attribute to represent one single feature3. We used one table to collect all the lemmata and seven tables, with a different number of attributes, to collect the various inflected forms: the table lemmata is formed by 3 attributes: a lemma, its PoS and its ID (integer). This table contains 34; 725 records. A number of lemmata belonging to the original version of Morph-it! have been excluded in our conversion: proper nouns, emoticons and cardinals beginning with a digit (e.g. 15mila). the tables det demo table, pro demo table, pronou table are used to collect inflected form of demonstrative determiners (116 records, 4 attributes: ID word, form, ID lemma, number, gender), demonstrative pronouns (95 records, 5 attributes: ID word, form, ID lemma, number, gender), personal pronouns (63 records, 7 attributes: ID word, form, ID lemma, person, number, gender, clitics). the tables adv table, adj table, nou table, ver table are used to collect inflected form of adverbs (1; 594 records, 3 attributes: ID word, form, ID lemma), adjectives (72; 367 records, 6 attributes: ID word, form, ID lemma, kind, number, gender), nouns (35; 618 records, 5 attributes: ID word, form, ID lemma, number, gender) and verbs (392; 139, 8 attributes: records: ID word, form, ID lemma, mode, time, person, number, gender) respectively.
The second resource we exploited for
populating SimpleLEX-IT is the “Vocabolario di base
della lingua italiana” (VdB-IT henceforth), a
collection of around 7; 000 words created by the
linguist Tullio De Mauro and his team4
4The second edition of the vocabulary has been
announced
The list of lemmata of VdB has been converted in SQL by using one single table, called lemmadema (6540 records), which have two attributes, i.e. an ID (integer) and the lemma.
The third resource that we used for the
lexicon creation is Wikipedia. Our reference grammar
A notable advantage of the SQL representation for linguistic resources is the possibility to extract intrinsic information with simple queries. Indeed, 5https://it.wikipedia.org/wiki/Verbi_ irregolari_italiani
6https://it.wikipedia.org/wiki/Verbi_ incoativi we found that Morph-it! and VdB share 4; 086 nouns and 1; 448 verbs, but there are 245 lemmas belonging to VdD and not belonging to Morph-it!: most of these words are nouns, for instance lavapiatti, chimica, incinta, but we found too a systematic difference for verbs. Indeed, VdB consider as proper reflexive a number of verbs, for instance avvantaggiarsi, sdraiarsi. In contrast, these verbs are are treated as improper reflexive in Morphit!, which annotates avvantaggiare and sdraiare as their lemmata. 3
In this section we describe the algorithm used to build the computational lexicon SimpleLEXIT, which is based on the three resources described in the Section 2, and that has been used in SimpleNLG-IT.
A computational lexicon can be split in two
major classes: open and closed classes. The closed
class, that are usually composed by function words
(i.e. prepositions, determiners, conjunctions,
pronouns, etc.) is one to which new words are very
rarely added. In contrast, the open classes, that
is usually composed by lexical words (i.e. nouns,
verbs, adjectives, adverbs), accept the addition of
new words. We adopted the same strategy of
In order to build the open class for
SimpleLEXIT we needed both a large coverage and a detailed
account of morphological irregularities, also
considering their high frequency in Italian.
Moreover, in order to have good time execution
performance in the realiser (cf.
We reported in Algorithm 1 the process used for the insertion and the annotation of the words belonging to the open classes in SimpleLEX-IT. Note that in order to recognize proper reflexive verbs, we check if the infinitive form of the verb else end foreach adverb 2 Morph-it! \ VdB-IT do
Add the adverb in normal form into SimpleLEX-IT end foreach adjective 2 Morph-it! \ VdB-IT do
Add the adjective in normal form (masculine-singular) and in feminine-singular, masculine-plural, feminine-plural forms, into SimpleLEX-IT end foreach noun 2 Morph-it! \ VdB-IT do
Add the noun in normal form (singular), the plural form, and the gender into SimpleLEX-IT end foreach verb 2 Morph-it! \ VdB-IT do if verb 2 VerIrr then
Add all the inflections for the indicativo presente, congiuntivo presente, futuro semplice, condizionale, imperfetto, participio passato, passato remoto into SimpleLEX-IT if verb 2 VerInc then
Set active the incoativo feature in the entry end if the verb is properly reflexive (i.e. ”...rsi”) then
Set active the reflexive feature in the entry end
Add the verb in normal form into SimpleLEX-IT end Algorithm 1: The algorithm for building the adverbs, adjectives, nouns and verbs in SimpleLEX-IT has the postfix “rsi”, since MorphIT! contains this inflection as its normal form. In Table 1 we reported some statistics about SimpleLEX-IT composition. Most of the lexicon is composed by nouns (58%), followed by verbs (21%), adjectives (19%), and adverbs (2%).
PoS Adverb Verb (irr.) Verb (reg.) Adjective
Noun Total
The Universal Dependency Treebank (UDT) is a
recent project that releases freely available
treebanks for 33 languages (in this work, version 1.2)
We are working on the idea of adding more information in SimpleLEX-IT by using UDT-IT, i.e. the Italian section of UDT. A specific case that we are currently considering regards auxiliary verbs. The current version of SimpleNLG-IT does not manage auxiliary verbs: in order to produce some complex verb tense, e.g. passato prossimo, the user needs to give in input to the realiser the correct auxiliary, i.e. essere (to be, e.g. Io sono nato a Napoli) or avere (to have, e.g. Io ho amato la scuola.). Our reference grammar reports complex rules based on lexical semantics in order to choose the correct auxiliary verb and, unfortunately, these rules have many exceptions. So, we can use UDTIT to empirically decide the correct auxiliary in SimpleNLG-IT. By following this idea, we converted UDT-IT in SQL by exploiting its original feature structuree. We used one table to collect information about the sentences, and one table to collect information about the words: the table sentence ud is formed by 4 attributes: an ID (integer), the original treebank (i.e. TUT, ISST, etc.), the original ID, the section (i.e. DEV, TRAIN, TEST). the table words ud is used to collect all the words of the UDT-IT. It uses 21 attributes: one attribute id sentence, contains the id of the sentence in the table sentence ud, and 20 attributes correspond to the featured used in the UD annotation.
In order to find the correct auxiliary for a specific verb in UDT-IT, we need to exclude passive, reflexive and modal verb constructions in the query. We found 512 verbs of SimpleLEX-IT that are used in UDT-IT with an auxiliary. It is interesting to note that 60 verbs are used both with the auxiliary essere and with the auxiliary avere: this is grammatical for some verbs (e.g. vivere), but more often we found an annotation error in the UDT-IT.
Finally, note that another possible use of UDTIT regards the evaluation of the lexicon. In future work we plan to quantify the coverage of SimpleLEX-IT by using the TEST section of the UDT-IT. 5
In this paper we have presented some issues about the computational lexicon SimpleLEX-IT. We described the algorithm used to build the lexicon, three SQL resources produced as side effects of the lexicon building and a work in progress about the extraction of syntactic information from UDIT.
All the resources described in this paper can be downloaded at https://github.com/ alexmazzei/SimpleLEX-IT.