The Universal Dependencies Project1 (Nivre, [9]; Nivre et al., [10]) is an ongoing effort towards creating a set of harmonised dependency treebanks that are annotated and structured according to universal guidelines. This paper reports on the addition of morphological features to the Irish Universal Dependencies Treebank (IUDT). Our feature set subscribes to the feature inventory of the UD Project and has been mapped from Irish morpho-syntactic tags - the output of a Finite State Morphological Analyser for Irish (Uí Dhonnchadha and van Genabith [16]). Irish, a Celtic language, has some relatively unusual morphological features that require language-specific labels not covered by the universal feature set. In this paper, we summarise the Irish-specific features that we have added to this set by explaining the linguistic properties that they each describe. We also report on the first parsing experiments using the IUDT by assessing the effect that the inclusion of morphological features has on parsing accuracy.
The motivation behind the Universal Dependencies Project (Nivre, [
Irish is a relatively low-resourced language amongst the UD treebank
collection. The inclusion of low-resourced languages in large projects like this means
that the language can benefit from the experience and contributions of the wider
research community. The Irish Universal Dependency Treebank (IUDT) was
developed as a result of a mapping of the Irish Dependency Treebank (IDT)
annotation scheme (Lynn [
In this paper, we discuss our extension of the IUDT to include morphological features. We follow the UD guidelines2 for feature inclusion. While the UD project defines a feature set that aims to cover linguistic universals shared across the numerous3 languages that are part of the project, it is also possible for each language group to define their own language-specific features where necessary. For example, one of the Irish-specific UD features relates to initial mutation – a linguistic phenomenon that is common across Celtic languages (see Section 4.1). However, initial mutation does not feature widely in other languages, and is therefore not deemed ‘universal’. Here, we list the morphological tags from the UD feature set that are relevant to Irish and additional new Irish-specific features.
Additionally, we consider that parsing models for morphologically rich
languages can benefit from the inclusion of morpho-syntactic features given sufficient
training data. This additional level of annotation in dependency treebanks, at the
morpho-syntactic level, contributes to a deeper understanding of the data that may
not be achieved through part-of-speech (POS)-tags or dependency labels alone.
This has been demonstrated in various studies, including shared tasks on the
parsing of morphologically rich languages (e.g. Seddah et al, [
In computational terms, Irish is regarded as a morphologically rich language (Lynn
et al [
lowing is a short summary of the main inflectional processes in Irish. For a more
detailed description, see The Christian Brothers [
Nouns In terms of nominal inflection, Modern Irish only uses three cases: common, genitive and vocative. The common case covers nominative, accusative and dative, yet it should be noted that the dative case is marked on personal pronouns. Each noun falls into one of five declensions. Case, declension and gender are all expressed through inflection.
Verbs Verbs can be marked for both tense and aspect, and inflect for person and number (e.g. ith ‘eat’; d’ithimis ‘we used to eat’).
Adjectives The Christian Brothers ([
The IDT and IUDT were built upon a gold-standard POS-tagged corpus of Modern
Irish, which was developed by Uí Dhonnchadha [
The following is an example sentence labelled with FST tagset output: Tá sé soiléir ‘It is clear’. Each token is followed by a string of POS tags and morphosyntactic features, separated by ‘+’. The string sé sé+Pron+Pers+3P+Sg+Masc tells us that sé (with the same lemma form) is a 3rd person singular personal pronoun, of masculine gender.
(1) Tá sé soiléir ‘It is clear’.
Tá bí+Verb+PresInd sé sé+Pron+Pers+3P+Sg+Masc soiléir soiléir+Adj+Base
Until now, the IUDT contained only the POS tags (coarse- and fine-grained) of this feature set, i.e. +Pron+Pers, indicating personal pronoun. Our purpose of now including additional morphosyntactic features in treebank data is to mark additional lexical and grammatical properties of words that are not available through POS tags alone. As shown in (1) above, such additional morphological features for Irish data are readily available for inclusion in the data set, but need to be converted to the correct representation and align with the UD guidelines.
The IUDT is in the CoNLL format (Buchhloz [
The Universal Dependencies feature set is a standardised list of morphological features, which is based on the Interset system (Zeman [17]). It is divided into lexical and inflectional features. Mapping to the UD feature set was straightforward for the majority of the tags in the Irish Morphosyntactic Tagset. However, we have introduced a number of non-universal features and values in order to fully represent specific linguistic phenomena in the Irish language (see Section 4.1). Table 1 presents the full inventory. Rather than discuss the details of entire feature set here, we only discuss in detail the newly introduced Irish language-specific features (see bolded features). For information on all other standard (universal) features, we point the reader to the Universal Dependencies website.5 4.1
There are a number of morphosyntactic features in Irish for which the UD universal feature set does not fully account. This is not surprising, as while all languages share certain linguistic universals, what makes languages unique is often seen in their differences across syntax or morphology. For a more fine-grained annotation, we extend the UD feature set to cover these nuances in Irish. Here, we provide a short description of each of the Irish-specific features.6 Dialect=Munster, Connaught, Ulster There are three main dialects of Irish and there are a number of lexical items that differ across them. Some surface forms differ, while sharing the same lemma. For example, the past tense form of the lemma cuir ‘put’ is chuireas in the Munster dialect, but chuir in the Connaught and Ulster dialects. Other lexical items differ in both surface and lemma form. For
5http://universaldependencies.org/u/feat/all.html 6Not requiring much discussion, note that the Feature Value NomAcc will replace the incorrectly used Com tag in future versions of the IUDT to indicate for the common case in Irish. Com is used in other UD treebanks to indicate the Comitative case. Feature Name
Degree Dialect† Definite Form† Gender Mood Negative NounType† NumType Number
Poss PrepForm† PronType Reflex Tense VerbForm Voice
Feature Value ILneflxeiccatiloonral? Dat, Gen, NomAcc†, Voc Inflectional Cmp, Pos, Sup Inflectional Connaught, Munster, Ulster Inflectional Def, Ind Inflectional Ecl, Emph, hPref, Len, VF Inflectional Fem, Masc Inflectional Cnd, Imp, Ind, Int†, Sub Inflectional Neg Inflectional NotSlender, Slender, Strong, Weak Inflectional Card, Ord, Pers Lexical Plur, Sing Inflectional IAndf,, NCummp,l,PCaot,mVpb,,CVoopc, Deg, Lexical 1, 2, 3 Inflectional Yes Lexical Cmpd Lexical Art, Dem, Ind, Int, Prs, Rel Lexical Yes Lexical Fut, Past, Pres Inflectional Cop†, Ger, Inf, Part Inflectional Auto† Inflectional Prevalence example, the word achan ‘every’ is used in Ulster dialect, but gach ‘every’ is used in the Munster and Connaught dialects.
Form=Ecl, Emph, Len, hPref Initial mutation is a feature of Celtic languages.
It is triggered by a preceding word and affects the spelling of nouns, adjectives
and verbs. Eclipsis (Ecl) on verbs occurs following clitics such as interrogative
particles (an, nach); complementisers (go, nach); and relativisers (a, nach) (Stenson
[
PartType=Ad, Cmpl, Comp, Cop, Deg, Inf, Num, Pat, Vb, Voc Irish makes use of a broad range of particles. It is important to differentiate these particles, because in some cases they share the same form, yet have different functions. For example, a can be a vocative particle, relative particle, infinitive particle or quantifier particle, and trigger a variety of different spelling changes (e.g. eclipsis, h-prefix or lenition).
PrepForm=Cmp Compound prepositions contain a simple preposition and a noun. Nouns that follow compound prepositions are inflected in the genitive case. For example, ar fud ‘throughout’ and domhan ‘world’, when combined, becomes: ar fud an domhain ‘throughout the world’.
VerbForm=Cop Irish has two verbs ‘to be’ – the copular verb and the substantive
verb (Stenson [
Voice=Auto Irish does not have an equivalent to the English passive construction
(The Christian Brothers (1988, p.120) and Stenson [
Treebank Features LAS (test) UAS (test) IUDT (baseline) no 68.27 76.29 IUDT yes 70.63 77.57 IUDT + Opt yes 72.18 78.3 IUDT universal only 69.26 76.99 IDT no 69 78.33
LAS (10-fold) UAS (10-fold) IUDT 10-fold yes 72.19 79.12 LAS (dev) UAS (dev) 70.59 78.73 71.79 79.45 73.33 79.52 70.70 78.66 70.66 79.8
In this section we discuss a number of parsing experiments carried out on the
IUDT. We use MaltParser (Nivre et al., [
In order to assess the benefit of including morphological features in the IUDT,
we report here on parsing experiments that are based on models trained on two
7MaltParser v1.7, available to download from http://www.maltparser.org/download.html
different versions of the treebank:
our baseline, which is the original IUDT (v1.0)8, (Lynn and Foster [
In this experiment, we follow the development/test/training treebank split, as per the UD convention, i.e. roughly 10%-10%-80%: dev (150 trees)/ test (150 trees)/ training (remaining = 720 trees). We can see from the results presented in Table 2 that the inclusion of the morphological features in the IUDT leads to a clear increase in parsing accuracy. As the only difference across the two experiments is the presence or absence of morphological data (e.g. there is no change in annotation scheme, standard treebank data or MaltParser settings), we can conclude that the morphological information has a positive impact on parsing accuracy. 5.2
With the promising results we get from including morphological information in
the training data, we are prompted to evaluate the use of MaltOptimizer
(Ballesteros [
MaltOptimizer defined the optimal training options for the IUDT dataset as the Nivre arc-eager parsing algorithm with normal root handling and a LIBLINEAR learner classifier. As we show in Table 2, this configuration resulted in a LAS increase of 1.55 and UAS increase of 0.73 on the test set. The improvement on the results of the dev set were similar for LAS (1.54), but less so for UAS (0.07). 5.3
In Section 4.1, we list and describe the Irish language-specific features that we have introduced to our feature set. We acknowledge that we are motivated to specifying these new features as we feel that this linguistic phenomena is integral to the syntax of the Irish language and should be captured in the data. This intuition comes 8http://hdl.handle.net/11234/1-1464 9https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-1827 10Covered roots are cases where the root (head value = 0) is crossed by one or more arcs. from a knowledge of the language, yet it is also important to empirically justify this inclusion. For this reason, we experiment on a version of the IUDT treebank that includes only the universal features, and excludes the Irish-specific features. Our results are also presented in Table 2, where we can see that removing the Irish specific features results in a drop in parsing accuracy, thus supporting the motivation for their general inclusion in the latest version.
On closer analysis of the parser output (development set), we observe exactly how some of these features help to improve parsing. For example, the feature Voice=Auto – which we explain in Section 4.1 indicates the autonomous verb – appears to help the parser in correctly identifying the direct object (dobj) argument of the verb. Without this feature, confusion arises when the parser assumes that the noun following the autonomous verb is the subject, thus assigning the nsubj label attachment. In some instances, correct identification of the verb–direct object attachment resulted in a knock-on effect of improved parsing of the overall tree. Likewise, we observe that the morphological feature PrepForm=Cmpd has helped the parser to more accurately identify the correct dependency attachment and labelling of case to compound prepositions, where previously parser confusion led to the incorrect assignment of either root or nmod labels. Again, improvements in this labelling had a knock-on effect in some cases, thus improving the overall parse of the tree. 5.4
To put the parsing results for Irish Universal Dependencies into context, we look at comparing our IUDT parsing results with those of the original Irish Dependency Treebank (IDT). It should be noted that the IDT does not yet contain morphological features. The results are presented towards the bottom of Table 2.
A number of factors can affect the quality of parsing, such as the design of
an annotation scheme, the number of dependency labels that are used (including
granularity of labels) and the settings used in a parsing framework. As reported
by Lynn and Foster [
We also note that adding morphological features to the IUDT increased parser accuracy to a level which is on a par with the IDT parser (without morphological features). This could suggest that the inclusion of morphological features overcomes the potential loss of information introduced when mapping treebank annotations from an Irish-language inspired scheme to a universal scheme. However, it should be noted that strong conclusions cannot be drawn from this, as the MaltParser settings chosen for these parsing experiments were optimised and tuned to the IDT and may not necessarily be optimal for the IUDT.11 5.5
It is worth noting that the baseline results provided here for the IDT are actually lower than those previously reported by Lynn (2016) (LAS 71.4% and UAS 80.1%). The reason for this is the difference in the data splitting approach for the dev/test/ train sets. As mentioned in Section 5.1, our IDT (and IUDT) results presented here are based on the data split convention of the UD project. Prior reported IDT experiments, however, have been carried out using k-fold cross-validation.
While a conventional data-split approach across a large project is well motivated, parsers trained and tested on small datasets may not always reflect the full truth of attainable parsing accuracy. Instead, parsing model evaluation is often carried out using k-fold cross-validation when working with smaller data sets. This approach involves splitting the data into k sets of equal size, one set used as a test set and the remaining as training data, and repeated iteratively until all partitions have served as a test set once. The result is an average across k test set results.
Thus, in order to acquire a more realistic set of results in our current experiments, we perform 10-fold cross-validation on the IUDT with morphological features to get a clearer idea of parsing quality. Our results show an increase across both LAS (70.63% – 72.19%) and UAS (77.57% – 79.12%). 6
We have described the morphological features recently introduced to the Irish Universal Dependency treebank, in the form of both standard UD features and features introduced specifically for the Irish language. Parsing experiments with MaltParser suggest that these morphological annotations are helpful, and results can be further improved using MaltOptimizer. Ablation experiments demonstrate that the Irishspecific morphological information has a useful role to play. 7
We would like to thank the anonymous reviewers for their valuable feedback and Lamia Tounsi for her analysis advice. This work has been supported by the ADAPT Centre for Digital Content Technology (www.adaptcentre.ie) at Dublin City University, funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
11The MaltParser configurations on previously reported Irish parsing experiments included stacklazy and liblinear learner algorithms. The feature set included word form, lemma and both coarse and fine-grained POS tags.