In this study, we aim to ofer linguistically motivated solutions to resolve the issues of the lack of representation of null morphemes, highly productive derivational processes, and syncretic morphemes of Turkish in the BOUN Treebank without diverging from the Universal Dependencies framework. In order to tackle these issues, new annotation conventions were introduced by splitting certain lemmas and employing the MISC (miscellaneous) tab in the UD framework to denote derivation. Representational capabilities of the re-annotated treebank were tested on a LSTM-based dependency parser and an updated version of the BoAT Tool is introduced.
and syntax of the target language, the better the performance of the NLP systems using that treebank as a resource.
In this paper, we aim to abide by the linguistic framework set by Bedir et al. [
The decisions made in the re-annotation process of the BOUN Treebank aim to ofer solutions to the issues posed by the morphologically rich and complex nature of Turkish: null morphemes are frequently employed, agglutinative processes are heavily used to create new forms, and numerous morphemes like copula and -ki are very syncretic. The main goal of this study is to illustrate these phenomena without compromising the compliance with the UD framework.
This paper is organized as follows. Previous attempts at creating dependency treebanks in Turkish are laid out in Section 2. Annotation changes made in the current version of the BOUN Treebank and their linguistic justification are discussed in Section 3. Statistics regarding the changes made to the previous version are stated in Section 4. Improvements made to the BoAT Annotation Tool are explained in Section 5. Finally, the performance of the parser trained using the current version of the treebank is reviewed in Section 6.
Shortly after the rfist dependency treebank for Turkish was presented by Atalay et al. [
Tourism is one of the two domain-specific dependency treebanks in Turkish and consists of hotel and restaurant reviews. The other one is ATIS treebank that covers the Turkish translation of English ATIS (Airline Travel Information System) corpus [13].
Other Turkish UD-style treebanks include Kenet UD Treebank, Penn Treebank, and FrameNet treebank. Penn Treebank consists of Turkish translations of English Penn Treebank [14] while FrameNet consists of 2,700 sentences from the Turkish FrameNet database [15].
With 9,761 sentences and 121,214 tokens randomly selected from Turkish National Corpus (TNC) [16], the BOUN Treebank is one of the largest UD-style treebanks in Turkish. Covering ifve diferent registers (broadsheet national newspapers, biographical texts, essays, popular culture articles, and instructional texts), it ofers word order and sentence length variance in addition to linguistically motivated dependency annotations (see Section 3).
The first step of the previous annotation process of the BOUN Treebank (see [
For the re-annotation process, a team of linguists detected shortcomings and problematic
annotations of the previous version of the BOUN Treebank [
Annotations were done by two linguists who are native speakers of Turkish. In order to ensure inter-annotator agreement, 100 sentences were double annotated. Unlabelled attachment and labelled attachment scores were calculated using Cohen’s Kappa measure. Their respective values are 98.61 and 97.81.
Having its focus on syntax, the UD framework falls short of representing derivational processes. The oficial guide of UD argues that the final derivational sufix in a word is opaque in the sense that it does not permit access to the morphemes that come before it. Hence in a construction as the one shown in Figure 1, numerous derivational and inflectional morphemes before the last derivational morpheme (-ki) are lost.
As one of the primary concerns of this study was finding ways to illustrate derivation processes like those in Figure 1 without diverging from the UD framework, two strategies were employed for diferent cases: • For morphemes like -lI (“with” ), df= function is introduced in the MISC tab. • Lemmas containing -ki morpheme are splitted.
These derivational morphemes imply that the host lemma and its modifier form a syntactic unit together. As a result, the correct bracketing of the -lI adjective, its modifier, and the constituent they modify together should look like (2) instead of (1). Representing the derivational morphemes like -lI and -sIz allows keeping this crucial syntactic information.2 (1) [ [ [kahverengi] tüy-lü] kedi]
brown fur-attr cat “a cat with brown fur” (2) [ [kahverengi tüy]-lü kedi]
brown fur-attr cat “a cat with brown fur”
With the new df= function proposed in this study, “tüylü” is not decomposed as two lemmas: “tüy” (“fur” ) and “lü” (“with” ). Instead, it is left intact and df=tüy (“derived from=fur") function is annotated in the MISC tab.
Another challenge regarding the derivational processes is posed by -ki. There are two diferent -ki morphemes in Turkish [18]. One is used as noun and the other derives adjectives from nouns.
Statistically, the vast majority of inflectional sufixes in Turkish occur after the derivational
ones. However, both types of the -ki morphemes diverge from this distribution as it can be
observed in Figure 1, where the inflectional sufix -ler precedes the derivational -ki. Hence
allowing -ki to “block access” [
Languages such as Turkish, Russian, Arabic, and Coptic have null morphemes, however, the UD framework does not oficially support such phenomena. As a result, independent strategies have emerged to represent null morphemes in these languages. For example, Coptic avoids null subject nodes by using fused forms[19], Marathi annotates the feature values of the null morphemes, however, does not introduce any information (i.e. annotation) to indicate that they are null morphemes[20]. Widely employing null morpheme for pluralization, Arabic distinguishes between two types of annotation: Form-based and function-based. In their UDstyle dependency treebank annotation, Marton et al.[21] follow a function-based annotation framework and annotate the feature values of null morphemes.
In Turkish, the copula can surface in three diferent forms [ 10]: i-, -y-, and ∅.After considering UD guidelines and particularities of Turkish copula, it was decided to employ the MISC tab 2Abbreviations: attr = attributive.
3It is stated in the UD guidelines that “the lemma does not remove derivational morphology, so the lemma of [en] “organizations" is “organization" not “organize" (nor “organ”).” See https://universaldependencies.org/u/ overview/morphology.html for the full picture.
4Refer to the Appendix to compare previous and updated annotation schemes for both -ki morphemes.
again by introducing two new functions for the null copula: nullcop=3s (singular) and
nullcop=3p (plural). By following a function-based annotation schema, we were able to ofer
more linguistically accurate annotations without diverging from the UD framework.
3.1.3. Copula
In Turkish, ol- copula has six distinct functions [
In an attempt to overcome the challenges thoroughly discussed in the previous subsection, a set of new XPOS tags and dependency relations were introduced in the updated version of the BOUN Treebank. A comprehensive list can be found in Table 1.
Lemma var UPOS XPOS Deprel
NOUN Exist root yok opla-r(taicfteirple) seoeml-nb(tieenndcdeesd) cool-n(sitnr ulicgthiotnvse)rb NOUN AUX AUX VERB Exist Ptcp root aux cop compound:lvc vooelr-ri(bna)tsratrnasnitsiivtieve -(kaidjectivizer) (-pkrionominal) VERB PART PRON
Attr Partic root dep:der
As part of the re-annotation process, 117,732 changes were made in the following tabs: UPOS, XPOS, Deprel, MISC, and Features. The majority of annotation changes targeted UPOS and XPOS tags (see Table 2). Since morphological information was automatically annotated in the previous version of the treebank by the parsing tool [17], refinements by the annotators were required in order to ensure accuracy.
11,396
63,829
27,098
23,32
4,973
The changes in UPOS values reflect the linguistics-based decisions made in the re-annotation process. Previous version of BOUN Treebank made no distinction between two -ki morphemes in Turkish. As a result, almost all -ki instances were labeled as CConj (clausal conjunction). After deciding to make a distinction between adjectivizer -ki and pronominal -ki, the UPOS tag of the former was changed to Part.
Field Change Count
Adj ->Noun 1,595
UPOS CConj ->Part Noun ->Propn 1,025 968
XPOS Verb ->Ptcp Verb ->Vnoun 2,459 1,664
ANum ->Indef 1,622
Due to the shortcomings of automatic morphological tagging, some proper nouns were labeled as nouns. Re-annotation process targeted them as well: UPOS tags of 986 proper nouns were changed. In addition, XPOS tags of verbal nouns and participle forms were updated (see Table 3).
The BoAT tool ofered in Türk et al. [
BoAT is a desktop application, written in Python and based on Qt. The Qt version has been incremented from 5 to 6. This resulted in a more modern-looking user interface (UI). With ample feedback from the annotators who had used the tool for the BOUN Treebank, some requested features and improvements have been surfaced.
Clutter: Annotation table’s columns were being shown or hidden by checkboxes above the table. These were removed and a textbox beside the other buttons has been added to replace them. This textbox serves exactly the same purpose while taking less space.
Autocompletion: Annotators use the tool for long hours at a time. Thus after a while, mistakes tend to occur. Another requested feature, autocompletion of the table fields, aims to prevent such mistakes. Many fields of the table have predetermined sets of values they can take. By not allowing values outside these sets and having a shorthand writing system whereby an annotator enters only the start of a value and it gets filled automatically, we implemented this much requested feature.
Saving as CoNLL-U : Another change was regarding saving of the CoNLL-U documents. The initial tool saved every edit automatically, yet treebanks with thousands of sentences tend to take time to save. This mishap seemed to slow our annotators. We added a save button instead of the autosave feature. Currently, a user uses the save button to edit the actual CoNLL-U file.
Shortcuts: The initial tool already had shortcut support. Going with the focus-oriented approach, all the new tasks have keyboard shortcuts associated with them as well. This alleviates the need to use a mouse while annotating via keyboard.
Dependency graphs: The vertical dependency graph in the initial version was replaced by the horizontal dependency graph of spaCy [22], displaCy, due to the spatial concerns.
Validation: The validation script for annotations has been upgraded to the latest version written by the UD framework, which has much more detailed explanations for why a specicfi annotation is invalid.
Within the scope of this study, an NLP task was conducted. BiLSTM-based biafine dependency parser proposed by Dozat and Manning [23] was trained using the updated BOUN Treebank. The train set contained 7.803 sentences, development set contained 982 sentences and test set contained 979 sentences. Considering the average arc length and average token count (see Table 4 for details) of each set, BOUN Treebank ofers a well-balanced data set.
2.91
The previous version of the BOUN Treebank [
In order to better account for the particularities of Turkish morphosyntax, four new dependency labels were introduced: dep:der for adjectivizer -ki, obl:tmod for obliques that ofer temporal information regarding the predicate, advmod:emph for dA clitics, and compound:lvc for light verb constructions with ol- copula. Moreover, new use cases for cop dependency label were ofered to represent diferent functions of the ol- copula. These changes in the annotation framework added four new dependency types, thus the total class number was increased.
A sum of 1,032 dep:der, 894 obl:tmod, 1,860 advmod:emph, and 1,545 compound:lvc tags were added. Newly introduced tags and classes introduced added complexity, hence they might be the reason behind the slight decrease in the LAS results.
The main annotation changes made as part of this study were focused on morphology yet BiLSTM-based biafine dependency parser proposed by Dozat and Manning [ 23] doesn’t refer to the morphological information. In fact, it almost completely ignores the morphology features while parsing. Hence the significance of the improvements ofered by this study can be better gauged by a morphology-aware parser or a morphological analysis based downstream task.
The UD framework highly emphasizes syntactic relations and aims to ofer a universal foundation to represent typologically diferent languages in a uniform way. While doing so, certain particularities of these languages tend to get lost in the annotation process in an attempt to abide by the UD convention. The aim of this study is to ofer linguistically sound solutions to illustrate syntactically relevant morphological features of Turkish such as null morpheme realizations and derivational processes without diverging significantly from the UD framework.
In order to test the morphological capabilities of the re-annotated BOUN Treebank, a parser that refers to morphological information can be implemented in further research.
This work was supported by Boğaziçi University Research Fund Grant Number 16909. TUBAGEBIP Award of the Turkish Science Academy (to A.O.) is gratefully acknowledged. [10] A. Göksel, C. Kerslake, Turkish: A comprehensive grammar, Routledge, 2004. [11] U. Sulubacak, G. Eryiğit, Implementing universal dependency, morphology, and multiword expression annotation standards for turkish language processing, Turkish Journal of Electrical Engineering & Computer Sciences 26 (2018) 1662–1672. [12] T. Pamay, U. Sulubacak, D. Torunoğlu-Selamet, G. Eryiğit, The annotation process of the itu web treebank, in: Proceedings of the 9th Linguistic Annotation Workshop, 2015, pp. 95–101. [13] C. T. Hemphill, J. J. Godfrey, G. R. Doddington, The atis spoken language systems pilot corpus, in: Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990, 1990. [14] A. Taylor, M. Marcus, B. Santorini, The penn treebank: an overview, Treebanks (2003) 5–22. [15] B. Marşan, N. Kara, M. Özçelik, B. N. Arıcan, N. Cesur, A. Kuzgun, E. Sanıyar, O. Kuyrukçu, O. T. Yıldız, Building the turkish framenet, in: Proceedings of the 11th Global Wordnet Conference, 2021, pp. 118–125. [16] Y. Aksan, M. Aksan, A. Koltuksuz, T. Sezer, Ü. Mersinli, U. U. Demirhan, H. Yılmazer, G. Atasoy, S. Öz, İ. Yıldız, et al., Construction of the turkish national corpus (tnc), in: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), 2012, pp. 3223–3227. [17] J. Kanerva, F. Ginter, N. Miekka, A. Leino, T. Salakoski, Turku neural parser pipeline: An end-to-end system for the conll 2018 shared task, in: Proceedings of the CoNLL 2018 Shared Task: Multilingual parsing from raw text to universal dependencies, 2018, pp. 133–142. [18] J. Hankamer, Why there are two–ki’s in turkish, Current research in Turkish linguistics,
Eastern Mediterranean University (2004). [19] A. Zeldes, M. Abrams, The coptic universal dependency treebank, in: Proceedings of the
Second Workshop on Universal Dependencies (UDW 2018), 2018, pp. 192–201. [20] V. Ravishankar, A universal dependencies treebank for marathi, in: Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, 2017, pp. 190–200. [21] Y. Marton, N. Habash, O. Rambow, Improving arabic dependency parsing with form-based and functional morphological features, in: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, 2011, pp. 1586–1596. [22] Explosion, spaCy, 2022. URL: https://spacy.io/, [Online; last accessed 27 March 2022]. [23] T. Dozat, P. Qi, C. D. Manning, Stanford’s graph-based neural dependency parser at the conll 2017 shared task, in: Proceedings of the CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies, 2017, pp. 20–30.
Form Lemma UPOS başındaki başındaki ADJ şapkası şapka NOUN PART NOUN nmod dep:der root
Features Case=Nom|Number=Sing
Deprel root PRON PRON PERS Partic
Features
Head
Deprel
MISC Case=Gen|Number=Sing|Person=2|PronType=Prs 2 Case=Nom|Number=Sing 0 nmod:poss root