This paper presents different approaches towards universal dependency parsing for Swiss German. Dealing with dialects is a challenging task in Natural Language Processing because of the huge linguistic variability, which is partly due to the lack of standard spelling rules. Building a statistical parser requires expensive resources which are only available for a few dozen high-resourced languages. In order to overcome the lowresource problem for dialects, approaches to cross-lingual learning are exploited. We apply different cross-lingual parsing strategies to Swiss German, making use of Standard German resources. The methods applied are annotation projection and model transfer. The results show around 60% Labelled Attachment Score for all approaches and provide a first substantial step towards Swiss German dependency parsing. The resources are available for further research on NLP applications for Swiss German dialects.
Swiss German is a dialect continuum of the Alemannic dialect group, comprising numerous varieties used in the German-speaking part of Switzerland.1 Unlike other dialect situations, the Swiss German dialects are deeply rooted in the Swiss culture and enjoy a high reputation, i.e. dialect speakers are not considered less educated as it is the case in other countries. On the basis of their high acceptance in the Swiss culture and with the introduction of digital communication, Swiss German has undergone a spread over all kinds of communication forms and social media. Despite being oral languages, the dialects are used increasingly in written contexts, and writers spell as they please.
For Natural Language Processing (NLP), lowresourced languages are challenging, particularly in cases like Swiss German where no orthographic rules are followed. Compiling NLP resources from scratch such as syntactically annotated text corpora (treebanks) is a laborious and expensive process. Thus, in such cases, cross-lingual approaches offer a perspective to get started with automatic processing of the respective language. Such approaches are especially promising if a closely related resource-rich language is available, which is the case for Swiss German.
The Universal Dependencies (UD) project aims at developing and setting a standard for crosslinguistically consistently annotated treebanks in order to facilitate multilingual parsing research. We support this idea by adopting the current UD standard as much as possible.
The information about which word of the sentence is dependent on which other one is important in order to correctly understand the meaning of a sentence. Thus, it is needed for numerous NLP applications like information extraction or grammar checking. The task of identifying these dependencies is done by a dependency parser (see Figure 1 for a Swiss German example in UD).
In this paper, we apply two different cross-lingual dependency parsing strategies, namely annotation projection as a lexicalised approach, and model transfer as a delexicalised approach. We manually create a gold standard in order to evaluate and compare the different strategies. Furthermore, we build and evaluate
Even though there have been several projects
involving Swiss German
Research in dependency parsing has increased
significantly since a collection of dependency treebanks has
become available, in particular through the CoNLL
shared tasks on dependency parsing
There are two main approaches to cross-lingual
syntactic dependency parsing. Firstly, the delexicalized
model transfer of which the goal is to abstract away
from language-specific parameters, i.e. train
delexicalised parsers. The idea is based on universal
features and model parameters that can be transferred
between related languages. Hence, this method assumes
a common feature representation across languages.
The advantage of the model transfer approach is that
no parallel data is nee
The main idea of the second approach, the
lexicalised annotation projection method, is the mapping
of labels across languages using parallel sentences and
automatic alignment. It includes projection heuristics
and usually post projection rules. The main drawback
of this approach is that it relies on sentence-aligned
parallel corpora. In order to deal with this restriction,
treebank translation has emerged where the training
data is automatically translated with a machine
translation system. The central point of this method is the
alignment along which the annotations are mapped
from one language to the other. Automatic word
alignment has already been used by Yarowsky et al.
(2001); Aepli et al. (2014) an
We use the German Universal Dependency treebank2
consisting of 13,814 sentences. It is annotated
according to the UD guidelines3 and contains
Universal POS (UPOS) tags
Annotation projection requires a parallel corpus.
The AGORA citizen linguistics project4 crowdsourced
Standard German translations of 6,197 Swiss German
sentences via the web site dindialaekt.ch. The
sentences are taken from the NOAH corpus
We apply two classical parsing approaches presented
in Section 2: model transfer with a delexicalised
parser and annotation projection with crowdsourced
parallel data. Within both approaches we test two
parsing frameworks; the MaltParser
The delexicalised model transfer approach is straightforward, working on the basis of POS tags only. For the training, the words in the Standard German corpus are replaced by their POS tags. Accordingly, at parsing time the Swiss German words are replaced by their POS tag before parsing and re-inserted afterwards. 4.1.1
Part-of-speech tagging is an important step prior to parsing because the syntactic structure builds upon the 5The Levenshtein distance (Levenshtein, 1966) measures the difference between two sequences of characters. Hence, the minimal edit distance between two words is the minimum number of characters to be changed (i.e. inserted, deleted or substituted), in order to make them equal. POS information. Obviously, when training delexicalised parsers, this step is crucial as the tags are the only information available to the parser.
For POS tagging Swiss German sentences, we used
the Wapiti
The CoNLL-format includes UPOS tags in addition to the fine-grained language-specific POS tags S(TTS in the case of German and Swiss German). We used the mapping provided by the UD project in order to infer the UPOS tags from the given STTS tags. 4.2
Annotation projection is not only more complex in
processing compared to model transfer but also needs
more resources. Most importantly, annotation
projection requires a word-aligned parallel corpus. Starting
from the crowdsourced sentences which are
sentencealigned, it is the task of a word aligner to compute
the most probable word alignments, i.e. the
information about which word of the (Swiss German) source
sentence corresponds to which word of the target
sentence, i.e. the translation. There are many tools for
this as it is a basic step also in machine translation
systems. We tested three of them: GIZA++
The idea of the annotation projection process is to use the tool (here: parser) of a resource-rich language on that language (here: German) and then project the generated information (here: universal dependency structures) along the word alignment to the target language (here: Swiss German). In practice, this means we train the parser on the Standard German treebank (see Section 3.1) and parse the Standard German translations of the Swiss German original sentences. Then we project the resulting parse structure along the word alignments from the German word to the corresponding Swiss German word. 4.2.1
The transfer is the core component of annotation projection. The parse of the Standard German translation is projected along the word alignment to its Swiss German correspondent. The input consists of the Standard German parse and the alignment between the Standard German sentence and its Swiss German version (GSW:DE). Algorithm 1 describes the projection process.
Result: DE parse transferred to GSW for word alignment in sentence do if 1:1 alignment then
transfer parse of DE else if 1:0 alignment (i.e. no DE word aligned) then attach GSW word to root as POS tag
ADV and dependency label advmod else 1:n alignment (i.e. several DE words aligned) transfer parse of aligned DE word with smallest edit (Levenshtein) distance end end
The case of 1:1 alignment where exactly one German word is aligned to the Swiss German word is easy; the only thing to do is projecting the dependency of the German word to the Swiss German word. If, however, there are several German words aligned to one Swiss German word (1:n), the algorithm has to decide which parse to transfer.6 In order to take this decision, the algorithm computes the Levenshtein 6Note that the case of a 1:n alignment between a GSW word and a DE multiword expression is not covered in this approach. distances (Levenshtein, 1966) between the Swiss German word and every aligned German word and takes the one with the smallest edit distance. The most challenging case is when no German word is aligned to the Swiss German token. A simple baseline approach attaches the corresponding Swiss German word as adverbial modifier to the root of the sentence.
The decision to treat every unaligned Swiss German word as an adverb is taken on the basis of the frequency distribution of POS tags; ADV is the second most frequent POS tag (after NN) in the Swiss German data. However, taking into consideration the word itself, some more sophisticated rules can be elaborated. Considering the differences between Standard German and Swiss German as described by Hollenstein and Aepli (2014), we can expect some words like infinitive particles (PTKINF) (e.g. go) or the past participle gsi (been) to remain unaligned. The former because these words do not exist in Standard German, the latter because Standard German simple past tense is expressed by perfect tense in Swiss German, typically resulting in a “spare” past participle in the alignment. Furthermore, there are unaligned articles because Swiss German requires articles in front of proper names. Also punctuation including the apostrophe is a source of errors which can easily be corrected. The application of these more elaborate rules have an impact of around 2 points on the evaluation scores.
Algorithm 1 transfers the German parses as they are, as a consequence the numbering of the token IDs is mixed up. Correcting the token IDs to be in ascending order (from 1 to the length of the sentence) requires the corresponding adjustment of the head references. Furthermore, one needs to make sure that there is exactly one root in a sentence.
end
for sentence in parse do if DE root was not projected to GSW parse then
take 1st VERB as root, else 1st NOUN else if head of a projected word was not projected to GSW parse then
attach it to the root end Algorithm 2: Correction of transferred parses.
Algorithm 2 goes through every sentence of the input file and first makes sure that there is one root for the sentence. If the root of the Standard German parse has not been transferred to the Swiss German sentence (missing word alignment), the first verb (UPOS VERB) is taken as root and if there is no VERB in the sentence, the firstNOUN is considered the root. 4.3
We tested two approaches for optimisation; preprocessing of the training set and postprocessing rules to be applied after parsing. 4.3.1
One frequent mistake mostly observed in the delexicalised approach is the wrong assignment of passive dependency labels instead of their active counterpart. The passive construction in Standard German is built with the auxiliary werden, which can, however, also be used in non-passive constructions. The combination of VA* and a perfect participle (VVPP) is very frequent in Swiss German, however, it is usually not a passive construction but rather a perfect tense. Therefore, a simple but effective solution is the introduction of a new “set” of POS tags in the German UD training set: VWFIN, VWINF, VWPP for finite verbs, infinitives and participles respectively of the verb werden. This means, all occurrences of the lemma werden as an auxiliary (i.e. UPOS: AUX and STTS: VA{INF|PP}) are replaced by VW{INF|PP}. In this way, the system learns to discriminate between the usage of werden as auxiliary versus the usage as full verb and, most of all, it learned to differentiate between the auxiliary werden and the other auxiliaries haben (to have) and sein (to be). Hence, the number of wrongly assigned passive dependency labels decreased, which leads to an improvement of around 2.5 to 3.5 points as presented in Section 5. 4.3.2
Some of the errors can easily be corrected with simple rules in a postprocessing step. One example is a frequent error caused by a remnant of the 1st UD version which is handled differently in UD version 2. The two labels oblique nominal (obl) and nominal modifier (nmod) are confused because the latter was used to modify nominals and predicates in UD v1. However, in UD v2, obl is used for a nominal functioning as an oblique argument, while nmod is used for nominal dependents of another noun (phrase) only. This means, if the head is a verb, adjective or adverb, the dependency label has to be obl. If, instead, the head is a noun, pronoun, name or number, the dependency label is nmod. 5
This section presents the different settings and combinations of aforementioned resources, approaches and tools. For the evaluation, we manually created a gold standard consisting of 100 Swiss German sentences taken from the resources presented in Section 3.2. We evaluated the approaches according to Labelled Attachment Score (LAS) and Unlabelled Attachment Score (UAS)7, not excluding punctuation. The results we present here are macro accuracy scores, that is, the scores are computed separately for each sentence and then averaged8. Note that there is a mismatch in the actual annotation of punctuation between the the Standard German UD treebank v2 and the official guidelines we were applying. This difference in the punctuation dependencies has an effect on the scores, i.e. it lowers the scores presented here. Furthermore, note that the test set containing 100 gold standard sentences is small and therefore these results have to be taken with a grain of salt. 5.1
In order to put the results into context, we checked the performance of the parsers on the German UD v2 treebank using their split of training and test set. In this setting, we left all the available information for the parser to use, including morphology and lemmas. The APPRART splitting is undone for the CoNLL-X MaltParser input, not so for the UDPipe which takes CoNLL-U as input format (and performs worse with the MaltParser-CoNLL-X input). MaltParser reaches a LAS of 79.71%, UDPipe 70.31% respectively. 5.2
As a comparison to the main approaches, we applied Standard German parsers directly to Swiss German.
7UAS is the percentage of tokens with the correct syntactic head, LAS the percentage of tokens assigned the correct syntactic head as well as the correct dependency label.
8Macro accuracy scores as opposed to the word-based micro scores, where the true positives are summed up over the whole treebank and divided by the total number of words.
This means, we used the training set of the German
UD treebank to train the MaltParser (using
MaltOptimizer to get the best hyperparameter settings) and
UDPipe. Before training, we removed the
morphology and lemma information because this information
is not available in the Swiss German test set and
therefore the parsers cannot rely on it. Furthermore, for
the MaltParser we converted the training set from
CoNLL-U to CoNLL-X format because
MaltOptimizer cannot handle the former. Testing the
MaltParser model on the gold standard with
automatically assigned POS tags by Wapiti results in an LAS
of 55.28%. UDPipe only reaches 21.19% LAS, one
reason for this low accuracy could be that UDPipe
relies on word embedding information
Instead of giving the parser the Standard German words as input like in the direct cross-lingual approach, in the delexicalised approach we provide the parser with POS information only. This means, the words are replaced by STTS POS tags while all the other columns stay the same. Given the small evaluation set and a negligible difference in the results, the two parsers’ performance can be considered the same: ∼57% LAS for both when trained on the preprocessed training set, i.e. differentiating the auxiliary werden vs. the auxiliaries haben (to have) and sein (to be) (see Section 4.3.1). 5.4
The results for the annotation projection approach vary substantially depending on the combination of aligner and parser. Starting from 46.45% LAS (MaltParser + Fastalign), the combination of UDPipe and Monolingual Greedy Aligner scores best in this approach with 53.39% LAS. This score is reached with the baseline transfer rules where unaligned words are simply attached to the root as adverbs. Applying more elaborate transfer rules (Section 4.2.1) results in an improvement of 2.09 points to 55.65% LAS. The preprocessing step does not improve the results in this approach. These results show that the Monolingual Greedy Aligner performs best in the task of DE/GSW alignment. MGA takes character-based word similarity into account which intuitively makes sense as the information about similar letters is valuable information when dealing with closely related languages such as Standard German and Swiss German. 5.5
The postprocessing rules do not show a huge impact on the parsing results; the nmod/obl confusions for example are still present. The reason for this is that the parser assigned wrong heads to many of the words and therefore the rule to correct the nmod/obl confusions does not work. The LAS scores improve by 1.62 points for the cross-lingual MaltParser and 2.07 points for delexicalised model transfer and annotation projection UDPipe approachs respectively, reaching nearly 60% LAS accuracy. 5.6
Table 1 shows the best results including the corresponding setting for every approach. The best LAS results of all the applied approaches are very close, hence there is no clear answer to the question of which approach works best. Annotation projection is the most laborious among the three and as such not the first option to choose. Furthermore, the transfer of the annotation is strongly dependent on the performance of the aligner, which in turn benefits from big parallel corpora to be trained on. However, such big parallel corpora do not exist yet for Swiss German dialects.
Contrary to our expectations, training specific models for different dialects does not have a huge impact on the results. The word ordering for the St Gallen dialect is closer to the Standard German word ordering while Bernese dialect speakers often change the order of the verbs. Due to these differences, we expected the model transfer approach to perform worse on the Bern dialect than annotation projection, where the word order changes should be handled by the aligner. Looking specifically at Bernese sentences with “switched” word order (e.g. ha aafo gra¨nne (’I started to cry’), gfunge hei gha (’have found’), het u¨bercho (’have gotten’)), there is no significant difference between the two approaches in our test set. 5.6.1
The results presented here are not perfect and certainly require further improvement in order for a system to be used in real-life applications. Compared with the Standard German parser accuracy, which reaches almost 80% LAS on the German UD v2 with standard settings of the parsers, there is room for improvement. However, these numbers have to be set in relation to the data we worked with. Even though we could make use of Swiss German novels and crowdsourced data, it is still a small data set. Furthermore, the enormous spelling variability in Swiss German dialects poses a serious challenge for all tools. Statistical tools work best if the observed events are frequent. However, they do not work well with sparse data consisting of a large amount of hapax legomena, i.e. word form which appears only once. Figure 4 shows the frequencies (on y-axis) of type frequencies (x) in a Swiss German text collection9 consisting of 6,155 sentences with 105,692 tokens and 20,882 unique token types. 14,099 types appear only once (i.e. hapax legomena), 2,804 appear twice (i.e. hapax dislegomena) 19,874 less than 10 times and 20,767 less than 100 times. 5.7
Following the direct cross-lingual parsing approach, we automatically parse 6,155 Swiss German sentences9 in order to create a silver treebank. A silver standard treebank, as opposed to a gold standard treebank which is assumed to be correctly annotated, is automatically annotated and may therefore contain errors. Then, we use this silver treebank to train a monolingual Swiss German parser and hence, create a first monolingual Swiss German dependency parsing model. The advantage of using a silver treebank is the fact that it becomes a monolingual task. However, this comes with the price of a faulty training set, which is not the best resource to build a parser.
9NOAH corpus plus 396 sentences from novels by Pedro Lenz and Renato Kaiser, excluding gold standard sentences.
Interestingly, the performance of the MaltParser trained on the silver treebank reaches the same performance as the direct cross-lingual parsing approach itself, which was used to generate the silver treebank: LAS 57.10%. Given that 6,000 sentences do not constitute a large training set for a statistical parser, a parser could probably profit from additional related Standard German material. However, combining the two training sets, i.e. the German Universal Dependency treebank and the silver treebank gives slightly worse results (LAS 55.46%). There are several opportunities for further improvement. Concerning the annotation projection approach, the crucial alignment information needs to be improved for example by ensembling over results from different word aligners. In cases where alignment does not work, adding further transfer and postprocessing rules would be important. In addition, a spelling normalisation strategy can help to deal with the data sparseness imposed by the phonetic and orthographic variability in Swiss German dialects. Moreover, the outputs of the three parsing approaches could be ensembled, e.g. via majority vote like for alignment as aforementioned, to get rid of the weaknesses of each approach. Furthermore, the silver treebank created could be manually corrected in order to generate a treebank which can be used as training set for a monolingual dependency parser for Swiss German. Finally, once the data sparseness for Swiss German varieties is mitigated, modern neural methods are promising as shown for example in the work by Ammar et al. (2016). Ammar et al. train one multilingual model that can be used to parse sentences in several languages. In order to do so, they use many resources including a bilingual dictionary for adding cross-lingual lexical information, and a monolingual corpus for training word embeddings. Such approaches need a big amount of data of the language to be parsed, which are still not available for Swiss German. 6
In this work, we experimented with a variety of crosslingual approaches for parsing texts written in Swiss German. For statistically driven systems, languages with non-standardised orthography are a demanding task. Swiss German dialects feature challenging Natural Language Processing (NLP) problems with their lack of orthographic spelling rules and a huge pronunciation variety. This is a situation which leads to a high degree of data sparseness and with it, a lack of resources and tools for NLP.
We tested a lexicalised annotation projection method as well as a delexicalised model transfer method. The annotation projection method requires parallel sentences in both the resource-rich and the low-resourced language while the delexicalised model transfer approach only requires a monolingual treebank of a closely related resource-rich language.
The evaluation on a manually annotated gold standard consisting of 100 sentences shows a 60% Labelled Attachment Score (LAS) with negligible differences between the different parsing approaches. However, the annotation projection approach is more complex than model transfer due to the transfer rules and the crucial word alignment process.
This work provides a first substantial step towards closing a big gap in Natural Language Processing tools for Swiss German and provides data10 to work on further improvements.
We thank the AGORA project “Citizen Linguistics” for making their translation data available and in particular all our volunteer translators. We also thank Renato Kaiser and Pedro Lenz for their permission to use their novels in our experiments.
10https://github.com/noe-eva/SwissGermanUD forschung/ressourcen/lexika/TagSets/ stts-1999.pdf.